Solr, mail # user - Re: ExtendedDisMax Question - Strange behaviour

ExtendedDisMax Question - Strange behaviour

André Maldonado 2012-06-04, 17:26

I'm doing a query with edismax.

When I don't tell solr which fields I want to do the search (so it does in
default field), it returns 2752 documents.

ex:
http://000.000.0.0:0000/solr/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro>

The same search, defining the fiels that composes the default field, it
returns 1434 docs.

ex:
http://000.000.0.0:0000/solr/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro&qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual>
qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro&qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual>

This is the important part of schema:

<defaultSearchField>textoboost</defaultSearchField><copyField source="
agrupamentos2" dest="textoboost" /><copyField source="agrupamentos" dest="
textoboost" /><copyField source="bairro" dest="textoboost" /><copyField
source="campanhalocalempreendimento" dest="textoboost" /><copyField source="
caracteristicas" dest="textoboost" /><copyField source="caracteristicacomum"
dest="textoboost" /><copyField source="categoria" dest="textoboost" /><
copyField source="cep" dest="textoboost" /><copyField source="chamada" dest
="textoboost" /><copyField source="cidade" dest="textoboost" /><copyField
source="codigoanuncio" dest="textoboost" /><copyField source="complemento"
dest="textoboost" /><copyField source="descricaopermuta" dest="textoboost"
/><copyField source="docid" dest="textoboost" /><copyField source="
empreendimento" dest="textoboost" /><copyField source="endereco" dest="
textoboost" /><copyField source="estado" dest="textoboost" /><copyField
source="informacoescomplementares" dest="textoboost" /><copyField source="
conteudoobservacao" dest="textoboost" /><copyField source="sigla" dest="
textoboost" /><copyField source="subtipoimovel" dest="textoboost" /><
copyField source="tipoimovel" dest="textoboost" /><copyField source="
transacao" dest="textoboost" /><copyField source="zapid" dest="textoboost"
/><copyField source="caminhomapa" dest="textoboost" /><copyField source="
codigooferta" dest="textoboost" /><copyField source="segmento" dest="
textoboost" /><copyField source="anuncianteorigem" dest="textoboost" /><
copyField source="zapidcorporativo" dest="textoboost" /><copyField source="
estagiodaobra" dest="textoboost" /><copyField source="condicoescomerciais"
dest="textoboost" /><copyField source="nomejornal" dest="textoboost" /><
copyField source="nomejornalordem" dest="textoboost" /> <copyField source="
textomanual" dest="textoboost" />

What's the problem?

Thank's

*
*
*"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*

*andre.maldonado*@gmail.com <[EMAIL PROTECTED]>
(11) 9112-4227

<http://www.orkut.com.br/Main#Profile?uid=2397703412199036664>
<http://www.orkut.com.br/Main#Profile?uid=2397703412199036664>
<http://www.facebook.com/profile.php?id=100000659376883>
  <http://twitter.com/andremaldonado> <http://www.delicious.com/andre.maldonado>
  <https://profiles.google.com/105605760943701739931>
<http://www.linkedin.com/pub/andr%C3%A9-maldonado/23/234/4b3>
  <http://www.youtube.com/andremaldonado>

Re: ExtendedDisMax Question - Strange behaviour

Erick Erickson 2012-06-06, 10:59

Sorry, but your post is really hard to read with all the data inline.

Try running with &debugQuery=on and looking at the parsed query, I suspect
your field lists aren't the same even though you think they are.
Perhaps a typo somewhere?

Best
Erick

On Mon, Jun 4, 2012 at 1:26 PM, André Maldonado
<[EMAIL PROTECTED]> wrote:
> I'm doing a query with edismax.
>
> When I don't tell solr which fields I want to do the search (so it does in
> default field), it returns 2752 documents.
>
> ex:
> http://000.000.0.0:0000/solr/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro>
>
> The same search, defining the fiels that composes the default field, it
> returns 1434 docs.
>
> ex:
> http://000.000.0.0:0000/solr/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro&qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual>
> qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual<http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueira&version=2.2&start=0&rows=10&indent=on&defType=dismax&mm=75%25&facet=true&facet.field=bairro&qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual>
>
> This is the important part of schema:
>
> <defaultSearchField>textoboost</defaultSearchField><copyField source="
> agrupamentos2" dest="textoboost" /><copyField source="agrupamentos" dest="
> textoboost" /><copyField source="bairro" dest="textoboost" /><copyField
> source="campanhalocalempreendimento" dest="textoboost" /><copyField source="
> caracteristicas" dest="textoboost" /><copyField source="caracteristicacomum"
> dest="textoboost" /><copyField source="categoria" dest="textoboost" /><
> copyField source="cep" dest="textoboost" /><copyField source="chamada" dest
> ="textoboost" /><copyField source="cidade" dest="textoboost" /><copyField
> source="codigoanuncio" dest="textoboost" /><copyField source="complemento"
> dest="textoboost" /><copyField source="descricaopermuta" dest="textoboost"
> /><copyField source="docid" dest="textoboost" /><copyField source="
> empreendimento" dest="textoboost" /><copyField source="endereco" dest="
> textoboost" /><copyField source="estado" dest="textoboost" /><copyField
> source="informacoescomplementares" dest="textoboost" /><copyField source="
> conteudoobservacao" dest="textoboost" /><copyField source="sigla" dest="
> textoboost" /><copyField source="subtipoimovel" dest="textoboost" /><

Re: ExtendedDisMax Question - Strange behaviour

André Maldonado 2012-06-06, 13:23

Re: ExtendedDisMax Question - Strange behaviour

Jack Krupansky 2012-06-06, 20:50

First, it appears that you are using the "dismax" query parser, not the
extended dismax ("edismax") query parser.

My hunch is that some of those fields may be non-tokenized "string" fields
in which one or more of your search keywords do appear but not as the full
string value or maybe with a different case than in the query. But when you
do a copyField from a string field to a tokenized "text" field those strings
would be broken up into individual keywords and probably lowercased. So, it
will be easier for a document to match the combined "text" field than the
source "string" fields. A fair percentage of the terms may occur in both
"text" and "string" fields, but it looks like a fair percentage may occur
only in the string fields.

Identify a specific document that is returned by the first query and not the
second. Then examine each non-text "string" field value of that document to
see if the query terms would match after text field analysis but are not
exact string matches for the string fields in which the terms do occur.

-- Jack Krupansky
-----Original Message-----
From: Andrï¿½ Maldonado
Sent: Wednesday, June 06, 2012 9:23 AM
To: [EMAIL PROTECTED]
Subject: Re: ExtendedDisMax Question - Strange behaviour

Erick, thanks for your reply and sorry for the confusion in last e-mail.
But it is hard to explain the situation without that bunch of code.
...

Re: ExtendedDisMax Question - Strange behaviour

André Maldonado 2012-06-08, 11:50

Thank's Jack. It is exactly this. My mistake.

Thank's
*
----------------------------------------------------------------------------------------------
*
*"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*

*andre.maldonado*@gmail.com <[EMAIL PROTECTED]>
(11) 9112-4227

<http://www.orkut.com.br/Main#Profile?uid=2397703412199036664>
<http://www.orkut.com.br/Main#Profile?uid=2397703412199036664>
<http://www.facebook.com/profile.php?id=100000659376883>
  <http://twitter.com/andremaldonado> <http://www.delicious.com/andre.maldonado>
  <https://profiles.google.com/105605760943701739931>
<http://www.linkedin.com/pub/andr%C3%A9-maldonado/23/234/4b3>
  <http://www.youtube.com/andremaldonado>
On Wed, Jun 6, 2012 at 5:50 PM, Jack Krupansky <[EMAIL PROTECTED]>wrote:

> First, it appears that you are using the "dismax" query parser, not the
> extended dismax ("edismax") query parser.
>
> My hunch is that some of those fields may be non-tokenized "string" fields
> in which one or more of your search keywords do appear but not as the full
> string value or maybe with a different case than in the query. But when you
> do a copyField from a string field to a tokenized "text" field those
> strings
> would be broken up into individual keywords and probably lowercased. So, it
> will be easier for a document to match the combined "text" field than the
> source "string" fields. A fair percentage of the terms may occur in both
> "text" and "string" fields, but it looks like a fair percentage may occur
> only in the string fields.
>
> Identify a specific document that is returned by the first query and not
> the
> second. Then examine each non-text "string" field value of that document to
> see if the query terms would match after text field analysis but are not
> exact string matches for the string fields in which the terms do occur.
>
> -- Jack Krupansky
> -----Original Message----- From: André Maldonado
> Sent: Wednesday, June 06, 2012 9:23 AM
> To: [EMAIL PROTECTED]
> Subject: Re: ExtendedDisMax Question - Strange behaviour
>
>
> Erick, thanks for your reply and sorry for the confusion in last e-mail.
> But it is hard to explain the situation without that bunch of code.
> ...
>
>