Hi

My index have fields named ad_title, ad_description & ad_post_date. Let's
suppose a user searches for more than one keyword, then i want the documents
with maximum occurence of all the keywords together should come on top. The
more closer the keywords in ad_title & ad_description should be given top
priority.

Also, i want that these results should be sorted on ad_post_date.

Please suggest!!!

--
Thanks,
Pawan Darira

Those are at least 3 different questions. Easiest first, sorting.
   add    &sort=ad_post_date+desc   (or asc)  for sorting on date,
descending or ascending

check out how   http://www.supermind.org/blog/378/lucene-scoring-for-dummies
Lucene  scores by default. It might close to what you want. The only thing
it isn't doing that you are looking for is the relative distance between
keywords in a document.

You can add a boost to the ad_title and ad_description fields to make them
more important to your search.

My guess is, although I haven't done this myself, the default Scoring
algorithm can be augmented or replaced with your own. That may be a route to
take if you are comfortable with java.
--
View this message in context: http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-tp1516587p1516691.html
Sent from the Solr - User mailing list archive at Nabble.com.

What is it about the standard relevance ranking that doesn't suit your
needs?

And note that if you sort by your date field, relevance doesn't matter at
all
because the date sort overrides all the scoring, by definition.

Best
Erick

On Fri, Sep 17, 2010 at 6:57 AM, Pawan Darira <[EMAIL PROTECTED]>wrote:

> Hi
>
> My index have fields named ad_title, ad_description & ad_post_date. Let's
> suppose a user searches for more than one keyword, then i want the
> documents
> with maximum occurence of all the keywords together should come on top. The
> more closer the keywords in ad_title & ad_description should be given top
> priority.
>
> Also, i want that these results should be sorted on ad_post_date.
>
> Please suggest!!!
>
> --
> Thanks,
> Pawan Darira
>

I'm a total Lucene/SOLR newbie, and I'm surprised to see that when there are
multiple search terms, term proximity isn't part of the scoring process. Has
anyone on the list done custom scoring that weights proximity?

Andy Cogan

-----Original Message-----
From: kenf_nc [mailto:[EMAIL PROTECTED]]
Sent: Friday, September 17, 2010 7:06 AM
To: [EMAIL PROTECTED]
Subject: Re: Can i do relavence and sorting together?
Those are at least 3 different questions. Easiest first, sorting.
   add    &sort=ad_post_date+desc   (or asc)  for sorting on date,
descending or ascending

check out how   http://www.supermind.org/blog/378/lucene-scoring-for-dummies
Lucene  scores by default. It might close to what you want. The only thing
it isn't doing that you are looking for is the relative distance between
keywords in a document.

You can add a boost to the ad_title and ad_description fields to make them
more important to your search.

My guess is, although I haven't done this myself, the default Scoring
algorithm can be augmented or replaced with your own. That may be a route to
take if you are comfortable with java.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-t
p1516587p1516691.html
Sent from the Solr - User mailing list archive at Nabble.com.

The problem, and it's a practical one, is that terms usually have to be
pretty
close to each other for proximity to matter, and you can get this with
phrase queries by varying the slop.

FWIW
Erick

On Fri, Sep 17, 2010 at 11:05 AM, Andrew Cogan
<[EMAIL PROTECTED]>wrote:

> I'm a total Lucene/SOLR newbie, and I'm surprised to see that when there
> are
> multiple search terms, term proximity isn't part of the scoring process.
> Has
> anyone on the list done custom scoring that weights proximity?
>
> Andy Cogan
>
> -----Original Message-----
> From: kenf_nc [mailto:[EMAIL PROTECTED]]
> Sent: Friday, September 17, 2010 7:06 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Can i do relavence and sorting together?
>
>
> Those are at least 3 different questions. Easiest first, sorting.
>   add    &sort=ad_post_date+desc   (or asc)  for sorting on date,
> descending or ascending
>
> check out how
> http://www.supermind.org/blog/378/lucene-scoring-for-dummies
> Lucene  scores by default. It might close to what you want. The only thing
> it isn't doing that you are looking for is the relative distance between
> keywords in a document.
>
> You can add a boost to the ad_title and ad_description fields to make them
> more important to your search.
>
> My guess is, although I haven't done this myself, the default Scoring
> algorithm can be augmented or replaced with your own. That may be a route
> to
> take if you are comfortable with java.
> --
> View this message in context:
>
> http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-t
> p1516587p1516691.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>

Well ..
> because the date sort overrides all the scoring, by
> definition.

THAT'S not good for what I want, LOL!

Is there any way to chain things like distance, date, relevancy, an integer field to force sort oder, like when using SQL 'SORT BY', the order of sort is the order of listing?
Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:

> From: Erick Erickson <[EMAIL PROTECTED]>
> Subject: Re: Can i do relavence and sorting together?
> To: [EMAIL PROTECTED]
> Date: Friday, September 17, 2010, 6:10 AM
> What is it about the standard
> relevance ranking that doesn't suit your
> needs?
>
> And note that if you sort by your date field, relevance
> doesn't matter at
> all
> because the date sort overrides all the scoring, by
> definition.
>
> Best
> Erick
>
> On Fri, Sep 17, 2010 at 6:57 AM, Pawan Darira <[EMAIL PROTECTED]>wrote:
>
> > Hi
> >
> > My index have fields named ad_title, ad_description
> & ad_post_date. Let's
> > suppose a user searches for more than one keyword,
> then i want the
> > documents
> > with maximum occurence of all the keywords together
> should come on top. The
> > more closer the keywords in ad_title &
> ad_description should be given top
> > priority.
> >
> > Also, i want that these results should be sorted on
> ad_post_date.
> >
> > Please suggest!!!
> >
> > --
> > Thanks,
> > Pawan Darira
> >
>

Sure, you can specify multiple sort fields. If the first sort field results
in a tie, then
the second is used to resolve. If both first and second match, then the
third is
used to break the tie.

Note that relevancy is tricky to include in the chain because it's
infrequent to have two
docs with exactly the same relevancy scores, so wherever relevancy is in the
chain,
sort criteria below that probably will have very little effect.

You could probably write some custom code to munge the relevancy scores into
buckets,
say quintiles, but that'd be somewhat tricky.

What is the use case for your sorting?

Best
Erick

On Fri, Sep 17, 2010 at 1:00 PM, Dennis Gearon <[EMAIL PROTECTED]>wrote:

> Well ..
> > because the date sort overrides all the scoring, by
> > definition.
>
> THAT'S not good for what I want, LOL!
>
> Is there any way to chain things like distance, date, relevancy, an integer
> field to force sort oder, like when using SQL 'SORT BY', the order of sort
> is the order of listing?
>
>
> Dennis Gearon
>
> Signature Warning
> ----------------
> EARTH has a Right To Life,
>  otherwise we all die.
>
> Read 'Hot, Flat, and Crowded'
> Laugh at http://www.yert.com/film.php
>
>
> --- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:
>
> > From: Erick Erickson <[EMAIL PROTECTED]>
> > Subject: Re: Can i do relavence and sorting together?
> > To: [EMAIL PROTECTED]
> > Date: Friday, September 17, 2010, 6:10 AM
> > What is it about the standard
> > relevance ranking that doesn't suit your
> > needs?
> >
> > And note that if you sort by your date field, relevance
> > doesn't matter at
> > all
> > because the date sort overrides all the scoring, by
> > definition.
> >
> > Best
> > Erick
> >
> > On Fri, Sep 17, 2010 at 6:57 AM, Pawan Darira <[EMAIL PROTECTED]
> >wrote:
> >
> > > Hi
> > >
> > > My index have fields named ad_title, ad_description
> > & ad_post_date. Let's
> > > suppose a user searches for more than one keyword,
> > then i want the
> > > documents
> > > with maximum occurence of all the keywords together
> > should come on top. The
> > > more closer the keywords in ad_title &
> > ad_description should be given top
> > > priority.
> > >
> > > Also, i want that these results should be sorted on
> > ad_post_date.
> > >
> > > Please suggest!!!
> > >
> > > --
> > > Thanks,
> > > Pawan Darira
> > >
> >
>

Yes. Just as you'd expect:

&sort=score asc,date desc,title asc      [url encoded of course]

The only trick is knowing the special key 'score' for sorting by relevancy. This is all in the wiki docs:  
http://wiki.apache.org/solr/CommonQueryParameters#sort

Also keep in mind, as the docs say, sorting only works properly on non-tokenized single-value fields, which makes sense if you think about it.
________________________________________
From: Dennis Gearon [[EMAIL PROTECTED]]
Sent: Friday, September 17, 2010 1:00 PM
To: [EMAIL PROTECTED]
Subject: Re: Can i do relavence and sorting together?

Well ..
> because the date sort overrides all the scoring, by
> definition.

THAT'S not good for what I want, LOL!

Is there any way to chain things like distance, date, relevancy, an integer field to force sort oder, like when using SQL 'SORT BY', the order of sort is the order of listing?
Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:

> From: Erick Erickson <[EMAIL PROTECTED]>
> Subject: Re: Can i do relavence and sorting together?
> To: [EMAIL PROTECTED]
> Date: Friday, September 17, 2010, 6:10 AM
> What is it about the standard
> relevance ranking that doesn't suit your
> needs?
>
> And note that if you sort by your date field, relevance
> doesn't matter at
> all
> because the date sort overrides all the scoring, by
> definition.
>
> Best
> Erick
>
> On Fri, Sep 17, 2010 at 6:57 AM, Pawan Darira <[EMAIL PROTECTED]>wrote:
>
> > Hi
> >
> > My index have fields named ad_title, ad_description
> & ad_post_date. Let's
> > suppose a user searches for more than one keyword,
> then i want the
> > documents
> > with maximum occurence of all the keywords together
> should come on top. The
> > more closer the keywords in ad_title &
> ad_description should be given top
> > priority.
> >
> > Also, i want that these results should be sorted on
> ad_post_date.
> >
> > Please suggest!!!
> >
> > --
> > Thanks,
> > Pawan Darira
> >
>

On Sep 17, 2010, at 10:00 AM, Dennis Gearon wrote:

> Well ..
>> because the date sort overrides all the scoring, by
>> definition.
>
> THAT'S not good for what I want, LOL!
>
> Is there any way to chain things like distance, date, relevancy, an integer field to force sort oder, like when using SQL 'SORT BY', the order of sort is the order of listing?

Boost functions, or function queries, may also be what you're looking for:

http://wiki.apache.org/solr/FunctionQuery

http://stackoverflow.com/questions/1486963/solr-boost-function-bf-to-increase-score-of-documents-whose-date-is-closest-t

The users will be able to choose the order of sort based on distance, data and time, relevancy.

More than likely, my first initial version will do range limits on distance, data and time. Then relevancy will sort, send it to browser.

After that, the user will sort it in the browser as desired.

I can't yet get into the application, but early next year I can. In fact, I most certainly will :-)

Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:

> From: Erick Erickson <[EMAIL PROTECTED]>
> Subject: Re: Can i do relavence and sorting together?
> To: [EMAIL PROTECTED]
> Date: Friday, September 17, 2010, 10:09 AM
> Sure, you can specify multiple sort
> fields. If the first sort field results
> in a tie, then
> the second is used to resolve. If both first and second
> match, then the
> third is
> used to break the tie.
>
> Note that relevancy is tricky to include in the chain
> because it's
> infrequent to have two
> docs with exactly the same relevancy scores, so wherever
> relevancy is in the
> chain,
> sort criteria below that probably will have very little
> effect.
>
> You could probably write some custom code to munge the
> relevancy scores into
> buckets,
> say quintiles, but that'd be somewhat tricky.
>
> What is the use case for your sorting?
>
> Best
> Erick
>
> On Fri, Sep 17, 2010 at 1:00 PM, Dennis Gearon <[EMAIL PROTECTED]>wrote:
>
> > Well ..
> > > because the date sort overrides all the scoring,
> by
> > > definition.
> >
> > THAT'S not good for what I want, LOL!
> >
> > Is there any way to chain things like distance, date,
> relevancy, an integer
> > field to force sort oder, like when using SQL 'SORT
> BY', the order of sort
> > is the order of listing?
> >
> >
> > Dennis Gearon
> >
> > Signature Warning
> > ----------------
> > EARTH has a Right To Life,
> >  otherwise we all die.
> >
> > Read 'Hot, Flat, and Crowded'
> > Laugh at http://www.yert.com/film.php
> >
> >
> > --- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]>
> wrote:
> >
> > > From: Erick Erickson <[EMAIL PROTECTED]>
> > > Subject: Re: Can i do relavence and sorting
> together?
> > > To: [EMAIL PROTECTED]
> > > Date: Friday, September 17, 2010, 6:10 AM
> > > What is it about the standard
> > > relevance ranking that doesn't suit your
> > > needs?
> > >
> > > And note that if you sort by your date field,
> relevance
> > > doesn't matter at
> > > all
> > > because the date sort overrides all the scoring,
> by
> > > definition.
> > >
> > > Best
> > > Erick
> > >
> > > On Fri, Sep 17, 2010 at 6:57 AM, Pawan Darira
> <[EMAIL PROTECTED]
> > >wrote:
> > >
> > > > Hi
> > > >
> > > > My index have fields named ad_title,
> ad_description
> > > & ad_post_date. Let's
> > > > suppose a user searches for more than one
> keyword,
> > > then i want the
> > > > documents
> > > > with maximum occurence of all the keywords
> together
> > > should come on top. The
> > > > more closer the keywords in ad_title &
> > > ad_description should be given top
> > > > priority.
> > > >
> > > > Also, i want that these results should be
> sorted on
> > > ad_post_date.
> > > >
> > > > Please suggest!!!
> > > >
> > > > --
> > > > Thanks,
> > > > Pawan Darira
> > > >
> > >
> >
>

HOw does one 'vary the slop'?

Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:

> From: Erick Erickson <[EMAIL PROTECTED]>
> Subject: Re: Can i do relavence and sorting together?
> To: [EMAIL PROTECTED]
> Date: Friday, September 17, 2010, 8:58 AM
> The problem, and it's a practical
> one, is that terms usually have to be
> pretty
> close to each other for proximity to matter, and you can
> get this with
> phrase queries by varying the slop.
>
> FWIW
> Erick
>
> On Fri, Sep 17, 2010 at 11:05 AM, Andrew Cogan
> <[EMAIL PROTECTED]>wrote:
>
> > I'm a total Lucene/SOLR newbie, and I'm surprised to
> see that when there
> > are
> > multiple search terms, term proximity isn't part of
> the scoring process.
> > Has
> > anyone on the list done custom scoring that weights
> proximity?
> >
> > Andy Cogan
> >
> > -----Original Message-----
> > From: kenf_nc [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, September 17, 2010 7:06 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Can i do relavence and sorting together?
> >
> >
> > Those are at least 3 different questions. Easiest
> first, sorting.
> >   add   
> &sort=ad_post_date+desc   (or asc) 
> for sorting on date,
> > descending or ascending
> >
> > check out how
> > http://www.supermind.org/blog/378/lucene-scoring-for-dummies
> > Lucene  scores by default. It might close to what
> you want. The only thing
> > it isn't doing that you are looking for is the
> relative distance between
> > keywords in a document.
> >
> > You can add a boost to the ad_title and ad_description
> fields to make them
> > more important to your search.
> >
> > My guess is, although I haven't done this myself, the
> default Scoring
> > algorithm can be augmented or replaced with your own.
> That may be a route
> > to
> > take if you are comfortable with java.
> > --
> > View this message in context:
> >
> > http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-t
> > p1516587p1516691.html
> > Sent from the Solr - User mailing list archive at
> Nabble.com.
> >
> >
>

http://wiki.apache.org/solr/CommonQueryParameters?action=fullsearch&context=180&value=slop&fullsearch=Text

On Fri, Sep 17, 2010 at 10:55 AM, Dennis Gearon <[EMAIL PROTECTED]> wrote:
> HOw does one 'vary the slop'?
>
> Dennis Gearon
>
> Signature Warning
> ----------------
> EARTH has a Right To Life,
>  otherwise we all die.
>
> Read 'Hot, Flat, and Crowded'
> Laugh at http://www.yert.com/film.php
>
>
> --- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]> wrote:
>
>> From: Erick Erickson <[EMAIL PROTECTED]>
>> Subject: Re: Can i do relavence and sorting together?
>> To: [EMAIL PROTECTED]
>> Date: Friday, September 17, 2010, 8:58 AM
>> The problem, and it's a practical
>> one, is that terms usually have to be
>> pretty
>> close to each other for proximity to matter, and you can
>> get this with
>> phrase queries by varying the slop.
>>
>> FWIW
>> Erick
>>
>> On Fri, Sep 17, 2010 at 11:05 AM, Andrew Cogan
>> <[EMAIL PROTECTED]>wrote:
>>
>> > I'm a total Lucene/SOLR newbie, and I'm surprised to
>> see that when there
>> > are
>> > multiple search terms, term proximity isn't part of
>> the scoring process.
>> > Has
>> > anyone on the list done custom scoring that weights
>> proximity?
>> >
>> > Andy Cogan
>> >
>> > -----Original Message-----
>> > From: kenf_nc [mailto:[EMAIL PROTECTED]]
>> > Sent: Friday, September 17, 2010 7:06 AM
>> > To: [EMAIL PROTECTED]
>> > Subject: Re: Can i do relavence and sorting together?
>> >
>> >
>> > Those are at least 3 different questions. Easiest
>> first, sorting.
>> >   add
>> &sort=ad_post_date+desc   (or asc)
>> for sorting on date,
>> > descending or ascending
>> >
>> > check out how
>> > http://www.supermind.org/blog/378/lucene-scoring-for-dummies
>> > Lucene  scores by default. It might close to what
>> you want. The only thing
>> > it isn't doing that you are looking for is the
>> relative distance between
>> > keywords in a document.
>> >
>> > You can add a boost to the ad_title and ad_description
>> fields to make them
>> > more important to your search.
>> >
>> > My guess is, although I haven't done this myself, the
>> default Scoring
>> > algorithm can be augmented or replaced with your own.
>> That may be a route
>> > to
>> > take if you are comfortable with java.
>> > --
>> > View this message in context:
>> >
>> > http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-t
>> > p1516587p1516691.html
>> > Sent from the Solr - User mailing list archive at
>> Nabble.com.
>> >
>> >
>>
>

--
Lance Norskog
[EMAIL PROTECTED]

'slop' is an actual argument!?!? LOL!

I thought you were just describing some ASPECT of the search process, not it's workings :-)
Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Fri, 9/17/10, Lance Norskog <[EMAIL PROTECTED]> wrote:

> From: Lance Norskog <[EMAIL PROTECTED]>
> Subject: Re: Can i do relavence and sorting together?
> To: [EMAIL PROTECTED]
> Date: Friday, September 17, 2010, 4:57 PM
> http://wiki.apache.org/solr/CommonQueryParameters?action=fullsearch&context=180&value=slop&fullsearch=Text
>
> On Fri, Sep 17, 2010 at 10:55 AM, Dennis Gearon <[EMAIL PROTECTED]>
> wrote:
> > HOw does one 'vary the slop'?
> >
> > Dennis Gearon
> >
> > Signature Warning
> > ----------------
> > EARTH has a Right To Life,
> >  otherwise we all die.
> >
> > Read 'Hot, Flat, and Crowded'
> > Laugh at http://www.yert.com/film.php
> >
> >
> > --- On Fri, 9/17/10, Erick Erickson <[EMAIL PROTECTED]>
> wrote:
> >
> >> From: Erick Erickson <[EMAIL PROTECTED]>
> >> Subject: Re: Can i do relavence and sorting
> together?
> >> To: [EMAIL PROTECTED]
> >> Date: Friday, September 17, 2010, 8:58 AM
> >> The problem, and it's a practical
> >> one, is that terms usually have to be
> >> pretty
> >> close to each other for proximity to matter, and
> you can
> >> get this with
> >> phrase queries by varying the slop.
> >>
> >> FWIW
> >> Erick
> >>
> >> On Fri, Sep 17, 2010 at 11:05 AM, Andrew Cogan
> >> <[EMAIL PROTECTED]>wrote:
> >>
> >> > I'm a total Lucene/SOLR newbie, and I'm
> surprised to
> >> see that when there
> >> > are
> >> > multiple search terms, term proximity isn't
> part of
> >> the scoring process.
> >> > Has
> >> > anyone on the list done custom scoring that
> weights
> >> proximity?
> >> >
> >> > Andy Cogan
> >> >
> >> > -----Original Message-----
> >> > From: kenf_nc [mailto:[EMAIL PROTECTED]]
> >> > Sent: Friday, September 17, 2010 7:06 AM
> >> > To: [EMAIL PROTECTED]
> >> > Subject: Re: Can i do relavence and sorting
> together?
> >> >
> >> >
> >> > Those are at least 3 different questions.
> Easiest
> >> first, sorting.
> >> >   add
> >> &sort=ad_post_date+desc   (or asc)
> >> for sorting on date,
> >> > descending or ascending
> >> >
> >> > check out how
> >> > http://www.supermind.org/blog/378/lucene-scoring-for-dummies
> >> > Lucene  scores by default. It might close to
> what
> >> you want. The only thing
> >> > it isn't doing that you are looking for is
> the
> >> relative distance between
> >> > keywords in a document.
> >> >
> >> > You can add a boost to the ad_title and
> ad_description
> >> fields to make them
> >> > more important to your search.
> >> >
> >> > My guess is, although I haven't done this
> myself, the
> >> default Scoring
> >> > algorithm can be augmented or replaced with
> your own.
> >> That may be a route
> >> > to
> >> > take if you are comfortable with java.
> >> > --
> >> > View this message in context:
> >> >
> >> > http://lucene.472066.n3.nabble.com/Can-i-do-relavence-and-sorting-together-t
> >> > p1516587p1516691.html
> >> > Sent from the Solr - User mailing list
> archive at
> >> Nabble.com.
> >> >
> >> >
> >>
> >
>
>
>
> --
> Lance Norskog
> [EMAIL PROTECTED]
>