The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
reasons it is recommended to open only one IndexSearcher and use it for all
of your searches."

However, to use NRT, it seems I have to do this for every search, which
contradicts the advice above:
    IndexSearcher myIndexSearcher = new
IndexSearcher(myIndexWriter.getReader());

Is there any way to take advantage of NRT and not run into these performance
problems under heavy load?

Is the advice from the javadoc above aimed more at
IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it also
aimed at  IndexSearcher(org.apache.lucene.index.IndexReader indexReader),
which I believe I have to use to get NRT (correct me if I am wrong)?
--
View this message in context: http://old.nabble.com/NRT-and-IndexSearcher-performance-tp27235434p27235434.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------

J,

The javadocs are illustrating there's no need to create new
IndexSearchers for each query.

Jason

On Tue, Jan 19, 2010 at 5:04 PM, jchang <[EMAIL PROTECTED]> wrote:
>
> The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
> reasons it is recommended to open only one IndexSearcher and use it for all
> of your searches."
>
> However, to use NRT, it seems I have to do this for every search, which
> contradicts the advice above:
>    IndexSearcher myIndexSearcher = new
> IndexSearcher(myIndexWriter.getReader());
>
> Is there any way to take advantage of NRT and not run into these performance
> problems under heavy load?
>
> Is the advice from the javadoc above aimed more at
> IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it also
> aimed at  IndexSearcher(org.apache.lucene.index.IndexReader indexReader),
> which I believe I have to use to get NRT (correct me if I am wrong)?
> --
> View this message in context: http://old.nabble.com/NRT-and-IndexSearcher-performance-tp27235434p27235434.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

---------------------------------------------------------------------

I think the question here really is the cost of creating new IndexReader
instances per query.

Calling IndexWriter.getReader() for each query has shown to be expensive
from our benchmark and previous discussions.

-John

On Tue, Jan 19, 2010 at 8:12 PM, Jason Rutherglen <
[EMAIL PROTECTED]> wrote:

> J,
>
> The javadocs are illustrating there's no need to create new
> IndexSearchers for each query.
>
> Jason
>
> On Tue, Jan 19, 2010 at 5:04 PM, jchang <[EMAIL PROTECTED]> wrote:
> >
> > The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
> > reasons it is recommended to open only one IndexSearcher and use it for
> all
> > of your searches."
> >
> > However, to use NRT, it seems I have to do this for every search, which
> > contradicts the advice above:
> >    IndexSearcher myIndexSearcher = new
> > IndexSearcher(myIndexWriter.getReader());
> >
> > Is there any way to take advantage of NRT and not run into these
> performance
> > problems under heavy load?
> >
> > Is the advice from the javadoc above aimed more at
> > IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it
> also
> > aimed at  IndexSearcher(org.apache.lucene.index.IndexReader indexReader),
> > which I believe I have to use to get NRT (correct me if I am wrong)?
> > --
> > View this message in context:
> http://old.nabble.com/NRT-and-IndexSearcher-performance-tp27235434p27235434.html
> > Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

Separate to this discussion:
We should fix the JavaDocs of IndexSearcher, that it should explain, that only the instances using Directory are costly. Creating a new IndexSearcher around an IndexReader is a neglectible cost, the important info is: "Do not recreate the IR for each search". Wrapping an IndexSearcher around an IndexReader is simply creating a new Java instance, nothing more.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]

> -----Original Message-----
> From: jchang [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, January 20, 2010 2:05 AM
> To: [EMAIL PROTECTED]
> Subject: NRT and IndexSearcher performance
>
>
> The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
> reasons it is recommended to open only one IndexSearcher and use it for
> all
> of your searches."
>
> However, to use NRT, it seems I have to do this for every search, which
> contradicts the advice above:
>     IndexSearcher myIndexSearcher = new
> IndexSearcher(myIndexWriter.getReader());
>
> Is there any way to take advantage of NRT and not run into these
> performance
> problems under heavy load?
>
> Is the advice from the javadoc above aimed more at
> IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it
> also
> aimed at  IndexSearcher(org.apache.lucene.index.IndexReader
> indexReader),
> which I believe I have to use to get NRT (correct me if I am wrong)?
> --
> View this message in context: http://old.nabble.com/NRT-and-
> IndexSearcher-performance-tp27235434p27235434.html
> Sent from the Lucene - Java Developer mailing list archive at
> Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]

---------------------------------------------------------------------

On Tue, Jan 19, 2010 at 8:04 PM, jchang <[EMAIL PROTECTED]> wrote:
>
> The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
> reasons it is recommended to open only one IndexSearcher and use it for all
> of your searches."

As Uwe pointed out, creating an IndexSearcher from an already created
IndexReader is very cheap.

> However, to use NRT, it seems I have to do this for every search, which
> contradicts the advice above:
>    IndexSearcher myIndexSearcher = new
> IndexSearcher(myIndexWriter.getReader());

You should test for yourself (and please report back!), but... if you
have a large index, and you are actively changing the index, and those
changes include any deletions (or calls to IW.updateDocument),
creating a new NRT reader for every search will likely be too costly
at high search load.

> Is there any way to take advantage of NRT and not run into these performance
> problems under heavy load?

It's usually best to throttle how often a new NRT reader is created to
something like 10X per second (but test yourself to see...).

> Is the advice from the javadoc above aimed more at
> IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it also
> aimed at  IndexSearcher(org.apache.lucene.index.IndexReader indexReader),
> which I believe I have to use to get NRT (correct me if I am wrong)?

The former.

Mike

---------------------------------------------------------------------

+1

Mike

On Wed, Jan 20, 2010 at 4:04 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Separate to this discussion:
> We should fix the JavaDocs of IndexSearcher, that it should explain, that only the instances using Directory are costly. Creating a new IndexSearcher around an IndexReader is a neglectible cost, the important info is: "Do not recreate the IR for each search". Wrapping an IndexSearcher around an IndexReader is simply creating a new Java instance, nothing more.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>> -----Original Message-----
>> From: jchang [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, January 20, 2010 2:05 AM
>> To: [EMAIL PROTECTED]
>> Subject: NRT and IndexSearcher performance
>>
>>
>> The javadocs for IndexSearcher in Lucene 3.0.0 read:  "For performance
>> reasons it is recommended to open only one IndexSearcher and use it for
>> all
>> of your searches."
>>
>> However, to use NRT, it seems I have to do this for every search, which
>> contradicts the advice above:
>>     IndexSearcher myIndexSearcher = new
>> IndexSearcher(myIndexWriter.getReader());
>>
>> Is there any way to take advantage of NRT and not run into these
>> performance
>> problems under heavy load?
>>
>> Is the advice from the javadoc above aimed more at
>> IndexSearcher(org.apache.lucene.store.Directory directory)?  Or is it
>> also
>> aimed at  IndexSearcher(org.apache.lucene.index.IndexReader
>> indexReader),
>> which I believe I have to use to get NRT (correct me if I am wrong)?
>> --
>> View this message in context: http://old.nabble.com/NRT-and-
>> IndexSearcher-performance-tp27235434p27235434.html
>> Sent from the Lucene - Java Developer mailing list archive at
>> Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

---------------------------------------------------------------------