|
|
-
Real time indexing with RAMDirectory
Rifflard Mickaël 2005-05-10, 14:47
Hi all,
Is it possible, with the RAMDirectory (or another Directory), to "flush" informations after each Document indexing ? I tried this but this "flush" appears to be able to be made after 2 indexing at best.
What do you think about it ? I forgot a configuration ?
Thanks, Mickaël
---------------------------------------------------------------------
+
Rifflard Mickaël 2005-05-10, 14:47
-
RE: Real time indexing with RAMDirectory
Rifflard Mickaël 2005-05-11, 06:34
Hi Otis,
My question was too much short cut.
Here is a sample :
import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexReader; import org.apache.lucene.store.RAMDirectory; import org.apache.lucene.analysis.standard.StandardAnalyzer; import java.io.IOException;
public class MySample { public static void main(String[] args) { try { RAMDirectory ramd = new RAMDirectory(); IndexWriter iw = new IndexWriter(ramd,new StandardAnalyzer(),true); iw.minMergeDocs = 1; iw.addDocument(new org.apache.lucene.document.Document()); IndexReader ir = IndexReader.open(ramd); System.out.println("Docs number : " + ir.numDocs()); ir.close(); iw.close(); } catch (IOException ioe) { ioe.printStackTrace(); } } }
If I execute this sample, the result is :
Docs number : 0
If a call close() method or optimize() method after added my document, the result is :
Docs number : 1
So, my question is : How to get the second result without close() or optimize() index (expensive methods I think) ?
Thanks,
Mickaël
-----Message d'origine----- De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] Envoyé : mardi 10 mai 2005 17:39 À : [EMAIL PROTECTED] Objet : Re: Real time indexing with RAMDirectory Hi Mickaël,
Have you tried using minMergeDocs=1 ? Will that do what you want?
Otis
--- Rifflard Mickaël <[EMAIL PROTECTED]> wrote: > Hi all, > > Is it possible, with the RAMDirectory (or another Directory), to > "flush" informations after each Document indexing ? > I tried this but this "flush" appears to be able to be made after 2 > indexing at best. > > What do you think about it ? I forgot a configuration ? > > Thanks, > > Mickaël > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
--------------------------------------------------------------------- ---------------------------------------------------------------------
+
Rifflard Mickaël 2005-05-11, 06:34
-
RE: Real time indexing with RAMDirectory
Otis Gospodnetic 2005-05-11, 19:38
What happens if you swap these 2 lines?
System.out.println("Docs number : " + ir.numDocs()); ir.close();
If I were you, I'd try using minMergeDocs instead of RAMDirectory. It makes things much simpler. You shouldn't need to optimize the index. Otis
--- Rifflard Micka���l <[EMAIL PROTECTED]> wrote: > Hi Otis, > > My question was too much short cut. > > Here is a sample : > > import org.apache.lucene.index.IndexWriter; > import org.apache.lucene.index.IndexReader; > import org.apache.lucene.store.RAMDirectory; > import org.apache.lucene.analysis.standard.StandardAnalyzer; > import java.io.IOException; > > public class MySample { > public static void main(String[] args) { > try { > RAMDirectory ramd = new RAMDirectory(); > IndexWriter iw = new IndexWriter(ramd,new > StandardAnalyzer(),true); > iw.minMergeDocs = 1; > iw.addDocument(new org.apache.lucene.document.Document()); > > IndexReader ir = IndexReader.open(ramd); > System.out.println("Docs number : " + ir.numDocs()); > ir.close(); > iw.close(); > } > catch (IOException ioe) { ioe.printStackTrace(); } > } > } > > If I execute this sample, the result is : > > Docs number : 0 > > If a call close() method or optimize() method after added my > document, the result is : > > Docs number : 1 > > So, my question is : How to get the second result without close() or > optimize() index > (expensive methods I think) ? > > Thanks, > > Micka���l > > > > -----Message d'origine----- > De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > Envoy��� : mardi 10 mai 2005 17:39 > ��� : [EMAIL PROTECTED] > Objet : Re: Real time indexing with RAMDirectory > > > Hi Micka���l, > > Have you tried using minMergeDocs=1 ? Will that do what you want? > > Otis > > --- Rifflard Micka���l <[EMAIL PROTECTED]> wrote: > > Hi all, > > > > Is it possible, with the RAMDirectory (or another Directory), to > > "flush" informations after each Document indexing ? > > I tried this but this "flush" appears to be able to be made after 2 > > indexing at best. > > > > What do you think about it ? I forgot a configuration ? > > > > Thanks, > > > > Micka���l > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
---------------------------------------------------------------------
+
Otis Gospodnetic 2005-05-11, 19:38
-
RE: Real time indexing with RAMDirectory
Rifflard Mickaël 2005-05-12, 07:13
Hi Otis,
If I swap these two lines, the result is the same.
I want to build an indexing process as fast as possible and I can't use a greater minMergeDocs because my need is to know all documents of my index in real time.
Do you think that a FSDirectory with minMergeDocs equals to 1 can be the solution ?
Thanks,
Mickaël
-----Message d'origine----- De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] Envoyé : mercredi 11 mai 2005 21:38 À : [EMAIL PROTECTED] Objet : RE: Real time indexing with RAMDirectory What happens if you swap these 2 lines?
System.out.println("Docs number : " + ir.numDocs()); ir.close();
If I were you, I'd try using minMergeDocs instead of RAMDirectory. It makes things much simpler. You shouldn't need to optimize the index. Otis
--- Rifflard Mickaël <[EMAIL PROTECTED]> wrote: > Hi Otis, > > My question was too much short cut. > > Here is a sample : > > import org.apache.lucene.index.IndexWriter; > import org.apache.lucene.index.IndexReader; > import org.apache.lucene.store.RAMDirectory; > import org.apache.lucene.analysis.standard.StandardAnalyzer; > import java.io.IOException; > > public class MySample { > public static void main(String[] args) { > try { > RAMDirectory ramd = new RAMDirectory(); > IndexWriter iw = new IndexWriter(ramd,new > StandardAnalyzer(),true); > iw.minMergeDocs = 1; > iw.addDocument(new org.apache.lucene.document.Document()); > > IndexReader ir = IndexReader.open(ramd); > System.out.println("Docs number : " + ir.numDocs()); > ir.close(); > iw.close(); > } > catch (IOException ioe) { ioe.printStackTrace(); } > } > } > > If I execute this sample, the result is : > > Docs number : 0 > > If a call close() method or optimize() method after added my > document, the result is : > > Docs number : 1 > > So, my question is : How to get the second result without close() or > optimize() index > (expensive methods I think) ? > > Thanks, > > Mickaël > > > > -----Message d'origine----- > De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > Envoyé : mardi 10 mai 2005 17:39 > À : [EMAIL PROTECTED] > Objet : Re: Real time indexing with RAMDirectory > > > Hi Mickaël, > > Have you tried using minMergeDocs=1 ? Will that do what you want? > > Otis > > --- Rifflard Mickaël <[EMAIL PROTECTED]> wrote: > > Hi all, > > > > Is it possible, with the RAMDirectory (or another Directory), to > > "flush" informations after each Document indexing ? > > I tried this but this "flush" appears to be able to be made after 2 > > indexing at best. > > > > What do you think about it ? I forgot a configuration ? > > > > Thanks, > > > > Mickaël > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
--------------------------------------------------------------------- ---------------------------------------------------------------------
+
Rifflard Mickaël 2005-05-12, 07:13
-
RE: Real time indexing with RAMDirectory
Otis Gospodnetic 2005-05-12, 07:37
Yes, try minMergeDocs = 1.... but keep in mind that you'll have to be re-opening a new IndexReader every time your index changes, and it sounds like this will be very frequent/constant. This may cost you...
Otis
--- Rifflard Micka���l <[EMAIL PROTECTED]> wrote:
> Hi Otis, > > If I swap these two lines, the result is the same. > > I want to build an indexing process as fast as possible and I can't > use > a greater minMergeDocs because my need is to know all documents of my > > index in real time. > > Do you think that a FSDirectory with minMergeDocs equals to 1 can > be the solution ? > > Thanks, > > Micka���l > > -----Message d'origine----- > De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > Envoy��� : mercredi 11 mai 2005 21:38 > ��� : [EMAIL PROTECTED] > Objet : RE: Real time indexing with RAMDirectory > > > What happens if you swap these 2 lines? > > System.out.println("Docs number : " + ir.numDocs()); > ir.close(); > > If I were you, I'd try using minMergeDocs instead of RAMDirectory. > It > makes things much simpler. You shouldn't need to optimize the index. > > > Otis > > --- Rifflard Micka���l <[EMAIL PROTECTED]> wrote: > > Hi Otis, > > > > My question was too much short cut. > > > > Here is a sample : > > > > import org.apache.lucene.index.IndexWriter; > > import org.apache.lucene.index.IndexReader; > > import org.apache.lucene.store.RAMDirectory; > > import org.apache.lucene.analysis.standard.StandardAnalyzer; > > import java.io.IOException; > > > > public class MySample { > > public static void main(String[] args) { > > try { > > RAMDirectory ramd = new RAMDirectory(); > > IndexWriter iw = new IndexWriter(ramd,new > > StandardAnalyzer(),true); > > iw.minMergeDocs = 1; > > iw.addDocument(new org.apache.lucene.document.Document()); > > > > IndexReader ir = IndexReader.open(ramd); > > System.out.println("Docs number : " + ir.numDocs()); > > ir.close(); > > iw.close(); > > } > > catch (IOException ioe) { ioe.printStackTrace(); } > > } > > } > > > > If I execute this sample, the result is : > > > > Docs number : 0 > > > > If a call close() method or optimize() method after added my > > document, the result is : > > > > Docs number : 1 > > > > So, my question is : How to get the second result without close() > or > > optimize() index > > (expensive methods I think) ? > > > > Thanks, > > > > Micka���l > > > > > > > > -----Message d'origine----- > > De : Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > > Envoy��� : mardi 10 mai 2005 17:39 > > ��� : [EMAIL PROTECTED] > > Objet : Re: Real time indexing with RAMDirectory > > > > > > Hi Micka���l, > > > > Have you tried using minMergeDocs=1 ? Will that do what you want? > > > > Otis > > > > --- Rifflard Micka���l <[EMAIL PROTECTED]> wrote: > > > Hi all, > > > > > > Is it possible, with the RAMDirectory (or another Directory), to > > > "flush" informations after each Document indexing ? > > > I tried this but this "flush" appears to be able to be made after > 2 > > > indexing at best. > > > > > > What do you think about it ? I forgot a configuration ? > > > > > > Thanks, > > > > > > Micka���l > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED]
+
Otis Gospodnetic 2005-05-12, 07:37
-
Re: Real time indexing with RAMDirectory
Otis Gospodnetic 2005-05-10, 15:39
Hi Micka���l,
Have you tried using minMergeDocs=1 ? Will that do what you want?
Otis
--- Rifflard Micka���l <[EMAIL PROTECTED]> wrote: > Hi all, > > Is it possible, with the RAMDirectory (or another Directory), to > "flush" informations after each Document indexing ? > I tried this but this "flush" appears to be able to be made after 2 > indexing at best. > > What do you think about it ? I forgot a configuration ? > > Thanks, > > Micka���l > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
---------------------------------------------------------------------
+
Otis Gospodnetic 2005-05-10, 15:39
|
|