Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Lucene and all its subprojects:

Switch to Threaded View
Lucene >> mail # user >> integrating RAMDirectory in FSDirectory


Copy link to this message
-
integrating RAMDirectory in FSDirectory
well actually I am doing a kind of a thesis regarding information
retrieval.and my tutor wanted me to be able to create a program that
firstly index a document in memory using RAMDirectory and then
flushing it to disk using FSDirectory periodically. I was having a
hard time implementing it in my source code. so that's why I am asking
what's the best way to put RAMDirectory methods in the source code
below? maybe you can give me an idea.eventhough I know it'll take
longer time to index due to the use of RAMDirectory and FSDirectory
but nonetheless I'd still be interested in how to integrate
RAMDirectory into FSDirectory.

thanks

PS : regarding the hijack thread, I thought because I was the one who
created it so I think I could hijack it.but thanks for the advice
though and now I am creating a new thread. :-)

On 10/12/10, Erick Erickson <[EMAIL PROTECTED]> wrote:
> It's a good idea to start a new thread when asking a different question,
> see:
> http://people.apache.org/~hossman/#threadhijack
>
> <http://people.apache.org/~hossman/#threadhijack>I have to ask why you want
> to integrate the RAM directory. If you're using it
> to speed up indexing, you're probably making way more work for yourself
> than you need to. If you're trying to do something with Near Real Time, one
> suggestion is to just not bother. Add docs to the RAM directory AND your
> FSDirectory simultaneously. The data you index to FSDir won't be visible
> until you reopen the FSDir reader, so your flush could then be just
> reopen everything...
>
> Best
> Erick
>
> On Mon, Oct 11, 2010 at 1:34 PM, Yakob <[EMAIL PROTECTED]> wrote:
>
>> On 9/29/10, Erick Erickson <[EMAIL PROTECTED]> wrote:
>> > Nope, never used jNotify, so I don't have any code handy...
>> >
>> > Good luck!
>> > Erick
>> >
>>
>> so I did try JNotify but there is seems to be some bug in it that I
>> find it hards to integrate in my lucene source code.so I had to try a
>> looping option instead.
>>
>>
>> http://stackoverflow.com/questions/3840844/error-exception-access-violation-in-jnotify
>>
>> so anyway, I had another question now. I was trying to make a lucene
>> source code that can do indexing and store them first in a memory
>> using RAMDirectory and then flush this index in a memory into a disk
>> using FSDirectory. I had done some modifications of this code but to
>> no avail. maybe some of you can help me out a bit.
>> here is the source code again.
>>
>> import java.io.File;
>> import java.io.FileReader;
>> import java.io.IOException;
>> import org.apache.lucene.analysis.SimpleAnalyzer;
>> import org.apache.lucene.document.Document;
>> import org.apache.lucene.document.Field;
>> import org.apache.lucene.index.IndexWriter;
>> import org.apache.lucene.store.FSDirectory;
>>
>>
>> public class SimpleFileIndexer {
>>
>>
>>        public static void main(String[] args) throws Exception   {
>>
>>                 int i=0;
>>                while(i<10) {
>>                 File indexDir = new
>> File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
>>                File dataDir = new
>> File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
>>                String suffix = "txt";
>>
>>                SimpleFileIndexer indexer = new SimpleFileIndexer();
>>
>>                int numIndex = indexer.index(indexDir, dataDir, suffix);
>>
>>                System.out.println("Total files indexed " + numIndex);
>>                 i++;
>>                Thread.sleep(10000);
>>
>>                }
>>        }
>>
>>
>>
>>        private int index(File indexDir, File dataDir, String suffix)
>> throws
>> Exception {
>>
>>                IndexWriter indexWriter = new IndexWriter(
>>                                FSDirectory.open(indexDir),
>>                                new SimpleAnalyzer(),
>>                                true,
>>                                IndexWriter.MaxFieldLength.LIMITED);
>>                indexWriter.setUseCompoundFile(false);
http://jacobian.web.id

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB