Hi,

Sorry if I am asking this question again in this forum..

Is there any plugin which I can use to do a realtime indexing?

I have a requirement where we have an application which sits on top of SQL
server DB and updates happen on day to day basis. Users would like to see
the changes made to the DB immediately in the search results. I am thinking
of using JMS queue for achieving this, but before that I just want to check
if anyone has implemented similar kind of requirement before?

Any help / suggestions would be greatly appreciated.

Thanks,
bb
--
View this message in context: http://lucene.472066.n3.nabble.com/Any-realtime-indexing-plugin-available-for-SOLR-tp845026p845026.html
Sent from the Solr - User mailing list archive at Nabble.com.

Maybe this will help you

http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin

Marco Martínez Bautista
http://www.paradigmatecnologico.com
Avenida de Europa, 26. Ática 5. 3ª Planta
28224 Pozuelo de Alarcón
Tel.: 91 352 59 42
2010/5/26 bbarani <[EMAIL PROTECTED]>

>
> Hi,
>
> Sorry if I am asking this question again in this forum..
>
> Is there any plugin which I can use to do a realtime indexing?
>
> I have a requirement where we have an application which sits on top of SQL
> server DB and updates happen on day to day basis. Users would like to see
> the changes made to the DB immediately in the search results. I am thinking
> of using JMS queue for achieving this, but before that I just want to check
> if anyone has implemented similar kind of requirement before?
>
> Any help / suggestions would be greatly appreciated.
>
> Thanks,
> bb
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Any-realtime-indexing-plugin-available-for-SOLR-tp845026p845026.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

I'm afraid nothing is completely 'real-time'. Even when doing your inserts on the database there is time taken for those operations to complete. Right now I have my solr server autocommiting every 30 seconds, which is 'real-time' enough for me. You need to figure out what your threshold is, and then tune your index, hardware, caching to achieve it. If you don't want the results to show up in the database before the search you could store an 'indexed' value in the DB which you flip after you've indexed the new data.

-Kallin Nagelberg

-----Original Message-----
From: bbarani [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, May 26, 2010 10:39 AM
To: solr[EMAIL PROTECTED]
Subject: Any realtime indexing plugin available for SOLR
Hi,

Sorry if I am asking this question again in this forum..

Is there any plugin which I can use to do a realtime indexing?

I have a requirement where we have an application which sits on top of SQL
server DB and updates happen on day to day basis. Users would like to see
the changes made to the DB immediately in the search results. I am thinking
of using JMS queue for achieving this, but before that I just want to check
if anyone has implemented similar kind of requirement before?

Any help / suggestions would be greatly appreciated.

Thanks,
bb
--
View this message in context: http://lucene.472066.n3.nabble.com/Any-realtime-indexing-plugin-available-for-SOLR-tp845026p845026.html
Sent from the Solr - User mailing list archive at Nabble.com.

I thought that if entries were COMMITed to the index, they were immediately visible?

Is this true, or am I smoking Java coffee beans?
Dennis Gearon

Signature Warning
----------------
EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Wed, 5/26/10, Marco Martinez <[EMAIL PROTECTED]> wrote:

> From: Marco Martinez <[EMAIL PROTECTED]>
> Subject: Re: Any realtime indexing plugin available for SOLR
> To: solr[EMAIL PROTECTED]
> Date: Wednesday, May 26, 2010, 7:51 AM
> Maybe this will help you
>
> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin
>
> Marco Martínez Bautista
> http://www.paradigmatecnologico.com
> Avenida de Europa, 26. Ática 5. 3ª Planta
> 28224 Pozuelo de Alarcón
> Tel.: 91 352 59 42
>
>
> 2010/5/26 bbarani <[EMAIL PROTECTED]>
>
> >
> > Hi,
> >
> > Sorry if I am asking this question again in this
> forum..
> >
> > Is there any plugin which I can use to do a realtime
> indexing?
> >
> > I have a requirement where we have an application
> which sits on top of SQL
> > server DB and updates happen on day to day basis.
> Users would like to see
> > the changes made to the DB immediately in the search
> results. I am thinking
> > of using JMS queue for achieving this, but before that
> I just want to check
> > if anyone has implemented similar kind of requirement
> before?
> >
> > Any help / suggestions would be greatly appreciated.
> >
> > Thanks,
> > bb
> > --
> > View this message in context:
> > http://lucene.472066.n3.nabble.com/Any-realtime-indexing-plugin-available-for-SOLR-tp845026p845026.html
> > Sent from the Solr - User mailing list archive at
> Nabble.com.
> >
>

If it is your app that is updating data in the DB, then you could have it update Solr at the same time

Regards
Stefan Maric
-----Original Message-----
From: bbarani [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, May 26, 2010 10:39 AM
To: solr[EMAIL PROTECTED]
Subject: Any realtime indexing plugin available for SOLR
Hi,

Sorry if I am asking this question again in this forum..

Is there any plugin which I can use to do a realtime indexing?

I have a requirement where we have an application which sits on top of SQL
server DB and updates happen on day to day basis. Users would like to see
the changes made to the DB immediately in the search results. I am thinking
of using JMS queue for achieving this, but before that I just want to check
if anyone has implemented similar kind of requirement before?

Any help / suggestions would be greatly appreciated.

Thanks,
bb
--
View this message in context: http://lucene.472066.n3.nabble.com/Any-realtime-indexing-plugin-available-for-SOLR-tp845026p845026.html
Sent from the Solr - User mailing list archive at Nabble.com.

On May 26, 2010, at 11:29 AM, Dennis Gearon wrote:

> I thought that if entries were COMMITed to the index, they were  
> immediately visible?
>
> Is this true, or am I smoking Java coffee beans?

They're visible after a commit AND warming are complete, yes.   But  
there could be a potentially substantial delay between a commit  
message being sent and the new documents actually searchable.

Erik

Funny enough, I've been looking for my own solution too.  The Zoie plugin does not work on multi-core setups, so that's bust for me.  Once you commit something to index, you need to "warm" a new searcher (load all the data from disk into memory/cache) like Erik says.  On a smaller index, this is very very quick, however on a larger index, not so much.

Solr 1.5 will (hopefully) have a new feature that will allow for near real time searching.  Check this out:

http://wiki.apache.org/solr/NearRealtimeSearch
On May 27, 2010, at 6:00 AM, Erik Hatcher wrote:

>
> On May 26, 2010, at 11:29 AM, Dennis Gearon wrote:
>
>> I thought that if entries were COMMITed to the index, they were immediately visible?
>>
>> Is this true, or am I smoking Java coffee beans?
>
> They're visible after a commit AND warming are complete, yes.   But there could be a potentially substantial delay between a commit message being sent and the new documents actually searchable.
>
> Erik
>

---
Antonio Lobato
Symplicity Corporation
www.symplicity.com
(703) 351-0200 x 8101
[EMAIL PROTECTED]