I have an indexer that reads data from database and indexes the data.
  foreach(db_row) {
      Document doc = new Document();
      doc.add(Field.Text("Product", productName);
      doc.add(Field.Text("Description", productDesc);
...
      writer.addDocument(doc);
  }
Once indexed, I would like to do a search that spans across multiple fields.
i.e. the user may enter "lawnmower" and it should perform a search across
all the indexed fields. Also, how do I pass user queries like  "lawnmower
-grass" to the query API?

---------------------------------------------------------------------

Shiv,

Searching in Lucene is field-based.  Thus you must specify the field to be
searched - the only 'exception' is that one field is defined as default.  If
you want to search across multiple fields, I believe you must create a
concatenation of the individual fields into a single one during the indexing
process (eg. productName+" "+productDesc), and then use that as the basis of
your subsequent searches.

HTH,

Terry

----- Original Message -----
From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Saturday, May 31, 2003 10:33 AM
Subject: searching data indexed from database??
> I have an indexer that reads data from database and indexes the data.
>   foreach(db_row) {
>       Document doc = new Document();
>       doc.add(Field.Text("Product", productName);
>       doc.add(Field.Text("Description", productDesc);
> ...
>       writer.addDocument(doc);
>   }
>
>
> Once indexed, I would like to do a search that spans across multiple
fields.
> i.e. the user may enter "lawnmower" and it should perform a search across
> all the indexed fields. Also, how do I pass user queries like  "lawnmower
> -grass" to the query API?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------

Thanks. That's what I suspected and was hoping that wasn't the case. Won't
this lead to duplication of data during indexing -- one piece under the
specific field ("Product") and the same one (along with others) under the
default field ("contents")?

-----Original Message-----
From: Terry Steichen
To: Lucene Users List
Sent: 5/31/03 8:06 AM
Subject: Re: searching data indexed from database??

Shiv,

Searching in Lucene is field-based.  Thus you must specify the field to
be
searched - the only 'exception' is that one field is defined as default.
If
you want to search across multiple fields, I believe you must create a
concatenation of the individual fields into a single one during the
indexing
process (eg. productName+" "+productDesc), and then use that as the
basis of
your subsequent searches.

HTH,

Terry

----- Original Message -----
From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Saturday, May 31, 2003 10:33 AM
Subject: searching data indexed from database??
> I have an indexer that reads data from database and indexes the data.
>   foreach(db_row) {
>       Document doc = new Document();
>       doc.add(Field.Text("Product", productName);
>       doc.add(Field.Text("Description", productDesc);
> ...
>       writer.addDocument(doc);
>   }
>
>
> Once indexed, I would like to do a search that spans across multiple
fields.
> i.e. the user may enter "lawnmower" and it should perform a search
across
> all the indexed fields. Also, how do I pass user queries like
"lawnmower
> -grass" to the query API?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------

---------------------------------------------------------------------

Well, if you don't store the data in the index it probably isn't too bad.
Alternatively, if you don't need to do any field-specific searching, then
you *only* index the combined field (and *not* the individual ones).  Then
there's no additional impact.

Regards,

Terry

----- Original Message -----
From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
To: "'Terry Steichen '" <[EMAIL PROTECTED]>; "'Lucene Users List '"
<[EMAIL PROTECTED]>
Sent: Saturday, May 31, 2003 11:31 AM
Subject: RE: searching data indexed from database??
> Thanks. That's what I suspected and was hoping that wasn't the case. Won't
> this lead to duplication of data during indexing -- one piece under the
> specific field ("Product") and the same one (along with others) under the
> default field ("contents")?
>
> -----Original Message-----
> From: Terry Steichen
> To: Lucene Users List
> Sent: 5/31/03 8:06 AM
> Subject: Re: searching data indexed from database??
>
> Shiv,
>
> Searching in Lucene is field-based.  Thus you must specify the field to
> be
> searched - the only 'exception' is that one field is defined as default.
> If
> you want to search across multiple fields, I believe you must create a
> concatenation of the individual fields into a single one during the
> indexing
> process (eg. productName+" "+productDesc), and then use that as the
> basis of
> your subsequent searches.
>
> HTH,
>
> Terry
>
> ----- Original Message -----
> From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Saturday, May 31, 2003 10:33 AM
> Subject: searching data indexed from database??
>
>
> > I have an indexer that reads data from database and indexes the data.
> >   foreach(db_row) {
> >       Document doc = new Document();
> >       doc.add(Field.Text("Product", productName);
> >       doc.add(Field.Text("Description", productDesc);
> > ...
> >       writer.addDocument(doc);
> >   }
> >
> >
> > Once indexed, I would like to do a search that spans across multiple
> fields.
> > i.e. the user may enter "lawnmower" and it should perform a search
> across
> > all the indexed fields. Also, how do I pass user queries like
> "lawnmower
> > -grass" to the query API?
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
---------------------------------------------------------------------

The user could either do a field search (on a specific field) or a full-text
search (that defaults to the default field "contents"). Is there a way to
optimize the storage in this case?

-----Original Message-----
From: Terry Steichen
To: 'Lucene Users List '
Sent: 5/31/03 9:18 AM
Subject: Re: searching data indexed from database??

Well, if you don't store the data in the index it probably isn't too
bad.
Alternatively, if you don't need to do any field-specific searching,
then
you *only* index the combined field (and *not* the individual ones).
Then
there's no additional impact.

Regards,

Terry

----- Original Message -----
From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
To: "'Terry Steichen '" <[EMAIL PROTECTED]>; "'Lucene Users List '"
<[EMAIL PROTECTED]>
Sent: Saturday, May 31, 2003 11:31 AM
Subject: RE: searching data indexed from database??
> Thanks. That's what I suspected and was hoping that wasn't the case.
Won't
> this lead to duplication of data during indexing -- one piece under
the
> specific field ("Product") and the same one (along with others) under
the
> default field ("contents")?
>
> -----Original Message-----
> From: Terry Steichen
> To: Lucene Users List
> Sent: 5/31/03 8:06 AM
> Subject: Re: searching data indexed from database??
>
> Shiv,
>
> Searching in Lucene is field-based.  Thus you must specify the field
to
> be
> searched - the only 'exception' is that one field is defined as
default.
> If
> you want to search across multiple fields, I believe you must create a
> concatenation of the individual fields into a single one during the
> indexing
> process (eg. productName+" "+productDesc), and then use that as the
> basis of
> your subsequent searches.
>
> HTH,
>
> Terry
>
> ----- Original Message -----
> From: "Venkatraman, Shiv" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Saturday, May 31, 2003 10:33 AM
> Subject: searching data indexed from database??
>
>
> > I have an indexer that reads data from database and indexes the
data.
> >   foreach(db_row) {
> >       Document doc = new Document();
> >       doc.add(Field.Text("Product", productName);
> >       doc.add(Field.Text("Description", productDesc);
> > ...
> >       writer.addDocument(doc);
> >   }
> >
> >
> > Once indexed, I would like to do a search that spans across multiple
> fields.
> > i.e. the user may enter "lawnmower" and it should perform a search
> across
> > all the indexed fields. Also, how do I pass user queries like
> "lawnmower
> > -grass" to the query API?
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
---------------------------------------------------------------------

---------------------------------------------------------------------

Take a look at org.apache.lucene.queryParse.MultiFieldQueryParser.  It
allows you to past a list of field names to search on rather than just a
single field name.

Regarding your second question, the query you specified works fine in
1.3 RC 1 although I am not familiar enough with 1.2 to know whether it
was supported there or not.

Sincerely,
Anthony Eden

Venkatraman, Shiv wrote:

>I have an indexer that reads data from database and indexes the data.
>  foreach(db_row) {
>      Document doc = new Document();
>      doc.add(Field.Text("Product", productName);
>      doc.add(Field.Text("Description", productDesc);
>...
>      writer.addDocument(doc);
>  }
>
>
>Once indexed, I would like to do a search that spans across multiple fields.
>i.e. the user may enter "lawnmower" and it should perform a search across
>all the indexed fields. Also, how do I pass user queries like  "lawnmower
>-grass" to the query API?
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]
>  
>
---------------------------------------------------------------------

Yes it is supported in 1.2 I used it in my search tag.
http://cvs.apache.org/viewcvs/jakarta-lucene-sandbox/contributions/taglib/WE
B-INF/classes/com/netwebapps/taglib/search/SearchTag.java

Relevant lines of code are
line 439, 461, 469,275
----- Original Message -----
From: "Anthony Eden" <[EMAIL PROTECTED]>
To: "Lucene Users List" <[EMAIL PROTECTED]>
Sent: Saturday, May 31, 2003 11:49 AM
Subject: Re: searching data indexed from database??
> Take a look at org.apache.lucene.queryParse.MultiFieldQueryParser.  It
> allows you to past a list of field names to search on rather than just a
> single field name.
>
> Regarding your second question, the query you specified works fine in
> 1.3 RC 1 although I am not familiar enough with 1.2 to know whether it
> was supported there or not.
>
> Sincerely,
> Anthony Eden
>
> Venkatraman, Shiv wrote:
>
> >I have an indexer that reads data from database and indexes the data.
> >  foreach(db_row) {
> >      Document doc = new Document();
> >      doc.add(Field.Text("Product", productName);
> >      doc.add(Field.Text("Description", productDesc);
> >...
> >      writer.addDocument(doc);
> >  }
> >
> >
> >Once indexed, I would like to do a search that spans across multiple
fields.
> >i.e. the user may enter "lawnmower" and it should perform a search across
> >all the indexed fields. Also, how do I pass user queries like  "lawnmower
> >-grass" to the query API?
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: [EMAIL PROTECTED]
> >For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------