|
|
-
Add "hint" to Solr QParser that it is a filter query
Smiley, David W. 2011-11-07, 19:32
I am implementing a custom Solr FieldType for Solr trunk. One method I override is: Query getFieldQuery(QParser parser, SchemaField field, String externalVal) My Query implementation can be vastly more efficient if the scores aren't needed vs. when they are needed. My code constructing this is basically: Filter f = makeFilter(...);//efficient ValueSource vs = makeValueSource(...);//potentially not needed and uses memory return new FilteredQuery( new FunctionQuery(vs), f ); I would love to know when it is okay to return a ConstantScoreQuery wrapping my Filter so that I needn't bother with my ValueSource. In my opinion, FieldType should have a getFieldFilter() method similar to getFieldQuery(). Perhaps a hint of some kind could be added to the QParser handed in -- a boolean flag or some special local-param to indicate the constant score to use. Solr could perform this when the QParser is constructed for a Filter Query.
Thoughts? Am I missing something?
~ David ---------------------------------------------------------------------
+
Smiley, David W. 2011-11-07, 19:32
-
Re: Add "hint" to Solr QParser that it is a filter query
Chris Hostetter 2011-11-07, 20:41
: I would love to know when it is okay to return a ConstantScoreQuery : wrapping my Filter so that I needn't bother with my ValueSource. In my : opinion, FieldType should have a getFieldFilter() method similar to : getFieldQuery(). Perhaps a hint of some kind could be added to the : QParser handed in -- a boolean flag or some special local-param to : indicate the constant score to use. Solr could perform this when the : QParser is constructed for a Filter Query.
my gut raction is that this seems like it could be dangerous in the general case -- it could lead to subtle bugs where a FieldType might inadvertantly produce a query that matches different documents depending on context (doesn't mean we shouldn't do it, just a risk i worry about)
if we do go this route, adding it as a hint on the QParser seems like a cleaner appraoch then a completely new method on FieldType (particularly because then generic QParsers might be able to leverage this info as well ... consider fq={!geofilt ... } vs q={!geofilt ... }).
As a first pass i would suggest that your FieldType could do this kind of logic as is right now by reqiring a special param (local or otherwise), something like...
fq={!field needScore=false f=yourSpecialField}foo
...where your field type has...
public Query getFieldQuery(QParser parser, SchemaField field, String externalVal) Filter f = makeFilter(...); if (parser.getLocalParams().getBool("needScore", true)) { ValueSource vs = makeValueSource(...); uses memory return new FilteredQuery( new FunctionQuery(vs), f ); } return new ConstantScoreQuery(f); }
...in general this type of explicit approach (based on a param) is apealing because it would help mitigate the risk i mentioned about queries matching diff docs being (accidently?) produced depending on context -- much easier to spot/understand if the query changes if-and-only-if a param changes.
-Hoss
---------------------------------------------------------------------
+
Chris Hostetter 2011-11-07, 20:41
-
Re: Add "hint" to Solr QParser that it is a filter query
Smiley, David W. 2011-11-08, 20:53
On Nov 7, 2011, at 3:41 PM, Chris Hostetter wrote:
> > : I would love to know when it is okay to return a ConstantScoreQuery > : wrapping my Filter so that I needn't bother with my ValueSource. In my > : opinion, FieldType should have a getFieldFilter() method similar to > : getFieldQuery(). Perhaps a hint of some kind could be added to the > : QParser handed in -- a boolean flag or some special local-param to > : indicate the constant score to use. Solr could perform this when the > : QParser is constructed for a Filter Query. > > my gut raction is that this seems like it could be dangerous in the > general case -- it could lead to subtle bugs where a FieldType might > inadvertantly produce a query that matches different documents depending > on context (doesn't mean we shouldn't do it, just a risk i worry about) > > if we do go this route, adding it as a hint on the QParser seems like a > cleaner appraoch then a completely new method on FieldType (particularly > because then generic QParsers might be able to leverage this info as well > ... consider fq={!geofilt ... } vs q={!geofilt ... }).
The QParser hint is definitely a practical solution that I can use right now as you suggest below. I'd like to see QueryComponent introduce this hint when it creates the Query from the fq param. I'll create a JIRA issue.
My proposal of a FieldType.getFilter() method is not a mutually exclusive suggestion with the QParser hint. FieldType already has a getFieldQuery(), getRangeQuery() and getValueSource() methods... a getFilter() method seems oddly missing to me. With this in place, and a QParser hint in place as well, the getFieldQuery() method in FieldType could check for the hint and return a new ConstantScoreQuery(getFilter(...)). If the hint is false then another method could be delegated to for FieldType subclasses that need custom behavior. Perhaps this addition would facilitate Solr's support for LUCENE-1536 "if a filter can support random access API, we should use it" I am not sure honestly. > As a first pass i would suggest that your FieldType could do this kind of > logic as is right now by reqiring a special param (local or otherwise), > something like... > > fq={!field needScore=false f=yourSpecialField}foo > > ...where your field type has... > > public Query getFieldQuery(QParser parser, SchemaField field, String externalVal) > Filter f = makeFilter(...); > if (parser.getLocalParams().getBool("needScore", true)) { > ValueSource vs = makeValueSource(...); > uses memory > return new FilteredQuery( new FunctionQuery(vs), f ); > } > return new ConstantScoreQuery(f); > } > > ...in general this type of explicit approach (based on a param) is > apealing because it would help mitigate the risk i mentioned about queries > matching diff docs being (accidently?) produced depending on context -- > much easier to spot/understand if the query changes if-and-only-if a param > changes. > > > > -Hoss > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > ---------------------------------------------------------------------
+
Smiley, David W. 2011-11-08, 20:53
|
|