-
RE: WordBreak and default dictionary crash SolrDyer, James 2012-06-15, 21:16
Carrie,
Thank you for trying out new features! I'm pretty sure you've found a bug here. Could you tell me whether you're using a build from Trunk or Solr_4x ? Also, do you know the svn revision or the Jenkins build # (or timestamp) you're working from?
Could you try instead to use DirectSolrSpellChecker instead of IndexBasedSpellChecker for your "default" dictionary? (In Trunk and the 4.x branch, the Solr Example now uses DirectSolrSpellChecker as its default.) It could be this is a problem related to using WordBreakSolrSpellChecker with the older IndexBasedSpellChecker. So if you have better luck with DirectSolrSpellChecker, that would be helpful in honing in on the exact problem.
Also, judging from the line that is failing, could it be you're using a build based on svn revision pre-r1346489 (Trunk) or pre-r1346499 (Branch_4x) ? https://issues.apache.org/jira/browse/SOLR-2993 Shortly after the initial commit of this feature, a bug similar to the one you're reporting was later fixed with these subsequent revisions.
James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311
-----Original Message-----
From: Carrie Coy [mailto:[EMAIL PROTECTED]]
Sent: Friday, June 15, 2012 2:46 PM
To: [EMAIL PROTECTED]
Subject: WordBreak and default dictionary crash Solr
Is this a configuration problem or a bug?
We use two dictionaries, default (spellcheckerFreq) and
solr.WordBreakSolrSpellChecker. When a query contains 2 misspellings,
one corrected by the default dictionary, and the other corrected by the
wordbreak dictionary ("strawberryn shortcake") , Solr crashes with error
below. It doesn't matter which dictionary is checked first.
java.lang.NullPointerException
at
org.apache.solr.handler.component.SpellCheckComponent.toNamedList(SpellCheckComponent.java:566)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:177)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1555)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Multiple errors corrected by the SAME dictionary (either wordbreak or
default) do not crash Solr. Here is excerpt from our solrconfig.xml:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="name">wordbreak</str>
<str name="classname">solr.WordBreakSolrSpellChecker</str>
<str name="field">spell</str>
<str name="combineWords">true</str>
<str name="breakWords">true</str>
<int name="maxChanges">1</int>
</lst>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">spell</str>
<str name="spellcheckIndexDir">spellcheckerFreq</str>
<str name="buildOnOptimize">true</str>
</lst>
</searchComponent>
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
.....
<str name="spellcheck.dictionary">wordbreak</str>
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck.count">3</str>
<str name="spellcheck.collate">true</str>
<str name="spellcheck.onlyMorePopular">false</str>
</lst>
</requestHandler>
Thank you for trying out new features! I'm pretty sure you've found a bug here. Could you tell me whether you're using a build from Trunk or Solr_4x ? Also, do you know the svn revision or the Jenkins build # (or timestamp) you're working from?
Could you try instead to use DirectSolrSpellChecker instead of IndexBasedSpellChecker for your "default" dictionary? (In Trunk and the 4.x branch, the Solr Example now uses DirectSolrSpellChecker as its default.) It could be this is a problem related to using WordBreakSolrSpellChecker with the older IndexBasedSpellChecker. So if you have better luck with DirectSolrSpellChecker, that would be helpful in honing in on the exact problem.
Also, judging from the line that is failing, could it be you're using a build based on svn revision pre-r1346489 (Trunk) or pre-r1346499 (Branch_4x) ? https://issues.apache.org/jira/browse/SOLR-2993 Shortly after the initial commit of this feature, a bug similar to the one you're reporting was later fixed with these subsequent revisions.
James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311
-----Original Message-----
From: Carrie Coy [mailto:[EMAIL PROTECTED]]
Sent: Friday, June 15, 2012 2:46 PM
To: [EMAIL PROTECTED]
Subject: WordBreak and default dictionary crash Solr
Is this a configuration problem or a bug?
We use two dictionaries, default (spellcheckerFreq) and
solr.WordBreakSolrSpellChecker. When a query contains 2 misspellings,
one corrected by the default dictionary, and the other corrected by the
wordbreak dictionary ("strawberryn shortcake") , Solr crashes with error
below. It doesn't matter which dictionary is checked first.
java.lang.NullPointerException
at
org.apache.solr.handler.component.SpellCheckComponent.toNamedList(SpellCheckComponent.java:566)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:177)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1555)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Multiple errors corrected by the SAME dictionary (either wordbreak or
default) do not crash Solr. Here is excerpt from our solrconfig.xml:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="name">wordbreak</str>
<str name="classname">solr.WordBreakSolrSpellChecker</str>
<str name="field">spell</str>
<str name="combineWords">true</str>
<str name="breakWords">true</str>
<int name="maxChanges">1</int>
</lst>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">spell</str>
<str name="spellcheckIndexDir">spellcheckerFreq</str>
<str name="buildOnOptimize">true</str>
</lst>
</searchComponent>
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
.....
<str name="spellcheck.dictionary">wordbreak</str>
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck.count">3</str>
<str name="spellcheck.collate">true</str>
<str name="spellcheck.onlyMorePopular">false</str>
</lst>
</requestHandler>
