Greetings,

I am in the process of updating custom code and schema from Solr 1.4 to
3.6.0 and have run into the following issue with our two custom Tokenizer
and Token Filter components.

I've been banging my head against this one for far too long, especially
since it must be something obvious I'm missing.

I have  custom Tokenizer and Token Filter components along with
corresponding factories. The code for all looks very similar to the
Tokenizer and TokenFilter (and Factory) code that is standard with 3.6.0
(and I have also read through
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters

I have ensured my custom code is on the classpath, it is
in ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar:
---output snip---
Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer load
INFO: loading shared library: /opt/test_artists_solr/jetty-solr/lib/en
Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/opt/test_artists_solr/jetty-solr/lib/en/ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar'
to classloader
Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/opt/test_artists_solr/jetty-solr/lib/en/ENUtil-1.0-SNAPSHOT-jar-with-dependencies.jar'
to classloader
Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer create

After successfully parsing the schema and creating many fields, etc.. the
following is logged:
---snip---
Jun 8, 2012 10:41:00 PM org.apache.solr.util.plugin.AbstractPluginLoader
load
INFO: created : com.company.MyCustomTokenizerFactory
Jun 8, 2012 10:41:00 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.ClassCastException: com.company.MyCustomTokenizerFactory
cannot be cast to org.apache.solr.analysis.TokenizerFactory
at org.apache.solr.schema.IndexSchema$5.init(IndexSchema.java:966)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:148)
at org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:986)
at org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:60)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:453)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:433)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140)
at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:490)
at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:123)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:481)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:102)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:748)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1222)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:676)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:455)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at
org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at
org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at
org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at
org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:260)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)

Jun 8, 2012 10:41:00 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: analyzer without class or
tokenizer & filter list
at org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:990)
at org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:60)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:453)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:433)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140)
at org.apache.solr.schema.IndexSchema.rea

Just in case it is helpful, here are the relevant pieces of my schema.xml:

---snip--
        <fieldtype name="customfield" class="solr.TextField"
positionIncrementGap="100">
            <analyzer type="index">
                <tokenizer class="com.company.MyCustomTokenizerFactory"/>
                <filter class="solr.StandardFilterFactory"/>
                <filter class="solr.ASCIIFoldingFilterFactory"/>
                <filter class="solr.LowerCaseFilterFactory"/>
                <filter class="solr.StopFilterFactory"
words="stopwords.txt" ignoreCase="true"/>
                <!--filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/-->
            </analyzer>
            <analyzer type="query">
                <tokenizer class="solr.KeywordTokenizerFactory"/>
                <filter class="solr.StandardFilterFactory"/>
                <filter class="solr.ASCIIFoldingFilterFactory"/>
                <filter class="solr.LowerCaseFilterFactory"/>
                <filter class="solr.StopFilterFactory"
words="stopwords.txt" ignoreCase="true"/>
                <!--filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/-->
            </analyzer>
        </fieldtype>
---snip---

and

---snip---
        <fieldtype name="customterms" class="solr.TextField"
positionIncrementGap="100">
            <analyzer>
                <tokenizer class="solr.KeywordTokenizerFactory"/>
                <filter class="com.company.MyCustomFilterFactory"/>
                <filter class="solr.StandardFilterFactory"/>
                <filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" expand="false"/>
                <filter class="solr.ASCIIFoldingFilterFactory"/>
                <filter class="solr.LowerCaseFilterFactory"/>
                <filter class="solr.PatternReplaceFilterFactory"
pattern="\-" replacement=" " replace="all"/>
                <filter class="solr.PatternReplaceFilterFactory"
pattern="&amp;" replacement="&" replace="all"/>
                <filter class="solr.PatternReplaceFilterFactory"
pattern="\s+" replacement=" " replace="all"/>
                <filter class="solr.TrimFilterFactory"/>
                <filter class="solr.SnowballPorterFilterFactory"
language="English" protected="protwords.txt"/>
            </analyzer>
        </fieldtype>
---snip---

On Sat, Jun 9, 2012 at 12:03 AM, Aaron Daubman <[EMAIL PROTECTED]> wrote:

> Greetings,
>
> I am in the process of updating custom code and schema from Solr 1.4 to
> 3.6.0 and have run into the following issue with our two custom Tokenizer
> and Token Filter components.
>
> I've been banging my head against this one for far too long, especially
> since it must be something obvious I'm missing.
>
> I have  custom Tokenizer and Token Filter components along with
> corresponding factories. The code for all looks very similar to the
> Tokenizer and TokenFilter (and Factory) code that is standard with 3.6.0
> (and I have also read through
> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
>
> I have ensured my custom code is on the classpath, it is
> in ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar:
> ---output snip---
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer load
> INFO: loading shared library: /opt/test_artists_solr/jetty-solr/lib/en
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
> replaceClassLoader
> INFO: Adding
> 'file:/opt/test_artists_solr/jetty-solr/lib/en/ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar'
> to classloader
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
> replaceClassLoader
> INFO: Adding
> 'file:/opt/test_artists_solr/jetty-solr/lib/en/ENUtil-1.0-SNAPSHOT-jar-with-dependencies.jar'
> to classloader
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer create
> --snip---
>
> After successfully parsing the schema and creating many fields, etc.. the
> following is logged:
> ---snip---
> Jun 8, 2012 10:41:00 PM org.apache.solr.util.plugin.AbstractPluginLoader

Make sure there are no stray jars/classes in your jar, especially any that
might contain BaseTokenizerFactory or TokenizerFactory. I notice that your
jar name says "-with-dependencies", raising a little suspicion. The
exception is as if your class was referring to a BaseTokenizerFactory, which
implements TokenizerFactory, coming from your jar (or a contained jar)
rather than getting resolved to Solr 3.6's own BaseTokenizerFactory and
TokenizerFactory.

-- Jack Krupansky

-----Original Message-----
From: Aaron Daubman
Sent: Saturday, June 09, 2012 12:03 AM
To: [EMAIL PROTECTED]
Subject: What would cause: "SEVERE: java.lang.ClassCastException:
com.company.MyCustomTokenizerFactory cannot be cast to
org.apache.solr.analysis.TokenizerFactory"

Greetings,

I am in the process of updating custom code and schema from Solr 1.4 to
3.6.0 and have run into the following issue with our two custom Tokenizer
and Token Filter components.

I've been banging my head against this one for far too long, especially
since it must be something obvious I'm missing.

I have  custom Tokenizer and Token Filter components along with
corresponding factories. The code for all looks very similar to the
Tokenizer and TokenFilter (and Factory) code that is standard with 3.6.0
(and I have also read through
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters

I have ensured my custom code is on the classpath, it is
in ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar:
---output snip---
Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer load
INFO: loading shared library: /opt/test_artists_solr/jetty-solr/lib/en
Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/opt/test_artists_solr/jetty-solr/lib/en/ENSolrComponents-1.0-SNAPSHOT-jar-with-dependencies.jar'
to classloader
Jun 8, 2012 10:41:00 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/opt/test_artists_solr/jetty-solr/lib/en/ENUtil-1.0-SNAPSHOT-jar-with-dependencies.jar'
to classloader
Jun 8, 2012 10:41:00 PM org.apache.solr.core.CoreContainer create

After successfully parsing the schema and creating many fields, etc.. the
following is logged:
---snip---
Jun 8, 2012 10:41:00 PM org.apache.solr.util.plugin.AbstractPluginLoader
load
INFO: created : com.company.MyCustomTokenizerFactory
Jun 8, 2012 10:41:00 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.ClassCastException: com.company.MyCustomTokenizerFactory
cannot be cast to org.apache.solr.analysis.TokenizerFactory
at org.apache.solr.schema.IndexSchema$5.init(IndexSchema.java:966)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:148)
at org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:986)
at org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:60)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:453)
at org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:433)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140)
at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:490)
at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:123)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:481)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:102)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:748)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1222)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:676)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:455)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at
org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at
org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at
org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at
org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at
org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:260)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.je

Jack,

Thanks - this was indeed the issue. I still don't understand exactly why
(the same local-nexus-hosted Solr jars were the ones being duplicated on
the classpath: included in my custom -with-dependencies jars as well as in
the solr war, which was build/distributed/and hosted from the same nexus
repo used to host my jars) but shading solr from my -with-dependencies jars
fixed the issue.
(if anybody could point me to reading on why this happened - e.g. the
classes on the classpath would be duplicated but identical, in
my naive understanding of the classloader this should have still just
worked - it would be appreciated)

Thanks again,
     Aaron

On Sat, Jun 9, 2012 at 2:40 PM, Jack Krupansky <[EMAIL PROTECTED]>wrote:

> Make sure there are no stray jars/classes in your jar, especially any that
> might contain BaseTokenizerFactory or TokenizerFactory. I notice that your
> jar name says "-with-dependencies", raising a little suspicion. The
> exception is as if your class was referring to a BaseTokenizerFactory,
> which implements TokenizerFactory, coming from your jar (or a contained
> jar) rather than getting resolved to Solr 3.6's own BaseTokenizerFactory
> and TokenizerFactory.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Aaron Daubman
> Sent: Saturday, June 09, 2012 12:03 AM
> To: [EMAIL PROTECTED]
> Subject: What would cause: "SEVERE: java.lang.ClassCastException:
> com.company.**MyCustomTokenizerFactory cannot be cast to
> org.apache.solr.analysis.**TokenizerFactory"
>
>
> Greetings,
>
> I am in the process of updating custom code and schema from Solr 1.4 to
> 3.6.0 and have run into the following issue with our two custom Tokenizer
> and Token Filter components.
>
> I've been banging my head against this one for far too long, especially
> since it must be something obvious I'm missing.
>
> I have  custom Tokenizer and Token Filter components along with
> corresponding factories. The code for all looks very similar to the
> Tokenizer and TokenFilter (and Factory) code that is standard with 3.6.0
> (and I have also read through
> http://wiki.apache.org/solr/**AnalyzersTokenizersTokenFilter**s<http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters>
>
> I have ensured my custom code is on the classpath, it is
> in ENSolrComponents-1.0-SNAPSHOT-**jar-with-dependencies.jar:
> ---output snip---
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.**CoreContainer load
> INFO: loading shared library: /opt/test_artists_solr/jetty-**solr/lib/en
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.**SolrResourceLoader
> replaceClassLoader
> INFO: Adding
> 'file:/opt/test_artists_solr/**jetty-solr/lib/en/**
> ENSolrComponents-1.0-SNAPSHOT-**jar-with-dependencies.jar'
> to classloader
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.**SolrResourceLoader
> replaceClassLoader
> INFO: Adding
> 'file:/opt/test_artists_solr/**jetty-solr/lib/en/ENUtil-1.0-**
> SNAPSHOT-jar-with-**dependencies.jar'
> to classloader
> Jun 8, 2012 10:41:00 PM org.apache.solr.core.**CoreContainer create
> --snip---
>
> After successfully parsing the schema and creating many fields, etc.. the
> following is logged:
> ---snip---
> Jun 8, 2012 10:41:00 PM org.apache.solr.util.plugin.**AbstractPluginLoader
> load
> INFO: created : com.company.**MyCustomTokenizerFactory
> Jun 8, 2012 10:41:00 PM org.apache.solr.common.**SolrException log
> SEVERE: java.lang.ClassCastException: com.company.**
> MyCustomTokenizerFactory
> cannot be cast to org.apache.solr.analysis.**TokenizerFactory
> at org.apache.solr.schema.**IndexSchema$5.init(**IndexSchema.java:966)
> at
> org.apache.solr.util.plugin.**AbstractPluginLoader.load(**
> AbstractPluginLoader.java:148)
> at org.apache.solr.schema.**IndexSchema.readAnalyzer(**
> IndexSchema.java:986)
> at org.apache.solr.schema.**IndexSchema.access$100(**IndexSchema.java:60)
> at org.apache.solr.schema.**IndexSchema$1.create(**IndexSchema.java:453)
> at org.apache.solr.schema.**IndexSchema$1.create(**IndexSchema.java:433)