Home | About | Sematext search-lucene.com search-hadoop.com
 Search Lucene and all its subprojects:

Switch to Threaded View
Solr, mail # user - Exception in DataImportHandler (stack overflow)
Jon Drukman
2012-05-15, 20:33
Michael Della Bitta
2012-05-15, 21:07
Jon Drukman
2012-05-15, 21:12
Jon Drukman
2012-05-15, 21:42
Jon Drukman
2012-05-15, 23:10
Shawn Heisey
2012-05-17, 20:55
Dyer, James
2012-05-17, 21:01
Shawn Heisey
2012-05-17, 21:47
Copy link to this message
-
RE: Exception in DataImportHandler (stack overflow)
Dyer, James 2012-05-15, 21:35
Shot in the dark here, but try adding readOnly="true" to your dataSource tag.

<dataSource readOnly="true" type="JdbcDataSource"  ... />

This sets autocommit to true and sets the Holdability to ResultSet.CLOSE_CURSORS_AT_COMMIT.  DIH does not explicitly close resultsets and maybe if your JDBC driver also manages this poorly you could end up with strange conditions like the one you're getting?  It could be a case where your data has grown just over the limit your setup can handle under such an unfortunate circumstance.

Let me know if this solves it.  If so, we probably should open a bug report and get this fixed in DIH.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311
-----Original Message-----
From: Jon Drukman [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 15, 2012 4:12 PM
To: [EMAIL PROTECTED]
Subject: Re: Exception in DataImportHandler (stack overflow)

i don't think so, my config is straightforward:

<dataConfig>
  <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
     url="jdbc:mysql://xxxxx/xxxxxx"
     user="xxxxx" password="xxxxx" batchSize="-1" />
  <document>
    <entity name="content"
       query="select content_id, description, title, add_date from
content_solr where active = '1'">
       <entity name="tag"
          query="select tag_id from tags_assoc where content_id '${content.content_id}'" />
       <entity name="likes"
          query="select count(1) as likes from votes where content_id '${content.content_id}'" />
       <entity name="views"
          query="select sum(views) as views from media_views mv join
content_media cm USING (media_id) WHERE cm.content_id '${content.content_id}'" />
    </entity>
  </document>
</dataConfig>

i'm triggering the import with:
http://localhost:8983/solr/dataimport?command=full-import&clean=true&commit=true

On Tue, May 15, 2012 at 2:07 PM, Michael Della Bitta <
[EMAIL PROTECTED]> wrote:

> Hi, Jon:
>
> Well, you don't see that every day!
>
> Is it possible that you have something weird going on in your DDL
> and/or queries, like a tree schema that now suddenly has a cyclical
> reference?
>
> Michael
>
> On Tue, May 15, 2012 at 4:33 PM, Jon Drukman <[EMAIL PROTECTED]> wrote:
> > I have a machine which does a full update using DataImportHandler every
> > hour.  It worked up until a little while ago.  I did not change the
> > dataconfig.xml or version of Solr.
> >
> > Here is the beginning of the error in the log (the real thing runs for
> > thousands of lines)
> >
> > 2012-05-15 12:44:30.724166500 SEVERE: Full Import
> > failed:org.apache.solr.handler.dataimport.DataImportHandlerException:
> > java.lang.StackOverflowError
> > 2012-05-15 12:44:30.724168500 at
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:669)
> > 2012-05-15 12:44:30.724169500 at
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
> > 2012-05-15 12:44:30.724171500 at
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
> > 2012-05-15 12:44:30.724219500 at
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
> > 2012-05-15 12:44:30.724221500 at
> >
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
> > 2012-05-15 12:44:30.724223500 at
> >
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
> > 2012-05-15 12:44:30.724224500 Caused by: java.lang.StackOverflowError
> > 2012-05-15 12:44:30.724225500 at
> > java.lang.String.checkBounds(String.java:404)
> > 2012-05-15 12:44:30.724234500 at java.lang.String.<init>(String.java:450)
> > 2012-05-15 12:44:30.724235500 at java.lang.String.<init>(String.java:523)
> > 2012-05-15 12:44:30.724236500 at
> > java.net.SocketOutputStream.socketWrite0(Native Method)
> > 2012-05-15 12:44:30.724238500 at
> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> > 2012-05-15 12:44:30.724239500 at
> > java.net.SocketOutputStream.write(SocketOutputStream.java:153)