Sudip Datta 2012-02-09, 07:26
-Re: Seed urls not getting crawled.
Lewis John Mcgibbney 2012-02-10, 21:00
On Thu, Feb 9, 2012 at 7:26 AM, Sudip Datta <[EMAIL PROTECTED]> wrote:
> While, this indicates that a reattempt will be made in 1 day, the
> 'url' never really gets the state db_fetched. On the other hand, if I
> set generate.max.count = -1, the page is indeed crawled but the crawl
> is painfully slow.
Do you have any idea about which part of the crawl is painfully slow?
How are you running your crawls?