Hello Rushikesh,

Are you using Nutch 1.3 or Nutch 1.13? If you're using Nutch 1.13, then you
could use the Tika boilerpipe implementation, on the nutch-site.xml you
need to enable this feature with:

<property>
  <name>tika.extractor</name>
  <value>boilerpipe</value>
  <description>
  Which text extraction algorithm to use. Valid values are: boilerpipe or
none.
  </description>
</property>

And configure the proper extractor with
the tika.extractor.boilerpipe.algorithm setting.

This is not a perfect solution, but I've used it successfully in the past,
of course, your results will depend on how is the structure (markup of the
website).

Other option could be to implement your own parser if you need to have more
control over what to include/exclude from the HTML. You can take a look at
this issue https://issues.apache.org/jira/browse/NUTCH-585 which contains
some info and old patches.

Best Regards,
Jorge

On Mon, Nov 13, 2017 at 8:58 PM Rushikesh K <[EMAIL PROTECTED]>
wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB