Hi,

Is it posible to know how long it takes to download an amount of data with
nutch.

I want to know how long it took to download the first 5 G.

Isidro

--
View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
Sent from the Nutch - User mailing list archive at Nabble.com.

Hi,

Unless you monitor the counters of a job while it's running: no.
However, you could, in theory, replay the fetch/download by looking at the fetch times, sum the content size and see when it hits the 5G. But you have to write your own tool for that.

Mathijs

On 3 aug. 2012, at 03:13, isidro <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Is it posible to know how long it takes to download an amount of data with
> nutch.
>
> I want to know how long it took to download the first 5 G.
>
> Isidro
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
> Sent from the Nutch - User mailing list archive at Nabble.com.

Hi,

Where can I get the content size and the fetch times for each fetched file ?

Isidro
On Thu, Aug 2, 2012 at 11:49 PM, Mathijs Homminga-3 [via Lucene] <
ml-node+[EMAIL PROTECTED]> wrote:

> Hi,
>
> Unless you monitor the counters of a job while it's running: no.
> However, you could, in theory, replay the fetch/download by looking at the
> fetch times, sum the content size and see when it hits the 5G. But you have
> to write your own tool for that.
>
> Mathijs
>
> On 3 aug. 2012, at 03:13, isidro <[hidden email]<http://user/SendEmail.jtp?type=node&node=3998947&i=0>>
> wrote:
>
> > Hi,
> >
> > Is it posible to know how long it takes to download an amount of data
> with
> > nutch.
> >
> > I want to know how long it took to download the first 5 G.
> >
> > Isidro
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
> > Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3998947.html
>  To unsubscribe from Is it posible to know how long it takes to download
> an amount of data with nutch., click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3998936&code=aXNpZHJvc2FAZ21haWwuY29tfDM5OTg5MzZ8MjAzMTY0OTM3MQ==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999163.html
Sent from the Nutch - User mailing list archive at Nabble.com.

What version of Nutch are you using?

On Aug 4, 2012, at 5:36 , isidro <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Where can I get the content size and the fetch times for each fetched file ?
>
> Isidro
>
>
> On Thu, Aug 2, 2012 at 11:49 PM, Mathijs Homminga-3 [via Lucene] <
> ml-node+[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Unless you monitor the counters of a job while it's running: no.
>> However, you could, in theory, replay the fetch/download by looking at the
>> fetch times, sum the content size and see when it hits the 5G. But you have
>> to write your own tool for that.
>>
>> Mathijs
>>
>> On 3 aug. 2012, at 03:13, isidro <[hidden email]<http://user/SendEmail.jtp?type=node&node=3998947&i=0>>
>> wrote:
>>
>>> Hi,
>>>
>>> Is it posible to know how long it takes to download an amount of data
>> with
>>> nutch.
>>>
>>> I want to know how long it took to download the first 5 G.
>>>
>>> Isidro
>>>
>>>
>>>
>>> --
>>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
>>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3998947.html
>> To unsubscribe from Is it posible to know how long it takes to download
>> an amount of data with nutch., click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3998936&code=aXNpZHJvc2FAZ21haWwuY29tfDM5OTg5MzZ8MjAzMTY0OTM3MQ==>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999163.html
> Sent from the Nutch - User mailing list archive at Nabble.com.

Release 1.5.1 - 07/02/2012

On Fri, Aug 3, 2012 at 10:53 PM, Mathijs Homminga-3 [via Lucene] <
ml-node+[EMAIL PROTECTED]> wrote:

> What version of Nutch are you using?
>
> On Aug 4, 2012, at 5:36 , isidro <[hidden email]<http://user/SendEmail.jtp?type=node&node=3999165&i=0>>
> wrote:
>
> > Hi,
> >
> > Where can I get the content size and the fetch times for each fetched
> file ?
> >
> > Isidro
> >
> >
> > On Thu, Aug 2, 2012 at 11:49 PM, Mathijs Homminga-3 [via Lucene] <
> > [hidden email] <http://user/SendEmail.jtp?type=node&node=3999165&i=1>>
> wrote:
> >
> >> Hi,
> >>
> >> Unless you monitor the counters of a job while it's running: no.
> >> However, you could, in theory, replay the fetch/download by looking at
> the
> >> fetch times, sum the content size and see when it hits the 5G. But you
> have
> >> to write your own tool for that.
> >>
> >> Mathijs
> >>
> >> On 3 aug. 2012, at 03:13, isidro <[hidden email]<
> http://user/SendEmail.jtp?type=node&node=3998947&i=0>>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Is it posible to know how long it takes to download an amount of data
> >> with
> >>> nutch.
> >>>
> >>> I want to know how long it took to download the first 5 G.
> >>>
> >>> Isidro
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
> >>> Sent from the Nutch - User mailing list archive at Nabble.com.
> >>
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >>
> >>
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3998947.html
> >> To unsubscribe from Is it posible to know how long it takes to download
> >> an amount of data with nutch., click here<
>
> >> .
> >> NAML<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> >>
> >
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999163.html
>
> > Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999165.html
>  To unsubscribe from Is it posible to know how long it takes to download
> an amount of data with nutch., click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3998936&code=aXNpZHJvc2FAZ21haWwuY29tfDM5OTg5MzZ8MjAzMTY0OTM3MQ==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999166.html
Sent from the Nutch - User mailing list archive at Nabble.com.

Hm, I'm not into segments these days, but I think you can use the SegmentReader to dump the rows from a segment. I guess you can take it from there.
On Aug 4, 2012, at 6:57 , isidro <[EMAIL PROTECTED]> wrote:

> Release 1.5.1 - 07/02/2012
>
> On Fri, Aug 3, 2012 at 10:53 PM, Mathijs Homminga-3 [via Lucene] <
> ml-node+[EMAIL PROTECTED]> wrote:
>
>> What version of Nutch are you using?
>>
>> On Aug 4, 2012, at 5:36 , isidro <[hidden email]<http://user/SendEmail.jtp?type=node&node=3999165&i=0>>
>> wrote:
>>
>>> Hi,
>>>
>>> Where can I get the content size and the fetch times for each fetched
>> file ?
>>>
>>> Isidro
>>>
>>>
>>> On Thu, Aug 2, 2012 at 11:49 PM, Mathijs Homminga-3 [via Lucene] <
>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3999165&i=1>>
>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Unless you monitor the counters of a job while it's running: no.
>>>> However, you could, in theory, replay the fetch/download by looking at
>> the
>>>> fetch times, sum the content size and see when it hits the 5G. But you
>> have
>>>> to write your own tool for that.
>>>>
>>>> Mathijs
>>>>
>>>> On 3 aug. 2012, at 03:13, isidro <[hidden email]<
>> http://user/SendEmail.jtp?type=node&node=3998947&i=0>>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Is it posible to know how long it takes to download an amount of data
>>>> with
>>>>> nutch.
>>>>>
>>>>> I want to know how long it took to download the first 5 G.
>>>>>
>>>>> Isidro
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936.html
>>>>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>> discussion
>>>> below:
>>>>
>>>>
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3998947.html
>>>> To unsubscribe from Is it posible to know how long it takes to download
>>>> an amount of data with nutch., click here<
>>
>>>> .
>>>> NAML<
>> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999163.html
>>
>>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999165.html
>> To unsubscribe from Is it posible to know how long it takes to download
>> an amount of data with nutch., click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3998936&code=aXNpZHJvc2FAZ21haWwuY29tfDM5OTg5MzZ8MjAzMTY0OTM3MQ==>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-it-posible-to-know-how-long-it-takes-to-download-an-amount-of-data-with-nutch-tp3998936p3999166.html
> Sent from the Nutch - User mailing list archive at Nabble.com.