Granted, I'm on a slow machine, but our tests take forever to run.  On an 2 core MBP, it takes well over an hour to run all the tests (I did just order a new MBP, so it will be faster, but it doesn't lend itself to a good OOTB experience for people)

One idea would be to add in parallel test execution in Maven.  I think this requires Mvn 3, but I am not sure.  Another is to take a look at our tests, especially the slow ones and see if we can speed them up.

When I try adding in parallel tests to Maven, I get a bunch of failures in the tests.

I was using:
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <forkMode>once</forkMode>
          <argLine>-Xms256m -Xmx512m</argLine>
          <testFailureIgnore>false</testFailureIgnore>
          <redirectTestOutputToFile>true</redirectTestOutputToFile>
          <parallel>classes</parallel>
          <threadCount>5</threadCount>
        </configuration>
      </plugin>

Anyone played around with this stuff?  I suspect the failures are due to tests stomping on each other, but I am still digging in.

-Grant

Comment out DistributedLanczosWhatsit. Zing!

A unit test takes a bit of code X and checks that code path A goes
"tick" and code path B goes "tock" and bogus input C throws an
exception. There's no such thing as a "unit test" that runs twelve M/R
jobs in a row.

There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
morass. This is a squib about how to do unit testing of mappers and
reducers with Mockito:

http://nubetech.co/testing-hadoop-map-reduce-jobs

What the Mahout jobs want is more of a regression test, which would
have two purposes:
1) does the whole orchestration still work, and
2) does it still acquire the information it is supposed to acquire?
2a) this requires some amount of real data and a "gold standard"
output to match against.

On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> Granted, I'm on a slow machine, but our tests take forever to run.  On an 2 core MBP, it takes well over an hour to run all the tests (I did just order a new MBP, so it will be faster, but it doesn't lend itself to a good OOTB experience for people)
>
> One idea would be to add in parallel test execution in Maven.  I think this requires Mvn 3, but I am not sure.  Another is to take a look at our tests, especially the slow ones and see if we can speed them up.
>
> When I try adding in parallel tests to Maven, I get a bunch of failures in the tests.
>
> I was using:
> <plugin>
>        <groupId>org.apache.maven.plugins</groupId>
>        <artifactId>maven-surefire-plugin</artifactId>
>        <configuration>
>          <forkMode>once</forkMode>
>          <argLine>-Xms256m -Xmx512m</argLine>
>          <testFailureIgnore>false</testFailureIgnore>
>          <redirectTestOutputToFile>true</redirectTestOutputToFile>
>          <parallel>classes</parallel>
>          <threadCount>5</threadCount>
>        </configuration>
>      </plugin>
>
> Anyone played around with this stuff?  I suspect the failures are due to tests stomping on each other, but I am still digging in.
>
> -Grant

--
Lance Norskog
[EMAIL PROTECTED]

This test is indeed by far the culprit. I already reduced its test input
size to hurry it up, but it's gone slow again.

Lance, indeed, these are not all unit tests -- nobody said they were. The
test is useful.

I do suggest, however, we comment it out. Jake suggested it coudl be made
faster but I don't think he followed up.

Sean

On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:

> Comment out DistributedLanczosWhatsit. Zing!
>
> A unit test takes a bit of code X and checks that code path A goes
> "tick" and code path B goes "tock" and bogus input C throws an
> exception. There's no such thing as a "unit test" that runs twelve M/R
> jobs in a row.
>
> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
> morass. This is a squib about how to do unit testing of mappers and
> reducers with Mockito:
>
> http://nubetech.co/testing-hadoop-map-reduce-jobs
>
> What the Mahout jobs want is more of a regression test, which would
> have two purposes:
> 1) does the whole orchestration still work, and
> 2) does it still acquire the information it is supposed to acquire?
> 2a) this requires some amount of real data and a "gold standard"
> output to match against.
>
> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
> wrote:
> > Granted, I'm on a slow machine, but our tests take forever to run.  On an
> 2 core MBP, it takes well over an hour to run all the tests (I did just
> order a new MBP, so it will be faster, but it doesn't lend itself to a good
> OOTB experience for people)
> >
> > One idea would be to add in parallel test execution in Maven.  I think
> this requires Mvn 3, but I am not sure.  Another is to take a look at our
> tests, especially the slow ones and see if we can speed them up.
> >
> > When I try adding in parallel tests to Maven, I get a bunch of failures
> in the tests.
> >
> > I was using:
> > <plugin>
> >        <groupId>org.apache.maven.plugins</groupId>
> >        <artifactId>maven-surefire-plugin</artifactId>
> >        <configuration>
> >          <forkMode>once</forkMode>
> >          <argLine>-Xms256m -Xmx512m</argLine>
> >          <testFailureIgnore>false</testFailureIgnore>
> >          <redirectTestOutputToFile>true</redirectTestOutputToFile>
> >          <parallel>classes</parallel>
> >          <threadCount>5</threadCount>
> >        </configuration>
> >      </plugin>
> >
> > Anyone played around with this stuff?  I suspect the failures are due to
> tests stomping on each other, but I am still digging in.
> >
> > -Grant
>
>
>
> --
> Lance Norskog
> [EMAIL PROTECTED]
>

Sure! Perhaps the long-running ones can move to a new 'regression'
area? examples/ is partly what these are, so examples/regression makes
sense.

On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
> This test is indeed by far the culprit. I already reduced its test input
> size to hurry it up, but it's gone slow again.
>
> Lance, indeed, these are not all unit tests -- nobody said they were. The
> test is useful.
>
> I do suggest, however, we comment it out. Jake suggested it coudl be made
> faster but I don't think he followed up.
>
> Sean
>
> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>
>> Comment out DistributedLanczosWhatsit. Zing!
>>
>> A unit test takes a bit of code X and checks that code path A goes
>> "tick" and code path B goes "tock" and bogus input C throws an
>> exception. There's no such thing as a "unit test" that runs twelve M/R
>> jobs in a row.
>>
>> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
>> morass. This is a squib about how to do unit testing of mappers and
>> reducers with Mockito:
>>
>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>>
>> What the Mahout jobs want is more of a regression test, which would
>> have two purposes:
>> 1) does the whole orchestration still work, and
>> 2) does it still acquire the information it is supposed to acquire?
>> 2a) this requires some amount of real data and a "gold standard"
>> output to match against.
>>
>> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>> wrote:
>> > Granted, I'm on a slow machine, but our tests take forever to run.  On an
>> 2 core MBP, it takes well over an hour to run all the tests (I did just
>> order a new MBP, so it will be faster, but it doesn't lend itself to a good
>> OOTB experience for people)
>> >
>> > One idea would be to add in parallel test execution in Maven.  I think
>> this requires Mvn 3, but I am not sure.  Another is to take a look at our
>> tests, especially the slow ones and see if we can speed them up.
>> >
>> > When I try adding in parallel tests to Maven, I get a bunch of failures
>> in the tests.
>> >
>> > I was using:
>> > <plugin>
>> >        <groupId>org.apache.maven.plugins</groupId>
>> >        <artifactId>maven-surefire-plugin</artifactId>
>> >        <configuration>
>> >          <forkMode>once</forkMode>
>> >          <argLine>-Xms256m -Xmx512m</argLine>
>> >          <testFailureIgnore>false</testFailureIgnore>
>> >          <redirectTestOutputToFile>true</redirectTestOutputToFile>
>> >          <parallel>classes</parallel>
>> >          <threadCount>5</threadCount>
>> >        </configuration>
>> >      </plugin>
>> >
>> > Anyone played around with this stuff?  I suspect the failures are due to
>> tests stomping on each other, but I am still digging in.
>> >
>> > -Grant
>>
>>
>>
>> --
>> Lance Norskog
>> [EMAIL PROTECTED]
>>
>

--
Lance Norskog
[EMAIL PROTECTED]

I really don't think that this distinction needs to be made.  The
distinction between unit and integration test is important from a technical
point of view, but as an organizing principle the topic and target of the
test is probably a better idea than whether the test is a functional or unit
test or whether it has randomized initial conditions or whether it has for
loops in it.  Tests should be organized by what they test.

On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:

> Sure! Perhaps the long-running ones can move to a new 'regression'
> area? examples/ is partly what these are, so examples/regression makes
> sense.
>
> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
> > This test is indeed by far the culprit. I already reduced its test input
> > size to hurry it up, but it's gone slow again.
> >
> > Lance, indeed, these are not all unit tests -- nobody said they were. The
> > test is useful.
> >
> > I do suggest, however, we comment it out. Jake suggested it coudl be made
> > faster but I don't think he followed up.
> >
> > Sean
> >
> > On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
> wrote:
> >
> >> Comment out DistributedLanczosWhatsit. Zing!
> >>
> >> A unit test takes a bit of code X and checks that code path A goes
> >> "tick" and code path B goes "tock" and bogus input C throws an
> >> exception. There's no such thing as a "unit test" that runs twelve M/R
> >> jobs in a row.
> >>
> >> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
> >> morass. This is a squib about how to do unit testing of mappers and
> >> reducers with Mockito:
> >>
> >> http://nubetech.co/testing-hadoop-map-reduce-jobs
> >>
> >> What the Mahout jobs want is more of a regression test, which would
> >> have two purposes:
> >> 1) does the whole orchestration still work, and
> >> 2) does it still acquire the information it is supposed to acquire?
> >> 2a) this requires some amount of real data and a "gold standard"
> >> output to match against.
> >>
> >> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
> >> wrote:
> >> > Granted, I'm on a slow machine, but our tests take forever to run.  On
> an
> >> 2 core MBP, it takes well over an hour to run all the tests (I did just
> >> order a new MBP, so it will be faster, but it doesn't lend itself to a
> good
> >> OOTB experience for people)
> >> >
> >> > One idea would be to add in parallel test execution in Maven.  I think
> >> this requires Mvn 3, but I am not sure.  Another is to take a look at
> our
> >> tests, especially the slow ones and see if we can speed them up.
> >> >
> >> > When I try adding in parallel tests to Maven, I get a bunch of
> failures
> >> in the tests.
> >> >
> >> > I was using:
> >> > <plugin>
> >> >        <groupId>org.apache.maven.plugins</groupId>
> >> >        <artifactId>maven-surefire-plugin</artifactId>
> >> >        <configuration>
> >> >          <forkMode>once</forkMode>
> >> >          <argLine>-Xms256m -Xmx512m</argLine>
> >> >          <testFailureIgnore>false</testFailureIgnore>
> >> >          <redirectTestOutputToFile>true</redirectTestOutputToFile>
> >> >          <parallel>classes</parallel>
> >> >          <threadCount>5</threadCount>
> >> >        </configuration>
> >> >      </plugin>
> >> >
> >> > Anyone played around with this stuff?  I suspect the failures are due
> to
> >> tests stomping on each other, but I am still digging in.
> >> >
> >> > -Grant
> >>
> >>
> >>
> >> --
> >> Lance Norskog
> >> [EMAIL PROTECTED]
> >>
> >
>
>
>
> --
> Lance Norskog
> [EMAIL PROTECTED]
>

It would likely help if we could get them to run in parallel, perhaps.  Also, seems like TestNG might have some better features on paper for this kind of stuff (I think you can annotate some things as "slow" or "fast" and then choose to run them separately).  I haven't explored much yet in this way.  Has anyone else used TestNG?

-Grant

On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:

> I really don't think that this distinction needs to be made.  The
> distinction between unit and integration test is important from a technical
> point of view, but as an organizing principle the topic and target of the
> test is probably a better idea than whether the test is a functional or unit
> test or whether it has randomized initial conditions or whether it has for
> loops in it.  Tests should be organized by what they test.
>
> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>
>> Sure! Perhaps the long-running ones can move to a new 'regression'
>> area? examples/ is partly what these are, so examples/regression makes
>> sense.
>>
>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
>>> This test is indeed by far the culprit. I already reduced its test input
>>> size to hurry it up, but it's gone slow again.
>>>
>>> Lance, indeed, these are not all unit tests -- nobody said they were. The
>>> test is useful.
>>>
>>> I do suggest, however, we comment it out. Jake suggested it coudl be made
>>> faster but I don't think he followed up.
>>>
>>> Sean
>>>
>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> Comment out DistributedLanczosWhatsit. Zing!
>>>>
>>>> A unit test takes a bit of code X and checks that code path A goes
>>>> "tick" and code path B goes "tock" and bogus input C throws an
>>>> exception. There's no such thing as a "unit test" that runs twelve M/R
>>>> jobs in a row.
>>>>
>>>> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
>>>> morass. This is a squib about how to do unit testing of mappers and
>>>> reducers with Mockito:
>>>>
>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>>>>
>>>> What the Mahout jobs want is more of a regression test, which would
>>>> have two purposes:
>>>> 1) does the whole orchestration still work, and
>>>> 2) does it still acquire the information it is supposed to acquire?
>>>> 2a) this requires some amount of real data and a "gold standard"
>>>> output to match against.
>>>>
>>>> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>>>> wrote:
>>>>> Granted, I'm on a slow machine, but our tests take forever to run.  On
>> an
>>>> 2 core MBP, it takes well over an hour to run all the tests (I did just
>>>> order a new MBP, so it will be faster, but it doesn't lend itself to a
>> good
>>>> OOTB experience for people)
>>>>>
>>>>> One idea would be to add in parallel test execution in Maven.  I think
>>>> this requires Mvn 3, but I am not sure.  Another is to take a look at
>> our
>>>> tests, especially the slow ones and see if we can speed them up.
>>>>>
>>>>> When I try adding in parallel tests to Maven, I get a bunch of
>> failures
>>>> in the tests.
>>>>>
>>>>> I was using:
>>>>> <plugin>
>>>>>       <groupId>org.apache.maven.plugins</groupId>
>>>>>       <artifactId>maven-surefire-plugin</artifactId>
>>>>>       <configuration>
>>>>>         <forkMode>once</forkMode>
>>>>>         <argLine>-Xms256m -Xmx512m</argLine>
>>>>>         <testFailureIgnore>false</testFailureIgnore>
>>>>>         <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>>         <parallel>classes</parallel>
>>>>>         <threadCount>5</threadCount>
>>>>>       </configuration>
>>>>>     </plugin>
>>>>>
>>>>> Anyone played around with this stuff?  I suspect the failures are due
>> to
>>>> tests stomping on each other, but I am still digging in.
>>>>>
>>>>> -Grant
>>>>
>>>>
>>>>
>>>> --
>>>> Lance Norskog
>>>> [EMAIL PROTECTED]
>>>>
>>>
>>
>>
>>
>> --
>> Lance Norskog

Grant Ingersoll

We have used testNg at work for just this segregation of fast and slow tests and it works pretty well. It is also useful for stripping out cascading failures. This means that integration tests can be executed only if the underlying unit tests succeed. Another nice thing is that testNg says how many tests were skipped.

Junit has recently been adding similar capabilities but I haven't kept up.

Sent from my iPhone

On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:

> It would likely help if we could get them to run in parallel, perhaps.  Also, seems like TestNG might have some better features on paper for this kind of stuff (I think you can annotate some things as "slow" or "fast" and then choose to run them separately).  I haven't explored much yet in this way.  Has anyone else used TestNG?
>
> -Grant
>
> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
>
>> I really don't think that this distinction needs to be made.  The
>> distinction between unit and integration test is important from a technical
>> point of view, but as an organizing principle the topic and target of the
>> test is probably a better idea than whether the test is a functional or unit
>> test or whether it has randomized initial conditions or whether it has for
>> loops in it.  Tests should be organized by what they test.
>>
>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>>
>>> Sure! Perhaps the long-running ones can move to a new 'regression'
>>> area? examples/ is partly what these are, so examples/regression makes
>>> sense.
>>>
>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
>>>> This test is indeed by far the culprit. I already reduced its test input
>>>> size to hurry it up, but it's gone slow again.
>>>>
>>>> Lance, indeed, these are not all unit tests -- nobody said they were. The
>>>> test is useful.
>>>>
>>>> I do suggest, however, we comment it out. Jake suggested it coudl be made
>>>> faster but I don't think he followed up.
>>>>
>>>> Sean
>>>>
>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>>> Comment out DistributedLanczosWhatsit. Zing!
>>>>>
>>>>> A unit test takes a bit of code X and checks that code path A goes
>>>>> "tick" and code path B goes "tock" and bogus input C throws an
>>>>> exception. There's no such thing as a "unit test" that runs twelve M/R
>>>>> jobs in a row.
>>>>>
>>>>> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
>>>>> morass. This is a squib about how to do unit testing of mappers and
>>>>> reducers with Mockito:
>>>>>
>>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>>>>>
>>>>> What the Mahout jobs want is more of a regression test, which would
>>>>> have two purposes:
>>>>> 1) does the whole orchestration still work, and
>>>>> 2) does it still acquire the information it is supposed to acquire?
>>>>> 2a) this requires some amount of real data and a "gold standard"
>>>>> output to match against.
>>>>>
>>>>> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>> Granted, I'm on a slow machine, but our tests take forever to run.  On
>>> an
>>>>> 2 core MBP, it takes well over an hour to run all the tests (I did just
>>>>> order a new MBP, so it will be faster, but it doesn't lend itself to a
>>> good
>>>>> OOTB experience for people)
>>>>>>
>>>>>> One idea would be to add in parallel test execution in Maven.  I think
>>>>> this requires Mvn 3, but I am not sure.  Another is to take a look at
>>> our
>>>>> tests, especially the slow ones and see if we can speed them up.
>>>>>>
>>>>>> When I try adding in parallel tests to Maven, I get a bunch of
>>> failures
>>>>> in the tests.
>>>>>>
>>>>>> I was using:
>>>>>> <plugin>
>>>>>>      <groupId>org.apache.maven.plugins</groupId>
>>>>>>      <artifactId>maven-surefire-plugin</artifactId>
>>>>>>      <configuration>
>>>>>>        <forkMode>once</forkMode>
>>>>>>        <argLine>-Xms256m -Xmx512m</argLine>

Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
should be able to run in a fairly constant amount of ram per JVM from
beginning to end, to avoid blowing up late in the game. Is there any
harness around that would do this?

Lance

On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
> We have used testNg at work for just this segregation of fast and slow tests and it works pretty well. It is also useful for stripping out cascading failures. This means that integration tests can be executed only if the underlying unit tests succeed. Another nice thing is that testNg says how many tests were skipped.
>
> Junit has recently been adding similar capabilities but I haven't kept up.
>
> Sent from my iPhone
>
> On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
>> It would likely help if we could get them to run in parallel, perhaps.  Also, seems like TestNG might have some better features on paper for this kind of stuff (I think you can annotate some things as "slow" or "fast" and then choose to run them separately).  I haven't explored much yet in this way.  Has anyone else used TestNG?
>>
>> -Grant
>>
>> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
>>
>>> I really don't think that this distinction needs to be made.  The
>>> distinction between unit and integration test is important from a technical
>>> point of view, but as an organizing principle the topic and target of the
>>> test is probably a better idea than whether the test is a functional or unit
>>> test or whether it has randomized initial conditions or whether it has for
>>> loops in it.  Tests should be organized by what they test.
>>>
>>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>>>
>>>> Sure! Perhaps the long-running ones can move to a new 'regression'
>>>> area? examples/ is partly what these are, so examples/regression makes
>>>> sense.
>>>>
>>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
>>>>> This test is indeed by far the culprit. I already reduced its test input
>>>>> size to hurry it up, but it's gone slow again.
>>>>>
>>>>> Lance, indeed, these are not all unit tests -- nobody said they were. The
>>>>> test is useful.
>>>>>
>>>>> I do suggest, however, we comment it out. Jake suggested it coudl be made
>>>>> faster but I don't think he followed up.
>>>>>
>>>>> Sean
>>>>>
>>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>
>>>>>> Comment out DistributedLanczosWhatsit. Zing!
>>>>>>
>>>>>> A unit test takes a bit of code X and checks that code path A goes
>>>>>> "tick" and code path B goes "tock" and bogus input C throws an
>>>>>> exception. There's no such thing as a "unit test" that runs twelve M/R
>>>>>> jobs in a row.
>>>>>>
>>>>>> There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23
>>>>>> morass. This is a squib about how to do unit testing of mappers and
>>>>>> reducers with Mockito:
>>>>>>
>>>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>>>>>>
>>>>>> What the Mahout jobs want is more of a regression test, which would
>>>>>> have two purposes:
>>>>>> 1) does the whole orchestration still work, and
>>>>>> 2) does it still acquire the information it is supposed to acquire?
>>>>>> 2a) this requires some amount of real data and a "gold standard"
>>>>>> output to match against.
>>>>>>
>>>>>> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>>>>>> wrote:
>>>>>>> Granted, I'm on a slow machine, but our tests take forever to run.  On
>>>> an
>>>>>> 2 core MBP, it takes well over an hour to run all the tests (I did just
>>>>>> order a new MBP, so it will be faster, but it doesn't lend itself to a
>>>> good
>>>>>> OOTB experience for people)
>>>>>>>
>>>>>>> One idea would be to add in parallel test execution in Maven.  I think
>>>>>> this requires Mvn 3, but I am not sure.  Another is to take a look at
>>>> our
>>>>>> tests, especially the slow ones and see if we can speed them up.

Lance Norskog
[EMAIL PROTECTED]

That would reduce memory requirements unilaterally? no, don't think so. I
have not run into memory problems.
The issue here is execution time and it's a big problem indeed.

Would anyone object to disabling this test for now? it's getting costly in
the dev cycle.

On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:

> Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
> should be able to run in a fairly constant amount of ram per JVM from
> beginning to end, to avoid blowing up late in the game. Is there any
> harness around that would do this?
>
> Lance
>
> On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
> > We have used testNg at work for just this segregation of fast and slow
> tests and it works pretty well. It is also useful for stripping out
> cascading failures. This means that integration tests can be executed only
> if the underlying unit tests succeed. Another nice thing is that testNg says
> how many tests were skipped.
> >
> > Junit has recently been adding similar capabilities but I haven't kept
> up.
> >
> > Sent from my iPhone
> >
> > On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> >
> >> It would likely help if we could get them to run in parallel, perhaps.
>  Also, seems like TestNG might have some better features on paper for this
> kind of stuff (I think you can annotate some things as "slow" or "fast" and
> then choose to run them separately).  I haven't explored much yet in this
> way.  Has anyone else used TestNG?
> >>
> >> -Grant
> >>
> >> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
> >>
> >>> I really don't think that this distinction needs to be made.  The
> >>> distinction between unit and integration test is important from a
> technical
> >>> point of view, but as an organizing principle the topic and target of
> the
> >>> test is probably a better idea than whether the test is a functional or
> unit
> >>> test or whether it has randomized initial conditions or whether it has
> for
> >>> loops in it.  Tests should be organized by what they test.
> >>>
> >>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> Sure! Perhaps the long-running ones can move to a new 'regression'
> >>>> area? examples/ is partly what these are, so examples/regression makes
> >>>> sense.
> >>>>
> >>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
> >>>>> This test is indeed by far the culprit. I already reduced its test
> input
> >>>>> size to hurry it up, but it's gone slow again.
> >>>>>
> >>>>> Lance, indeed, these are not all unit tests -- nobody said they were.
> The
> >>>>> test is useful.
> >>>>>
> >>>>> I do suggest, however, we comment it out. Jake suggested it coudl be
> made
> >>>>> faster but I don't think he followed up.
> >>>>>
> >>>>> Sean
> >>>>>
> >>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
> >>>> wrote:
> >>>>>
> >>>>>> Comment out DistributedLanczosWhatsit. Zing!
> >>>>>>
> >>>>>> A unit test takes a bit of code X and checks that code path A goes
> >>>>>> "tick" and code path B goes "tock" and bogus input C throws an
> >>>>>> exception. There's no such thing as a "unit test" that runs twelve
> M/R
> >>>>>> jobs in a row.
> >>>>>>
> >>>>>> There's MRUnit, which seems trapped in the Hadoop
> 0.20/0.21/0.22/0.23
> >>>>>> morass. This is a squib about how to do unit testing of mappers and
> >>>>>> reducers with Mockito:
> >>>>>>
> >>>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
> >>>>>>
> >>>>>> What the Mahout jobs want is more of a regression test, which would
> >>>>>> have two purposes:
> >>>>>> 1) does the whole orchestration still work, and
> >>>>>> 2) does it still acquire the information it is supposed to acquire?
> >>>>>> 2a) this requires some amount of real data and a "gold standard"
> >>>>>> output to match against.
> >>>>>>
> >>>>>> On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <
> [EMAIL PROTECTED]>
> >>>>>> wrote:

It would be nice if it could still run somewhere, even if it isn't run every time locally.

-Grant

On Aug 11, 2011, at 2:54 AM, Sean Owen wrote:

> That would reduce memory requirements unilaterally? no, don't think so. I
> have not run into memory problems.
> The issue here is execution time and it's a big problem indeed.
>
> Would anyone object to disabling this test for now? it's getting costly in
> the dev cycle.
>
> On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>
>> Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
>> should be able to run in a fairly constant amount of ram per JVM from
>> beginning to end, to avoid blowing up late in the game. Is there any
>> harness around that would do this?
>>
>> Lance
>>
>> On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
>>> We have used testNg at work for just this segregation of fast and slow
>> tests and it works pretty well. It is also useful for stripping out
>> cascading failures. This means that integration tests can be executed only
>> if the underlying unit tests succeed. Another nice thing is that testNg says
>> how many tests were skipped.
>>>
>>> Junit has recently been adding similar capabilities but I haven't kept
>> up.
>>>
>>> Sent from my iPhone
>>>
>>> On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>>>
>>>> It would likely help if we could get them to run in parallel, perhaps.
>> Also, seems like TestNG might have some better features on paper for this
>> kind of stuff (I think you can annotate some things as "slow" or "fast" and
>> then choose to run them separately).  I haven't explored much yet in this
>> way.  Has anyone else used TestNG?
>>>>
>>>> -Grant
>>>>
>>>> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
>>>>
>>>>> I really don't think that this distinction needs to be made.  The
>>>>> distinction between unit and integration test is important from a
>> technical
>>>>> point of view, but as an organizing principle the topic and target of
>> the
>>>>> test is probably a better idea than whether the test is a functional or
>> unit
>>>>> test or whether it has randomized initial conditions or whether it has
>> for
>>>>> loops in it.  Tests should be organized by what they test.
>>>>>
>>>>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]>
>> wrote:
>>>>>
>>>>>> Sure! Perhaps the long-running ones can move to a new 'regression'
>>>>>> area? examples/ is partly what these are, so examples/regression makes
>>>>>> sense.
>>>>>>
>>>>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
>>>>>>> This test is indeed by far the culprit. I already reduced its test
>> input
>>>>>>> size to hurry it up, but it's gone slow again.
>>>>>>>
>>>>>>> Lance, indeed, these are not all unit tests -- nobody said they were.
>> The
>>>>>>> test is useful.
>>>>>>>
>>>>>>> I do suggest, however, we comment it out. Jake suggested it coudl be
>> made
>>>>>>> faster but I don't think he followed up.
>>>>>>>
>>>>>>> Sean
>>>>>>>
>>>>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Comment out DistributedLanczosWhatsit. Zing!
>>>>>>>>
>>>>>>>> A unit test takes a bit of code X and checks that code path A goes
>>>>>>>> "tick" and code path B goes "tock" and bogus input C throws an
>>>>>>>> exception. There's no such thing as a "unit test" that runs twelve
>> M/R
>>>>>>>> jobs in a row.
>>>>>>>>
>>>>>>>> There's MRUnit, which seems trapped in the Hadoop
>> 0.20/0.21/0.22/0.23
>>>>>>>> morass. This is a squib about how to do unit testing of mappers and
>>>>>>>> reducers with Mockito:
>>>>>>>>
>>>>>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>>>>>>>>
>>>>>>>> What the Mahout jobs want is more of a regression test, which would
>>>>>>>> have two purposes:
>>>>>>>> 1) does the whole orchestration still work, and
>>>>>>>> 2) does it still acquire the information it is supposed to acquire?

Grant Ingersoll

Oops, different context: I meant about automated testing of the mahout
multi-stage jobs.  If one of the stages gets unruly in its memory use,
that is a regression in software quality that is hard to spot.

On Wed, Aug 10, 2011 at 11:54 PM, Sean Owen <[EMAIL PROTECTED]> wrote:
> That would reduce memory requirements unilaterally? no, don't think so. I
> have not run into memory problems.
> The issue here is execution time and it's a big problem indeed.
>
> Would anyone object to disabling this test for now? it's getting costly in
> the dev cycle.
>
> On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>
>> Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
>> should be able to run in a fairly constant amount of ram per JVM from
>> beginning to end, to avoid blowing up late in the game. Is there any
>> harness around that would do this?
>>
>> Lance
>>
>> On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
>> > We have used testNg at work for just this segregation of fast and slow
>> tests and it works pretty well. It is also useful for stripping out
>> cascading failures. This means that integration tests can be executed only
>> if the underlying unit tests succeed. Another nice thing is that testNg says
>> how many tests were skipped.
>> >
>> > Junit has recently been adding similar capabilities but I haven't kept
>> up.
>> >
>> > Sent from my iPhone
>> >
>> > On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>> >
>> >> It would likely help if we could get them to run in parallel, perhaps.
>>  Also, seems like TestNG might have some better features on paper for this
>> kind of stuff (I think you can annotate some things as "slow" or "fast" and
>> then choose to run them separately).  I haven't explored much yet in this
>> way.  Has anyone else used TestNG?
>> >>
>> >> -Grant
>> >>
>> >> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
>> >>
>> >>> I really don't think that this distinction needs to be made.  The
>> >>> distinction between unit and integration test is important from a
>> technical
>> >>> point of view, but as an organizing principle the topic and target of
>> the
>> >>> test is probably a better idea than whether the test is a functional or
>> unit
>> >>> test or whether it has randomized initial conditions or whether it has
>> for
>> >>> loops in it.  Tests should be organized by what they test.
>> >>>
>> >>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]>
>> wrote:
>> >>>
>> >>>> Sure! Perhaps the long-running ones can move to a new 'regression'
>> >>>> area? examples/ is partly what these are, so examples/regression makes
>> >>>> sense.
>> >>>>
>> >>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]> wrote:
>> >>>>> This test is indeed by far the culprit. I already reduced its test
>> input
>> >>>>> size to hurry it up, but it's gone slow again.
>> >>>>>
>> >>>>> Lance, indeed, these are not all unit tests -- nobody said they were.
>> The
>> >>>>> test is useful.
>> >>>>>
>> >>>>> I do suggest, however, we comment it out. Jake suggested it coudl be
>> made
>> >>>>> faster but I don't think he followed up.
>> >>>>>
>> >>>>> Sean
>> >>>>>
>> >>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Comment out DistributedLanczosWhatsit. Zing!
>> >>>>>>
>> >>>>>> A unit test takes a bit of code X and checks that code path A goes
>> >>>>>> "tick" and code path B goes "tock" and bogus input C throws an
>> >>>>>> exception. There's no such thing as a "unit test" that runs twelve
>> M/R
>> >>>>>> jobs in a row.
>> >>>>>>
>> >>>>>> There's MRUnit, which seems trapped in the Hadoop
>> 0.20/0.21/0.22/0.23
>> >>>>>> morass. This is a squib about how to do unit testing of mappers and
>> >>>>>> reducers with Mockito:
>> >>>>>>
>> >>>>>> http://nubetech.co/testing-hadoop-map-reduce-jobs
>> >>>>>>
>> >>>>>> What the Mahout jobs want is more of a regression test, which would

Lance Norskog
[EMAIL PROTECTED]

As long as nobody's changing the logic going into Lanczos, disabling it
should be fine.  But I agree with Grant, that we should have it running
somewhere (continuous integration?).

I can try and squeeze it's size down, but you don't get very good
convergence
on a small matrix when using these iterative matrix multiplier-style
algorithms,
which makes it hard to see that it's still working as expected.

  -jake

On Wed, Aug 10, 2011 at 11:54 PM, Sean Owen <[EMAIL PROTECTED]> wrote:

> That would reduce memory requirements unilaterally? no, don't think so. I
> have not run into memory problems.
> The issue here is execution time and it's a big problem indeed.
>
> Would anyone object to disabling this test for now? it's getting costly in
> the dev cycle.
>
> On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:
>
> > Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
> > should be able to run in a fairly constant amount of ram per JVM from
> > beginning to end, to avoid blowing up late in the game. Is there any
> > harness around that would do this?
> >
> > Lance
> >
> > On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
> > > We have used testNg at work for just this segregation of fast and slow
> > tests and it works pretty well. It is also useful for stripping out
> > cascading failures. This means that integration tests can be executed
> only
> > if the underlying unit tests succeed. Another nice thing is that testNg
> says
> > how many tests were skipped.
> > >
> > > Junit has recently been adding similar capabilities but I haven't kept
> > up.
> > >
> > > Sent from my iPhone
> > >
> > > On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> > >
> > >> It would likely help if we could get them to run in parallel, perhaps.
> >  Also, seems like TestNG might have some better features on paper for
> this
> > kind of stuff (I think you can annotate some things as "slow" or "fast"
> and
> > then choose to run them separately).  I haven't explored much yet in this
> > way.  Has anyone else used TestNG?
> > >>
> > >> -Grant
> > >>
> > >> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
> > >>
> > >>> I really don't think that this distinction needs to be made.  The
> > >>> distinction between unit and integration test is important from a
> > technical
> > >>> point of view, but as an organizing principle the topic and target of
> > the
> > >>> test is probably a better idea than whether the test is a functional
> or
> > unit
> > >>> test or whether it has randomized initial conditions or whether it
> has
> > for
> > >>> loops in it.  Tests should be organized by what they test.
> > >>>
> > >>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]>
> > wrote:
> > >>>
> > >>>> Sure! Perhaps the long-running ones can move to a new 'regression'
> > >>>> area? examples/ is partly what these are, so examples/regression
> makes
> > >>>> sense.
> > >>>>
> > >>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]>
> wrote:
> > >>>>> This test is indeed by far the culprit. I already reduced its test
> > input
> > >>>>> size to hurry it up, but it's gone slow again.
> > >>>>>
> > >>>>> Lance, indeed, these are not all unit tests -- nobody said they
> were.
> > The
> > >>>>> test is useful.
> > >>>>>
> > >>>>> I do suggest, however, we comment it out. Jake suggested it coudl
> be
> > made
> > >>>>> faster but I don't think he followed up.
> > >>>>>
> > >>>>> Sean
> > >>>>>
> > >>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[EMAIL PROTECTED]>
> > >>>> wrote:
> > >>>>>
> > >>>>>> Comment out DistributedLanczosWhatsit. Zing!
> > >>>>>>
> > >>>>>> A unit test takes a bit of code X and checks that code path A goes
> > >>>>>> "tick" and code path B goes "tock" and bogus input C throws an
> > >>>>>> exception. There's no such thing as a "unit test" that runs twelve
> > M/R
> > >>>>>> jobs in a row.
> > >>>>>>
> > >>>>>> There's MRUnit, which seems trapped in the Hadoop

(I disabled it. I had already tried to squeeze down its size without making
it fail. I will write a note on the back of my hand to make sure to enable
it before, say, releasing again.)

On Tue, Aug 16, 2011 at 7:16 PM, Jake Mannix <[EMAIL PROTECTED]> wrote:

> As long as nobody's changing the logic going into Lanczos, disabling it
> should be fine.  But I agree with Grant, that we should have it running
> somewhere (continuous integration?).
>
> I can try and squeeze it's size down, but you don't get very good
> convergence
> on a small matrix when using these iterative matrix multiplier-style
> algorithms,
> which makes it hard to see that it's still working as expected.
>
>  -jake
>
> On Wed, Aug 10, 2011 at 11:54 PM, Sean Owen <[EMAIL PROTECTED]> wrote:
>
> > That would reduce memory requirements unilaterally? no, don't think so. I
> > have not run into memory problems.
> > The issue here is execution time and it's a big problem indeed.
> >
> > Would anyone object to disabling this test for now? it's getting costly
> in
> > the dev cycle.
> >
> > On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]>
> wrote:
> >
> > > Another aspect of testing Map/Reduce jobs is memory bounds. An M/R job
> > > should be able to run in a fairly constant amount of ram per JVM from
> > > beginning to end, to avoid blowing up late in the game. Is there any
> > > harness around that would do this?
> > >
> > > Lance
> > >
> > > On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]>
> > wrote:
> > > > We have used testNg at work for just this segregation of fast and
> slow
> > > tests and it works pretty well. It is also useful for stripping out
> > > cascading failures. This means that integration tests can be executed
> > only
> > > if the underlying unit tests succeed. Another nice thing is that testNg
> > says
> > > how many tests were skipped.
> > > >
> > > > Junit has recently been adding similar capabilities but I haven't
> kept
> > > up.
> > > >
> > > > Sent from my iPhone
> > > >
> > > > On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > >> It would likely help if we could get them to run in parallel,
> perhaps.
> > >  Also, seems like TestNG might have some better features on paper for
> > this
> > > kind of stuff (I think you can annotate some things as "slow" or "fast"
> > and
> > > then choose to run them separately).  I haven't explored much yet in
> this
> > > way.  Has anyone else used TestNG?
> > > >>
> > > >> -Grant
> > > >>
> > > >> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
> > > >>
> > > >>> I really don't think that this distinction needs to be made.  The
> > > >>> distinction between unit and integration test is important from a
> > > technical
> > > >>> point of view, but as an organizing principle the topic and target
> of
> > > the
> > > >>> test is probably a better idea than whether the test is a
> functional
> > or
> > > unit
> > > >>> test or whether it has randomized initial conditions or whether it
> > has
> > > for
> > > >>> loops in it.  Tests should be organized by what they test.
> > > >>>
> > > >>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]>
> > > wrote:
> > > >>>
> > > >>>> Sure! Perhaps the long-running ones can move to a new 'regression'
> > > >>>> area? examples/ is partly what these are, so examples/regression
> > makes
> > > >>>> sense.
> > > >>>>
> > > >>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]>
> > wrote:
> > > >>>>> This test is indeed by far the culprit. I already reduced its
> test
> > > input
> > > >>>>> size to hurry it up, but it's gone slow again.
> > > >>>>>
> > > >>>>> Lance, indeed, these are not all unit tests -- nobody said they
> > were.
> > > The
> > > >>>>> test is useful.
> > > >>>>>
> > > >>>>> I do suggest, however, we comment it out. Jake suggested it coudl
> > be
> > > made
> > > >>>>> faster but I don't think he followed up.
> > > >>>>>
> > > >>>>> Sean
> > > >>>>>
> > > >>>>> On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <

Thankfully the tests for Mahout-627 (complete patch coming today), are
taking 57 seconds to run.

On Tue, Aug 16, 2011 at 2:19 PM, Sean Owen <[EMAIL PROTECTED]> wrote:

> (I disabled it. I had already tried to squeeze down its size without making
> it fail. I will write a note on the back of my hand to make sure to enable
> it before, say, releasing again.)
>
> On Tue, Aug 16, 2011 at 7:16 PM, Jake Mannix <[EMAIL PROTECTED]>
> wrote:
>
> > As long as nobody's changing the logic going into Lanczos, disabling it
> > should be fine.  But I agree with Grant, that we should have it running
> > somewhere (continuous integration?).
> >
> > I can try and squeeze it's size down, but you don't get very good
> > convergence
> > on a small matrix when using these iterative matrix multiplier-style
> > algorithms,
> > which makes it hard to see that it's still working as expected.
> >
> >  -jake
> >
> > On Wed, Aug 10, 2011 at 11:54 PM, Sean Owen <[EMAIL PROTECTED]> wrote:
> >
> > > That would reduce memory requirements unilaterally? no, don't think so.
> I
> > > have not run into memory problems.
> > > The issue here is execution time and it's a big problem indeed.
> > >
> > > Would anyone object to disabling this test for now? it's getting costly
> > in
> > > the dev cycle.
> > >
> > > On Thu, Aug 11, 2011 at 5:52 AM, Lance Norskog <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Another aspect of testing Map/Reduce jobs is memory bounds. An M/R
> job
> > > > should be able to run in a fairly constant amount of ram per JVM from
> > > > beginning to end, to avoid blowing up late in the game. Is there any
> > > > harness around that would do this?
> > > >
> > > > Lance
> > > >
> > > > On Sun, Aug 7, 2011 at 9:04 PM, Ted Dunning <[EMAIL PROTECTED]>
> > > wrote:
> > > > > We have used testNg at work for just this segregation of fast and
> > slow
> > > > tests and it works pretty well. It is also useful for stripping out
> > > > cascading failures. This means that integration tests can be executed
> > > only
> > > > if the underlying unit tests succeed. Another nice thing is that
> testNg
> > > says
> > > > how many tests were skipped.
> > > > >
> > > > > Junit has recently been adding similar capabilities but I haven't
> > kept
> > > > up.
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > On Aug 7, 2011, at 19:18, Grant Ingersoll <[EMAIL PROTECTED]>
> > wrote:
> > > > >
> > > > >> It would likely help if we could get them to run in parallel,
> > perhaps.
> > > >  Also, seems like TestNG might have some better features on paper for
> > > this
> > > > kind of stuff (I think you can annotate some things as "slow" or
> "fast"
> > > and
> > > > then choose to run them separately).  I haven't explored much yet in
> > this
> > > > way.  Has anyone else used TestNG?
> > > > >>
> > > > >> -Grant
> > > > >>
> > > > >> On Aug 7, 2011, at 9:12 PM, Ted Dunning wrote:
> > > > >>
> > > > >>> I really don't think that this distinction needs to be made.  The
> > > > >>> distinction between unit and integration test is important from a
> > > > technical
> > > > >>> point of view, but as an organizing principle the topic and
> target
> > of
> > > > the
> > > > >>> test is probably a better idea than whether the test is a
> > functional
> > > or
> > > > unit
> > > > >>> test or whether it has randomized initial conditions or whether
> it
> > > has
> > > > for
> > > > >>> loops in it.  Tests should be organized by what they test.
> > > > >>>
> > > > >>> On Sun, Aug 7, 2011 at 5:08 PM, Lance Norskog <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >>>
> > > > >>>> Sure! Perhaps the long-running ones can move to a new
> 'regression'
> > > > >>>> area? examples/ is partly what these are, so examples/regression
> > > makes
> > > > >>>> sense.
> > > > >>>>
> > > > >>>> On Sun, Aug 7, 2011 at 11:11 AM, Sean Owen <[EMAIL PROTECTED]>
> > > wrote:
> > > > >>>>> This test is indeed by far the culprit. I already reduced its
> > test
> > > > input
> > > > >>>>> size to hurry it up, but it's gone slow again.

What failures do you see?
The tests ought to be isolated as they (should) reserve unique temp
directories in which to work.
Does forkMode = once mean there's one JVM? That could be the problem, due to
RNG differences. It really needs a JVM per thread.

On Sat, Aug 6, 2011 at 8:34 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:

> Granted, I'm on a slow machine, but our tests take forever to run.  On an 2
> core MBP, it takes well over an hour to run all the tests (I did just order
> a new MBP, so it will be faster, but it doesn't lend itself to a good OOTB
> experience for people)
>
> One idea would be to add in parallel test execution in Maven.  I think this
> requires Mvn 3, but I am not sure.  Another is to take a look at our tests,
> especially the slow ones and see if we can speed them up.
>
> When I try adding in parallel tests to Maven, I get a bunch of failures in
> the tests.
>
> I was using:
> <plugin>
>        <groupId>org.apache.maven.plugins</groupId>
>        <artifactId>maven-surefire-plugin</artifactId>
>        <configuration>
>          <forkMode>once</forkMode>
>          <argLine>-Xms256m -Xmx512m</argLine>
>          <testFailureIgnore>false</testFailureIgnore>
>          <redirectTestOutputToFile>true</redirectTestOutputToFile>
>          <parallel>classes</parallel>
>          <threadCount>5</threadCount>
>        </configuration>
>      </plugin>
>
> Anyone played around with this stuff?  I suspect the failures are due to
> tests stomping on each other, but I am still digging in.
>
> -Grant

I'll have to run again, but perhaps it was just my machine.  Try the config below if you get a chance and see if you get failures.  I too was wondering about the forkMode as the culprit, but didn't have time to test it just yet.

-Grant

On Aug 8, 2011, at 3:01 AM, Sean Owen wrote:

> What failures do you see?
> The tests ought to be isolated as they (should) reserve unique temp
> directories in which to work.
> Does forkMode = once mean there's one JVM? That could be the problem, due to
> RNG differences. It really needs a JVM per thread.
>
> On Sat, Aug 6, 2011 at 8:34 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
>> Granted, I'm on a slow machine, but our tests take forever to run.  On an 2
>> core MBP, it takes well over an hour to run all the tests (I did just order
>> a new MBP, so it will be faster, but it doesn't lend itself to a good OOTB
>> experience for people)
>>
>> One idea would be to add in parallel test execution in Maven.  I think this
>> requires Mvn 3, but I am not sure.  Another is to take a look at our tests,
>> especially the slow ones and see if we can speed them up.
>>
>> When I try adding in parallel tests to Maven, I get a bunch of failures in
>> the tests.
>>
>> I was using:
>> <plugin>
>>       <groupId>org.apache.maven.plugins</groupId>
>>       <artifactId>maven-surefire-plugin</artifactId>
>>       <configuration>
>>         <forkMode>once</forkMode>
>>         <argLine>-Xms256m -Xmx512m</argLine>
>>         <testFailureIgnore>false</testFailureIgnore>
>>         <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>         <parallel>classes</parallel>
>>         <threadCount>5</threadCount>
>>       </configuration>
>>     </plugin>
>>
>> Anyone played around with this stuff?  I suspect the failures are due to
>> tests stomping on each other, but I am still digging in.
>>
>> -Grant

--------------------------------------------
Grant Ingersoll

<forkMode>always</forkMode> works for me. I think that it's also good to run
with <parallel>classes<parallel> so that it only spawns a JVM per class. Try
that?

On Mon, Aug 8, 2011 at 12:18 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

> I'll have to run again, but perhaps it was just my machine.  Try the config
> below if you get a chance and see if you get failures.  I too was wondering
> about the forkMode as the culprit, but didn't have time to test it just yet.
>
> -Grant
>
> On Aug 8, 2011, at 3:01 AM, Sean Owen wrote:
>
> > What failures do you see?
> > The tests ought to be isolated as they (should) reserve unique temp
> > directories in which to work.
> > Does forkMode = once mean there's one JVM? That could be the problem, due
> to
> > RNG differences. It really needs a JVM per thread.
> >
> > On Sat, Aug 6, 2011 at 8:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
> wrote:
> >
> >> Granted, I'm on a slow machine, but our tests take forever to run.  On
> an 2
> >> core MBP, it takes well over an hour to run all the tests (I did just
> order
> >> a new MBP, so it will be faster, but it doesn't lend itself to a good
> OOTB
> >> experience for people)
> >>
> >> One idea would be to add in parallel test execution in Maven.  I think
> this
> >> requires Mvn 3, but I am not sure.  Another is to take a look at our
> tests,
> >> especially the slow ones and see if we can speed them up.
> >>
> >> When I try adding in parallel tests to Maven, I get a bunch of failures
> in
> >> the tests.
> >>
> >> I was using:
> >> <plugin>
> >>       <groupId>org.apache.maven.plugins</groupId>
> >>       <artifactId>maven-surefire-plugin</artifactId>
> >>       <configuration>
> >>         <forkMode>once</forkMode>
> >>         <argLine>-Xms256m -Xmx512m</argLine>
> >>         <testFailureIgnore>false</testFailureIgnore>
> >>         <redirectTestOutputToFile>true</redirectTestOutputToFile>
> >>         <parallel>classes</parallel>
> >>         <threadCount>5</threadCount>
> >>       </configuration>
> >>     </plugin>
> >>
> >> Anyone played around with this stuff?  I suspect the failures are due to
> >> tests stomping on each other, but I am still digging in.
> >>
> >> -Grant
>
> --------------------------------------------
> Grant Ingersoll
>
>
>

Will do.

On Aug 8, 2011, at 1:08 PM, Sean Owen wrote:

> <forkMode>always</forkMode> works for me. I think that it's also good to run
> with <parallel>classes<parallel> so that it only spawns a JVM per class. Try
> that?
>
> On Mon, Aug 8, 2011 at 12:18 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:
>
>> I'll have to run again, but perhaps it was just my machine.  Try the config
>> below if you get a chance and see if you get failures.  I too was wondering
>> about the forkMode as the culprit, but didn't have time to test it just yet.
>>
>> -Grant
>>
>> On Aug 8, 2011, at 3:01 AM, Sean Owen wrote:
>>
>>> What failures do you see?
>>> The tests ought to be isolated as they (should) reserve unique temp
>>> directories in which to work.
>>> Does forkMode = once mean there's one JVM? That could be the problem, due
>> to
>>> RNG differences. It really needs a JVM per thread.
>>>
>>> On Sat, Aug 6, 2011 at 8:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> Granted, I'm on a slow machine, but our tests take forever to run.  On
>> an 2
>>>> core MBP, it takes well over an hour to run all the tests (I did just
>> order
>>>> a new MBP, so it will be faster, but it doesn't lend itself to a good
>> OOTB
>>>> experience for people)
>>>>
>>>> One idea would be to add in parallel test execution in Maven.  I think
>> this
>>>> requires Mvn 3, but I am not sure.  Another is to take a look at our
>> tests,
>>>> especially the slow ones and see if we can speed them up.
>>>>
>>>> When I try adding in parallel tests to Maven, I get a bunch of failures
>> in
>>>> the tests.
>>>>
>>>> I was using:
>>>> <plugin>
>>>>      <groupId>org.apache.maven.plugins</groupId>
>>>>      <artifactId>maven-surefire-plugin</artifactId>
>>>>      <configuration>
>>>>        <forkMode>once</forkMode>
>>>>        <argLine>-Xms256m -Xmx512m</argLine>
>>>>        <testFailureIgnore>false</testFailureIgnore>
>>>>        <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>        <parallel>classes</parallel>
>>>>        <threadCount>5</threadCount>
>>>>      </configuration>
>>>>    </plugin>
>>>>
>>>> Anyone played around with this stuff?  I suspect the failures are due to
>>>> tests stomping on each other, but I am still digging in.
>>>>
>>>> -Grant
>>
>> --------------------------------------------
>> Grant Ingersoll
>>
>>
>>

--------------------------
Grant Ingersoll

>From the Sonar page today: (August 10, 2011):

https://analysis.apache.org/drilldown/measures/63921?metric=test_failures&rids[]=63933#

129993 ms testLanczosSolver

Lanczos taking too long! Are you in the debugger? :)
java.lang.AssertionError: Lanczos taking too long! Are you in the debugger? :)
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.mahout.math.decomposer.lanczos.TestLanczosSolver.testLanczosSolver(TestLanczosSolver.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597

On Mon, Aug 8, 2011 at 11:11 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> Will do.
>
> On Aug 8, 2011, at 1:08 PM, Sean Owen wrote:
>
>> <forkMode>always</forkMode> works for me. I think that it's also good to run
>> with <parallel>classes<parallel> so that it only spawns a JVM per class. Try
>> that?
>>
>> On Mon, Aug 8, 2011 at 12:18 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:
>>
>>> I'll have to run again, but perhaps it was just my machine.  Try the config
>>> below if you get a chance and see if you get failures.  I too was wondering
>>> about the forkMode as the culprit, but didn't have time to test it just yet.
>>>
>>> -Grant
>>>
>>> On Aug 8, 2011, at 3:01 AM, Sean Owen wrote:
>>>
>>>> What failures do you see?
>>>> The tests ought to be isolated as they (should) reserve unique temp
>>>> directories in which to work.
>>>> Does forkMode = once mean there's one JVM? That could be the problem, due
>>> to
>>>> RNG differences. It really needs a JVM per thread.
>>>>
>>>> On Sat, Aug 6, 2011 at 8:34 PM, Grant Ingersoll <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>>> Granted, I'm on a slow machine, but our tests take forever to run.  On
>>> an 2
>>>>> core MBP, it takes well over an hour to run all the tests (I did just
>>> order
>>>>> a new MBP, so it will be faster, but it doesn't lend itself to a good
>>> OOTB
>>>>> experience for people)
>>>>>
>>>>> One idea would be to add in parallel test execution in Maven.  I think
>>> this
>>>>> requires Mvn 3, but I am not sure.  Another is to take a look at our
>>> tests,
>>>>> especially the slow ones and see if we can speed them up.
>>>>>
>>>>> When I try adding in parallel tests to Maven, I get a bunch of failures
>>> in
>>>>> the tests.
>>>>>
>>>>> I was using:
>>>>> <plugin>
>>>>>      <groupId>org.apache.maven.plugins</groupId>
>>>>>      <artifactId>maven-surefire-plugin</artifactId>
>>>>>      <configuration>
>>>>>        <forkMode>once</forkMode>
>>>>>        <argLine>-Xms256m -Xmx512m</argLine>
>>>>>        <testFailureIgnore>false</testFailureIgnore>
>>>>>        <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>>        <parallel>classes</parallel>
>>>>>        <threadCount>5</threadCount>
>>>>>      </configuration>
>>>>>    </plugin>
>>>>>
>>>>> Anyone played around with this stuff?  I suspect the failures are due to
>>>>> tests stomping on each other, but I am still digging in.
>>>>>
>>>>> -Grant
>>>
>>> --------------------------------------------
>>> Grant Ingersoll
>>>
>>>
>>>
>
> --------------------------
> Grant Ingersoll
>
>
>
>

--
Lance Norskog
[EMAIL PROTECTED]