Looked at the clustering changes. They don't seem to be hadoop version
related but are improving the checking of the unit tests. If they run
successfully I'm +1

Jeff
On 3/8/12 4:43 PM, [EMAIL PROTECTED] (Commented) (JIRA) wrote:
>      [ https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225681#comment-13225681 ]
>
> [EMAIL PROTECTED] commented on MAHOUT-822:
> ------------------------------------------------------
>
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4237/#review5754
> -----------------------------------------------------------
>
>
>
> trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
> <https://reviews.apache.org/r/4237/#comment12516>
>
>      Extra white spaces generate style warnings but are notoriously hard to eliminate. I have no problems with them but it was a subject of review notes from other committers before, I am just reproducing collective mind here.
>
>
>
> trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
> <https://reviews.apache.org/r/4237/#comment12518>
>
>      is this really an issue with Hadoop 0.20.203? i don't think we care about classic hadoop 0.20.2 anymore.
>
>
>
> trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java
> <https://reviews.apache.org/r/4237/#comment12552>
>
>      This and some other changes look suspiciously like a functional change rather than a hadoop compatibility change. I think it would help somebody else more familiar with this code to assure it is indeed benign..
>
>
> - Dmitriy
>
>
> On 2012-03-08 01:24:21, tom pierce wrote:
> bq.
> bq.  -----------------------------------------------------------
> bq.  This is an automatically generated e-mail. To reply, visit:
> bq.  https://reviews.apache.org/r/4237/
> bq.  -----------------------------------------------------------
> bq.
> bq.  (Updated 2012-03-08 01:24:21)
> bq.
> bq.
> bq.  Review request for mahout and Dmitriy Lyubimov.
> bq.
> bq.
> bq.  Summary
> bq.  -------
> bq.
> bq.  This is the current patch for MAHOUT-822 (as posted by Bilung Lee).
> bq.
> bq.
> bq.  This addresses bug MAHOUT-822.
> bq.      https://issues.apache.org/jira/browse/MAHOUT-822
> bq.
> bq.
> bq.  Diffs
> bq.  -----
> bq.
> bq.    trunk/core/pom.xml 1296318
> bq.    trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/MockContext.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/PartialSequentialBuilder.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/Step1MapperTest.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/common/DummyCounter.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/common/DummyStatusReporter.java 1296318
> bq.    trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java 1296318
> bq.    trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java 1296318
> bq.    trunk/pom.xml 1296318
> bq.
> bq.  Diff: https://reviews.apache.org/r/4237/diff
> bq.
> bq.
> bq.  Testing
> bq.  -------
> bq.
> bq.  Passes unit tests under default config as well as under hadoop 0.23.1.

Unfortunately, I can't duplicate this test failure.  Is this something
that may clear itself up when the next timer-initiated build starts?

If anyone out there can duplicate the test failure, please let me know
what OS/Java version you are using.  You can run just the one test with:

   mvn -U -DfailIfNoTests=false -Dtest=TestDistributedRowMatrix test

-tom

On 03/13/2012 08:21 AM, Hudson (Commented) (JIRA) wrote:
>      [ https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228353#comment-13228353 ]
>
> Hudson commented on MAHOUT-822:
> -------------------------------
>
> Integrated in Mahout-Quality #1392 (See [https://builds.apache.org/job/Mahout-Quality/1392/])
>      MAHOUT-822: Make Mahout compatible with Hadoop 0.23.1. (Revision 1299770)
>
>       Result = FAILURE
> tcp : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1299770
> Files :
> * /mahout/trunk/core/pom.xml
> * /mahout/trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/MockContext.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/PartialSequentialBuilder.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/Step1MapperTest.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyCounter.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyStatusReporter.java
> * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java
> * /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java
> * /mahout/trunk/pom.xml
>
>
>> Mahout needs to be made compatible with Hadoop .23 releases
>> -----------------------------------------------------------
>>
>>                  Key: MAHOUT-822
>>                  URL: https://issues.apache.org/jira/browse/MAHOUT-822
>>              Project: Mahout
>>           Issue Type: Improvement
>>           Components: build
>>     Affects Versions: 0.6
>>             Reporter: Roman Shaposhnik
>>             Assignee: tom pierce
>>               Labels: bigtop
>>          Attachments: MAHOUT-822-build.patch.txt, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch
>>
>>
>> As part of the Hadoop stack integration project (Apache Bigtop) we are now trying to compile Mahout's upcoming 0.6 release against Hadoop 0.22 and 0.23.
>> I'm attaching the patch to Mahout's Maven build system that made it possible. I would also like to request help in solving the real issues that poped
>> up when we tried to compile Mahout: http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22/COMPONENT=mahout,label=centos5/6/console
>> {noformat}
>> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[182,33] org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be instantiated
>> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[218,9] org.apache.mahout.df.mapreduce.partial.Step0JobTest.Step0Context is not abstract and does not override abstract method getInputSplit() in org.apache.hadoop.mapreduce.MapContext

I am not able to reproduce it, however I did face similar problem once.
Look into ClusterClassificationDriverTest's
testVectorClassificationWithOutlierRemovalMR method. Here I am
explicitly deleting the classifiedOutputPath.

If I don't delete it, I get a similar error ( similar to

Output directory file:/tmp/mahout-TestDistributedRowMatrix-1962901348178391040/testdata/transpose-65 already exists).
when I run it for second time.

Explicitly deleting the file might help in the test case ( I agree that
this is not the best solution, but, I am not able to find the actual
problem for such behavior ).

Paritosh

On 13-03-2012 19:12, tom pierce wrote:
> Unfortunately, I can't duplicate this test failure.  Is this something
> that may clear itself up when the next timer-initiated build starts?
>
> If anyone out there can duplicate the test failure, please let me know
> what OS/Java version you are using.  You can run just the one test with:
>
>   mvn -U -DfailIfNoTests=false -Dtest=TestDistributedRowMatrix test
>
> -tom
>
> On 03/13/2012 08:21 AM, Hudson (Commented) (JIRA) wrote:
>>      [
>> https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228353#comment-13228353
>> ]
>>
>> Hudson commented on MAHOUT-822:
>> -------------------------------
>>
>> Integrated in Mahout-Quality #1392 (See
>> [https://builds.apache.org/job/Mahout-Quality/1392/])
>>      MAHOUT-822: Make Mahout compatible with Hadoop 0.23.1. (Revision
>> 1299770)
>>
>>       Result = FAILURE
>> tcp :
>> http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1299770
>> Files :
>> * /mahout/trunk/core/pom.xml
>> *
>> /mahout/trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/MockContext.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/PartialSequentialBuilder.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/Step1MapperTest.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyCounter.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyStatusReporter.java
>> *
>> /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java
>> *
>> /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java
>> * /mahout/trunk/pom.xml
>>
>>
>>> Mahout needs to be made compatible with Hadoop .23 releases
>>> -----------------------------------------------------------
>>>
>>>                  Key: MAHOUT-822
>>>                  URL: https://issues.apache.org/jira/browse/MAHOUT-822
>>>              Project: Mahout
>>>           Issue Type: Improvement
>>>           Components: build
>>>     Affects Versions: 0.6
>>>             Reporter: Roman Shaposhnik
>>>             Assignee: tom pierce
>>>               Labels: bigtop
>>>          Attachments: MAHOUT-822-build.patch.txt, MAHOUT-822.patch,
>>> MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch,
>>> MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch
>>>
>>>
>>> As part of the Hadoop stack integration project (Apache Bigtop) we
>>> are now trying to compile Mahout's upcoming 0.6 release against
>>> Hadoop 0.22 and 0.23.
>>> I'm attaching the patch to Mahout's Maven build system that made it
>>

And its ubuntu 11.10, 32 bit ( running as a vm on windows 7, 64 bit ).

On 13-03-2012 20:22, Paritosh Ranjan wrote:
> I am not able to reproduce it, however I did face similar problem once.
> Look into ClusterClassificationDriverTest's
> testVectorClassificationWithOutlierRemovalMR method. Here I am
> explicitly deleting the classifiedOutputPath.
>
> If I don't delete it, I get a similar error ( similar to
>
> Output directory
> file:/tmp/mahout-TestDistributedRowMatrix-1962901348178391040/testdata/transpose-65
> already exists).
>
>
> when I run it for second time.
>
> Explicitly deleting the file might help in the test case ( I agree
> that this is not the best solution, but, I am not able to find the
> actual problem for such behavior ).
>
> Paritosh
>
> On 13-03-2012 19:12, tom pierce wrote:
>> Unfortunately, I can't duplicate this test failure.  Is this
>> something that may clear itself up when the next timer-initiated
>> build starts?
>>
>> If anyone out there can duplicate the test failure, please let me
>> know what OS/Java version you are using.  You can run just the one
>> test with:
>>
>>   mvn -U -DfailIfNoTests=false -Dtest=TestDistributedRowMatrix test
>>
>> -tom
>>
>> On 03/13/2012 08:21 AM, Hudson (Commented) (JIRA) wrote:
>>>      [
>>> https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228353#comment-13228353
>>> ]
>>>
>>> Hudson commented on MAHOUT-822:
>>> -------------------------------
>>>
>>> Integrated in Mahout-Quality #1392 (See
>>> [https://builds.apache.org/job/Mahout-Quality/1392/])
>>>      MAHOUT-822: Make Mahout compatible with Hadoop 0.23.1.
>>> (Revision 1299770)
>>>
>>>       Result = FAILURE
>>> tcp :
>>> http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1299770
>>> Files :
>>> * /mahout/trunk/core/pom.xml
>>> *
>>> /mahout/trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/MockContext.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/PartialSequentialBuilder.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/Step1MapperTest.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyCounter.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyStatusReporter.java
>>> *
>>> /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java
>>> *
>>> /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java
>>> * /mahout/trunk/pom.xml
>>>
>>>
>>>> Mahout needs to be made compatible with Hadoop .23 releases
>>>> -----------------------------------------------------------
>>>>
>>>>                  Key: MAHOUT-822
>>>>                  URL: https://issues.apache.org/jira/browse/MAHOUT-822
>>>>              Project: Mahout
>>>>           Issue Type: Improvement
>>>>           Components: build
>>>>     Affects Versions: 0.6
>>>>             Reporter: Roman Shaposhnik
>>>>             Assignee: tom pierce
>>>>               Labels: bigtop
>>>>          Attachments: MAHOUT-822-build.patch.txt, MAHOUT-822.patch,
>>>> MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch,
>>>> MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch
>>>>
>>>>
>>>> As part of the Hadoop stack integration project (Apache Bigtop) we

Thanks, Paritosh.  I'll give this a try.  Though looking at this briefly
I don't think we should need these deletes (in either case), which is
bothersome.  As is the fact that we're seeing different results on
different OS/JVM combos.  Would be good to get to the bottom of this
problem.

-t

On 03/13/2012 11:17 AM, Paritosh Ranjan wrote:
> And its ubuntu 11.10, 32 bit ( running as a vm on windows 7, 64 bit ).
>
> On 13-03-2012 20:22, Paritosh Ranjan wrote:
>> I am not able to reproduce it, however I did face similar problem once.
>> Look into ClusterClassificationDriverTest's
>> testVectorClassificationWithOutlierRemovalMR method. Here I am
>> explicitly deleting the classifiedOutputPath.
>>
>> If I don't delete it, I get a similar error ( similar to
>>
>> Output directory
>> file:/tmp/mahout-TestDistributedRowMatrix-1962901348178391040/testdata/transpose-65
>> already exists).
>>
>>
>> when I run it for second time.
>>
>> Explicitly deleting the file might help in the test case ( I agree
>> that this is not the best solution, but, I am not able to find the
>> actual problem for such behavior ).
>>
>> Paritosh
>>
>> On 13-03-2012 19:12, tom pierce wrote:
>>> Unfortunately, I can't duplicate this test failure.  Is this
>>> something that may clear itself up when the next timer-initiated
>>> build starts?
>>>
>>> If anyone out there can duplicate the test failure, please let me
>>> know what OS/Java version you are using.  You can run just the one
>>> test with:
>>>
>>>   mvn -U -DfailIfNoTests=false -Dtest=TestDistributedRowMatrix test
>>>
>>> -tom
>>>
>>> On 03/13/2012 08:21 AM, Hudson (Commented) (JIRA) wrote:
>>>>      [
>>>> https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228353#comment-13228353
>>>> ]
>>>>
>>>> Hudson commented on MAHOUT-822:
>>>> -------------------------------
>>>>
>>>> Integrated in Mahout-Quality #1392 (See
>>>> [https://builds.apache.org/job/Mahout-Quality/1392/])
>>>>      MAHOUT-822: Make Mahout compatible with Hadoop 0.23.1.
>>>> (Revision 1299770)
>>>>
>>>>       Result = FAILURE
>>>> tcp :
>>>> http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1299770
>>>> Files :
>>>> * /mahout/trunk/core/pom.xml
>>>> *
>>>> /mahout/trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/MockContext.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/PartialSequentialBuilder.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/df/mapreduce/partial/Step1MapperTest.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyCounter.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/common/DummyStatusReporter.java
>>>> *
>>>> /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java
>>>> *
>>>> /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java
>>>> * /mahout/trunk/pom.xml
>>>>
>>>>
>>>>> Mahout needs to be made compatible with Hadoop .23 releases
>>>>> -----------------------------------------------------------
>>>>>
>>>>>                  Key: MAHOUT-822
>>>>>                  URL:
>>>>> https://issues.apache.org/jira/browse/MAHOUT-822
>>>>>              Project: Mahout
>>>>