+1

I tested on OS X 10.6.8, indexing first 100K docs from Wikipedia and
running a few searches.

Mike McCandless

http://blog.mikemccandless.com

On Tue, Aug 21, 2012 at 3:37 AM, Thomas Koch <[EMAIL PROTECTED]> wrote:
> Thanks, JCC 2.14 now builds just fine with MS-VC9 on Windows. Could also
> build pylucene 3.6.1 and run the tests (except of the usual suspects on
> windows that fail due to the file lock issue).
>
> So +1 for release.
>
> kind regards
> Thomas
> --
> OrbiTeam Software GmbH & Co. KG
> http://www.orbiteam.de
>
>> -----Ursprüngliche Nachricht-----
>> Von: Andi Vajda [mailto:[EMAIL PROTECTED]]
>> Gesendet: Dienstag, 21. August 2012 07:14
>> An: [EMAIL PROTECTED]
>> Cc: [EMAIL PROTECTED]
>> Betreff: [VOTE] Release PyLucene 3.6.1-2
>>
>>
>> The PyLucene 3.6.1-2 release tracking the (not so) recent release of
> Apache
>> Lucene 3.6.1 is ready.
>>
>> A release candidate is available from:
>> http://people.apache.org/~vajda/staging_area/
>>
>> A list of changes in this release can be seen at:
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_6/
>> CHANGES
>>
>> PyLucene 3.6.1 is built with JCC 2.14 included in these release artifacts:
>> http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES
>>
>> A list of Lucene Java changes can be seen at:
>> http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_1/lucen
>> e/CHANGES.txt
>>
>> Please vote to release these artifacts as PyLucene 3.6.1-2.
>>
>> Thanks !
>>
>> Andi..
>>
>> ps: the KEYS file for PyLucene release signing is at:
>> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
>> http://people.apache.org/~vajda/staging_area/KEYS
>>
>> pps: here is my +1
>
>

On Wed, 22 Aug 2012, Robert Muir wrote:

> On Tue, Aug 21, 2012 at 1:14 AM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>>
>> Please vote to release these artifacts as PyLucene 3.6.1-2.
>>
>
> +1 (from Linux).
>
> some unrelated notes:
> something is up in my browser at least for the links to pylucene and
> jcc install instructions on http://lucene.apache.org/pylucene/ and
> http://lucene.apache.org/pylucene/install.html
> the direct link from jcc/INSTALL works fine
> (http://lucene.apache.org/pylucene/jcc/install.html) but it seems to
> have lost its CSS.

Yes, the PyLucene site is a crying shame right now and it has been that way
for a year, ever since the switch to the CMS was made. I've had no time to
do a thorough cleanup there to get it into shape - and so has no one else in
the PyLucene user community.

I view this CMS switch as an unfunded (timewise) mandate :-)

> I noticed this because i tried on a new linux system where i didnt
> have setuptools and was trying to get shared mode working (applied
> patch etc), ultimately I just ended out testing with shared mode
> disabled (which worked fine).

It's good to know this configuration is also still working.
Thank you for your vote !

Andi..

On Wed, Aug 22, 2012 at 3:40 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
> Yes, the PyLucene site is a crying shame right now and it has been that way
> for a year, ever since the switch to the CMS was made. I've had no time to
> do a thorough cleanup there to get it into shape - and so has no one else in
> the PyLucene user community.
>
> I view this CMS switch as an unfunded (timewise) mandate :-)
>

I'll see if i can fix the links... I don't really know what I am doing
with the CMS either so no promises.

--
lucidworks.com

Am 21.08.2012 07:14, schrieb Andi Vajda:
> Please vote to release these artifacts as PyLucene 3.6.1-2.

-1, sorry :)

test_ICUTransformFilter.py is segfaulting on my system. I haven't been
able to get a sensible stack trace yet. I'm going to recompile
everything with debug infos tomorrow.

OS: Ubuntu 12.04
Arch: x86_64
Python: 2.7.3 (self compiled with UCS-2)
ICU4C: 49.1.1
PyICU: 1.4

Christian

On Thu, 23 Aug 2012, Christian Heimes wrote:

> Am 21.08.2012 07:14, schrieb Andi Vajda:
>> Please vote to release these artifacts as PyLucene 3.6.1-2.
>
> -1, sorry :)
>
> test_ICUTransformFilter.py is segfaulting on my system. I haven't been
> able to get a sensible stack trace yet. I'm going to recompile
> everything with debug infos tomorrow.

I'm getting errors there too (if I build PyICU into PyLucene's install tree,
otherwise these tests are not enabled). I believe the problem lies with the
utr30.dat resource file. Their format requirements probably changed with the
upgrade to ICU 49.

I remember having a lot of trouble reverse engineering what format these
files should be transformed into when taken from the Java version, such as
for example, resources/org/apache/lucene/analysis/icu/utr30.dat.

As this feature is optional and rather experimental, I'm not prepared to
block the release for that bug. Not that it doesn't need to be fixed but the
tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's utr30.nrm via
the /usr/local/sbin/icupkg utility needs to be figured out again for this to
work properly.

Andi..

>
> OS: Ubuntu 12.04
> Arch: x86_64
> Python: 2.7.3 (self compiled with UCS-2)
> ICU4C: 49.1.1
> PyICU: 1.4
>
> Christian
>

On Thu, Aug 23, 2012 at 12:21 AM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
> On Thu, 23 Aug 2012, Christian Heimes wrote:
>
>> Am 21.08.2012 07:14, schrieb Andi Vajda:
>>>
>>> Please vote to release these artifacts as PyLucene 3.6.1-2.
>>
>>
>> -1, sorry :)
>>
>> test_ICUTransformFilter.py is segfaulting on my system. I haven't been
>> able to get a sensible stack trace yet. I'm going to recompile
>> everything with debug infos tomorrow.
>
>
> I'm getting errors there too (if I build PyICU into PyLucene's install tree,
> otherwise these tests are not enabled). I believe the problem lies with the
> utr30.dat resource file. Their format requirements probably changed with the
> upgrade to ICU 49.
>

Hi Andi, you are correct. these files only work with 4.8.1.1

If you are interested in working with 49 (which uses a newer format
version), then you have to either regenerate yourself or if you want
you can just use our files from
http://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x/lucene/analysis/icu/src/resources/
since Steven Rowe already upgraded our 4.x branch to 49 and
regenerated them for that.

Of course if you do this you will break any users using 4.8.x as well
unless you include both and conditionalize it based on the version:
our code/resources only support the version we ship with (not
arbitrary versions of ICU), so we don't worry about that.

--
lucidworks.com

Am 23.08.2012 06:21, schrieb Andi Vajda:
> I'm getting errors there too (if I build PyICU into PyLucene's install
> tree, otherwise these tests are not enabled). I believe the problem lies
> with the utr30.dat resource file. Their format requirements probably
> changed with the upgrade to ICU 49.
>
> I remember having a lot of trouble reverse engineering what format these
> files should be transformed into when taken from the Java version, such as
> for example, resources/org/apache/lucene/analysis/icu/utr30.dat.
>
> As this feature is optional and rather experimental, I'm not prepared to
> block the release for that bug. Not that it doesn't need to be fixed but
> the tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's
> utr30.nrm via the /usr/local/sbin/icupkg utility needs to be figured out
> again for this to work properly.

I concur with your analysis. The tests are passing when I use Ubuntu's
default version 48.1.

With 49.1.1 test suite segfaults somehwere in
lucene/ICUTransformFilter.py. I'm unable to get the exact line number as
the values were optimized out by the compiler.

Perhaps you could disable all ICU features when icu.ICU_VERSION is 49 or
newer? That would prohibit people from getting segfaults.

Christian

On Aug 23, 2012, at 6:09, Christian Heimes <[EMAIL PROTECTED]> wrote:

> Am 23.08.2012 06:21, schrieb Andi Vajda:
>> I'm getting errors there too (if I build PyICU into PyLucene's install
>> tree, otherwise these tests are not enabled). I believe the problem lies
>> with the utr30.dat resource file. Their format requirements probably
>> changed with the upgrade to ICU 49.
>>
>> I remember having a lot of trouble reverse engineering what format these
>> files should be transformed into when taken from the Java version, such as
>> for example, resources/org/apache/lucene/analysis/icu/utr30.dat.
>>
>> As this feature is optional and rather experimental, I'm not prepared to
>> block the release for that bug. Not that it doesn't need to be fixed but
>> the tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's
>> utr30.nrm via the /usr/local/sbin/icupkg utility needs to be figured out
>> again for this to work properly.
>
> I concur with your analysis. The tests are passing when I use Ubuntu's
> default version 48.1.
>
> With 49.1.1 test suite segfaults somehwere in
> lucene/ICUTransformFilter.py. I'm unable to get the exact line number as
> the values were optimized out by the compiler.
>
> Perhaps you could disable all ICU features when icu.ICU_VERSION is 49 or
> newer? That would prohibit people from getting segfaults.

I agree, a version check should be added to the tests and a proper warning should be emitted in case of mismatch.

Is it worth respinning a release candidate for this or should I go ahead and proceed with the release since the vote now has received the required three PMC votes for release ?

Andi..

>
> Christian

On Mon, 20 Aug 2012, Andi Vajda wrote:

> The PyLucene 3.6.1-2 release tracking the (not so) recent release of Apache
> Lucene 3.6.1 is ready.
>
> A release candidate is available from:
> http://people.apache.org/~vajda/staging_area/
>
> A list of changes in this release can be seen at:
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_6/CHANGES
>
> PyLucene 3.6.1 is built with JCC 2.14 included in these release artifacts:
> http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES
>
> A list of Lucene Java changes can be seen at:
> http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_1/lucene/CHANGES.txt
>
> Please vote to release these artifacts as PyLucene 3.6.1-2.
>
> Thanks !
>
> Andi..
>
> ps: the KEYS file for PyLucene release signing is at:
> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
> http://people.apache.org/~vajda/staging_area/KEYS
>
> pps: here is my +1

This vote has passed. Thank you all who voted.
Expect a release announcement shortly.

Andi..

On Thu, 23 Aug 2012, Christian Heimes wrote:

> Am 21.08.2012 07:14, schrieb Andi Vajda:
>> Please vote to release these artifacts as PyLucene 3.6.1-2.
>
> -1, sorry :)
>
> test_ICUTransformFilter.py is segfaulting on my system. I haven't been
> able to get a sensible stack trace yet. I'm going to recompile
> everything with debug infos tomorrow.
>
> OS: Ubuntu 12.04
> Arch: x86_64
> Python: 2.7.3 (self compiled with UCS-2)
> ICU4C: 49.1.1
> PyICU: 1.4
>
> Christian

I investigated this and found that there are actually two problems here:

   - for test/test_ICUFoldingFilter.py, the version of ICU must be 4.8 with
     Lucene 3.x and 49 with Lucene 4.x (as explained by Robert earlier)

   - the segfault with test/test_ICUTransformFilter.py is due to a bug in
     PyICU 1.4: a UBool was passed to parseArgs where an int was expected,
     causing the stack to get corrupted

I fixed the PyICU bug in rev 200 of its svn repository.
I added a version check for test/ICUFoldingFilter.py to skip the test and
emit a warning in case of an ICU_VERSION mismatch.

I also got the ICU-based tests to work on 4.0 (see below).

Andi..

$ ls test/test_ICU* | xargs -n 1 _install/bin/python
.
----------------------------------------------------------------------
Ran 1 test in 0.009s

OK
..
----------------------------------------------------------------------
Ran 2 tests in 0.009s

OK
....
----------------------------------------------------------------------
Ran 4 tests in 0.199s

OK