On Sun, Dec 5, 2010 at 1:50 AM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
> With the recent releases of Lucene Java 2.9.4 and 3.0.3, the PyLucene
> 2.9.4-1 and 3.0.3-1 releases closely tracking them are ready.
>
> Release candidates are available from:
>
>    http://people.apache.org/~vajda/staging_area/
>
> A list of changes in this release can be seen at:
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_9/CHANGES
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_0/CHANGES
>
> All versions of PyLucene are built with the same version of JCC, currently
> version 2.7, included in these release artifacts.
>
> A list of Lucene Java changes can be seen at:
> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9/CHANGES.txt
> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_3_0/CHANGES.txt
>
> Please vote to release these artifacts as PyLucene 2.9.4-1 and 3.0.3-1.
>

+1, everything looks in order, building pylucene and running 'make
test' seemed fine on both versions.

Hi Andi

I downloaded the release candidate from your staging area, and will be
running pylucene on OS X.
I am not sure if this is intentional, but was there a reason for this
specific change?

adil:681 lucene $ diff pylucene-3.0.2-1/jcc/setup.py
pylucene-3.0.3-1/jcc/setup.py
...
47c47
<     'darwin':
'/System/Library/Frameworks/JavaVM.framework/Versions/Current',
---
>     'darwin':
'/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
...

I have the standard JDK installation, which is in /System.

Also, this is my system config:
adil:689 lucene $ /usr/bin/sw_vers
ProductName: Mac OS X
ProductVersion: 10.5.8
BuildVersion: 9L31a

adil:690 lucene $ python -V
Python 2.6.6

adil:691 lucene $ java -version
java version "1.5.0_26"
Java(TM) 2 Runtime Environment, Standard Edition (build
1.5.0_26-b03-376-9M3263)
Java HotSpot(TM) Client VM (build 1.5.0_26-156, mixed mode, sharing)
adil:692 lucene

And for both pylucene-3.0.3-1 (rc) and pylucene-3.0.2-1 when I run the make
tests, most of the tests run fine, but a couple of them crash Python.
...
/opt/local/bin/python test/test_PhraseQuery.py
........
----------------------------------------------------------------------
Ran 8 tests in 0.162s

OK
/opt/local/bin/python test/test_PositionIncrement.py
Invalid memory access of location 0x0 eip=0x240ba3b
make: *** [test] Error 1

I have tested this on three of our 10.5 systems, and it is reproduceable on
all. (The unit tests complete okay on 10.6). I have tried switching to Java
6 and/or Python 2.7, and I still get the same behaviour.  This crash is not
a show-stopper for us. Pylucene is working fine for what we need.

Cheers,
-Adil

On Sat, Dec 4, 2010 at 10:50 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:

>
> With the recent releases of Lucene Java 2.9.4 and 3.0.3, the PyLucene
> 2.9.4-1 and 3.0.3-1 releases closely tracking them are ready.
>
> Release candidates are available from:
>
>    http://people.apache.org/~vajda/staging_area/
>
> A list of changes in this release can be seen at:
>
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_9/CHANGES
>
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_0/CHANGES
>
> All versions of PyLucene are built with the same version of JCC, currently
> version 2.7, included in these release artifacts.
>
> A list of Lucene Java changes can be seen at:
> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9/CHANGES.txt
> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_3_0/CHANGES.txt
>
> Please vote to release these artifacts as PyLucene 2.9.4-1 and 3.0.3-1.
>
> Thanks !
>
> Andi..
>
> ps: the KEYS file for PyLucene release signing is at:
>    http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
>    http://people.apache.org/~vajda/staging_area/KEYS
>
> pps: here is my +1
>

  Hi Adil,

On Tue, 7 Dec 2010, Adil Lotia wrote:

> I downloaded the release candidate from your staging area, and will be
> running pylucene on OS X.
> I am not sure if this is intentional, but was there a reason for this
> specific change?
>
> adil:681 lucene $ diff pylucene-3.0.2-1/jcc/setup.py
> pylucene-3.0.3-1/jcc/setup.py
> ...
> 47c47
> <     'darwin':
> '/System/Library/Frameworks/JavaVM.framework/Versions/Current',
> ---
>>     'darwin':
> '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
> ...
>
> I have the standard JDK installation, which is in /System.

Yes, two things made me do this:
   1. Bill Janssen said on this list that an Apple developer he talked to
      said that the /Developer tree is the correct one to use.
   2. After the last Java update on Mac OS X 10.6.x, the header files
      disappeared from the Java framework tree under /System, so I switched
      to the tree under /Developer.

Andi..

>
> Also, this is my system config:
> adil:689 lucene $ /usr/bin/sw_vers
> ProductName: Mac OS X
> ProductVersion: 10.5.8
> BuildVersion: 9L31a
>
> adil:690 lucene $ python -V
> Python 2.6.6
>
> adil:691 lucene $ java -version
> java version "1.5.0_26"
> Java(TM) 2 Runtime Environment, Standard Edition (build
> 1.5.0_26-b03-376-9M3263)
> Java HotSpot(TM) Client VM (build 1.5.0_26-156, mixed mode, sharing)
> adil:692 lucene
>
> And for both pylucene-3.0.3-1 (rc) and pylucene-3.0.2-1 when I run the make
> tests, most of the tests run fine, but a couple of them crash Python.
> ...
> /opt/local/bin/python test/test_PhraseQuery.py
> ........
> ----------------------------------------------------------------------
> Ran 8 tests in 0.162s
>
> OK
> /opt/local/bin/python test/test_PositionIncrement.py
> Invalid memory access of location 0x0 eip=0x240ba3b
> make: *** [test] Error 1
>
> I have tested this on three of our 10.5 systems, and it is reproduceable on
> all. (The unit tests complete okay on 10.6). I have tried switching to Java
> 6 and/or Python 2.7, and I still get the same behaviour.  This crash is not
> a show-stopper for us. Pylucene is working fine for what we need.
>
> Cheers,
> -Adil
>
> On Sat, Dec 4, 2010 at 10:50 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>>
>> With the recent releases of Lucene Java 2.9.4 and 3.0.3, the PyLucene
>> 2.9.4-1 and 3.0.3-1 releases closely tracking them are ready.
>>
>> Release candidates are available from:
>>
>>    http://people.apache.org/~vajda/staging_area/
>>
>> A list of changes in this release can be seen at:
>>
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_9/CHANGES
>>
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_0/CHANGES
>>
>> All versions of PyLucene are built with the same version of JCC, currently
>> version 2.7, included in these release artifacts.
>>
>> A list of Lucene Java changes can be seen at:
>> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9/CHANGES.txt
>> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_3_0/CHANGES.txt
>>
>> Please vote to release these artifacts as PyLucene 2.9.4-1 and 3.0.3-1.
>>
>> Thanks !
>>
>> Andi..
>>
>> ps: the KEYS file for PyLucene release signing is at:
>>    http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
>>    http://people.apache.org/~vajda/staging_area/KEYS
>>
>> pps: here is my +1
>>
>

Andi Vajda <[EMAIL PROTECTED]> wrote:

> Yes, two things made me do this:
>   1. Bill Janssen said on this list that an Apple developer he talked to
>      said that the /Developer tree is the correct one to use.

On a 10.6 system.  Adil is talking about a 10.5 system, which will
(obviously?) not have a MacOSX10.6.sdk directory on it.

I don't think there is, any longer, a reliable way to specify a JDK on
an OS X system.  We need a script like the one for Windows.

Bill

Here's a patch to jcc/setup.py.  With this it builds on both my 10.6 and
my 10.5 systems.

Bill

diff -u pylucene-3.0.3-1/jcc/setup.py.orig pylucene-3.0.3-1/jcc/setup.py
--- pylucene-3.0.3-1/jcc/setup.py.orig 2010-12-07 18:50:25.000000000 -0800
+++ pylucene-3.0.3-1/jcc/setup.py 2010-12-07 19:02:44.000000000 -0800
@@ -43,8 +43,23 @@
 else:
     JAVAHOME = None
 
+if platform in ("darwin"):
+    # possibly we shouldn't overwrite the platform module in the first place
+    import platform as _platform
+    import re
+    JAVAHOME = None
+    path = ("/Developer/SDKs/MacOSX" +
+            re.match("[0-9]+\.[0-9]+", _platform.mac_ver()[0]).group(0) +
+            ".sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current")
+    if os.path.exists(path):
+        JAVAHOME = path
+    else:
+        path = "/System/Library/Frameworks/JavaVM.framework/Versions/Current"
+        if os.path.isdir(path) and os.path.exists(os.path.join(path, "Headers", "jni.h")):
+            JAVAHOME = path
+
 JDK = {
-    'darwin': '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
+    'darwin': JAVAHOME,
     'ipod': '/usr/include/gcc',
     'linux2': '/usr/lib/jvm/java-6-openjdk',
     'sunos5': '/usr/jdk/instances/jdk1.6.0',

On Tue, 7 Dec 2010, Bill Janssen wrote:

> Here's a patch to jcc/setup.py.  With this it builds on both my 10.6 and
> my 10.5 systems.
>
> Bill
>
> diff -u pylucene-3.0.3-1/jcc/setup.py.orig pylucene-3.0.3-1/jcc/setup.py
> --- pylucene-3.0.3-1/jcc/setup.py.orig 2010-12-07 18:50:25.000000000 -0800
> +++ pylucene-3.0.3-1/jcc/setup.py 2010-12-07 19:02:44.000000000 -0800
> @@ -43,8 +43,23 @@
> else:
>     JAVAHOME = None
>
> +if platform in ("darwin"):
> +    # possibly we shouldn't overwrite the platform module in the first place
> +    import platform as _platform
> +    import re
> +    JAVAHOME = None
> +    path = ("/Developer/SDKs/MacOSX" +
> +            re.match("[0-9]+\.[0-9]+", _platform.mac_ver()[0]).group(0) +
> +            ".sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current")
> +    if os.path.exists(path):
> +        JAVAHOME = path
> +    else:
> +        path = "/System/Library/Frameworks/JavaVM.framework/Versions/Current"
> +        if os.path.isdir(path) and os.path.exists(os.path.join(path, "Headers", "jni.h")):
> +            JAVAHOME = path
> +
> JDK = {
> -    'darwin': '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
> +    'darwin': JAVAHOME,
>     'ipod': '/usr/include/gcc',
>     'linux2': '/usr/lib/jvm/java-6-openjdk',
>     'sunos5': '/usr/jdk/instances/jdk1.6.0',

With this patch, it's going to pick the 10.5 SDK on my system. I have
apparently both installed under /Developer/SDKs. It'd be nice if it picked
the latest one.

Andi..

Andi Vajda <[EMAIL PROTECTED]> wrote:

> On Tue, 7 Dec 2010, Bill Janssen wrote:
>
> > Here's a patch to jcc/setup.py.  With this it builds on both my 10.6 and
> > my 10.5 systems.
> >
> > Bill
> >
> > diff -u pylucene-3.0.3-1/jcc/setup.py.orig pylucene-3.0.3-1/jcc/setup.py
> > --- pylucene-3.0.3-1/jcc/setup.py.orig 2010-12-07 18:50:25.000000000 -0800
> > +++ pylucene-3.0.3-1/jcc/setup.py 2010-12-07 19:02:44.000000000 -0800
> > @@ -43,8 +43,23 @@
> > else:
> >     JAVAHOME = None
> >
> > +if platform in ("darwin"):
> > +    # possibly we shouldn't overwrite the platform module in the first place
> > +    import platform as _platform
> > +    import re
> > +    JAVAHOME = None
> > +    path = ("/Developer/SDKs/MacOSX" +
> > +            re.match("[0-9]+\.[0-9]+", _platform.mac_ver()[0]).group(0) +
> > +            ".sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current")
> > +    if os.path.exists(path):
> > +        JAVAHOME = path
> > +    else:
> > +        path = "/System/Library/Frameworks/JavaVM.framework/Versions/Current"
> > +        if os.path.isdir(path) and os.path.exists(os.path.join(path, "Headers", "jni.h")):
> > +            JAVAHOME = path
> > +
> > JDK = {
> > -    'darwin': '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
> > +    'darwin': JAVAHOME,
> >     'ipod': '/usr/include/gcc',
> >     'linux2': '/usr/lib/jvm/java-6-openjdk',
> >     'sunos5': '/usr/jdk/instances/jdk1.6.0',
>
> With this patch, it's going to pick the 10.5 SDK on my system. I have
> apparently both installed under /Developer/SDKs. It'd be nice if it
> picked the latest one.

It tries to pick the one that matches your OS.  If you're running 10.5,
it should pick that -- it did on my system.

Bill

Andi Vajda <[EMAIL PROTECTED]> wrote:

> Yes, two things made me do this:
>   1. Bill Janssen said on this list that an Apple developer he talked to
>      said that the /Developer tree is the correct one to use.

I asked again, and got this answer.  Looks like I was (wrong? talking
about a pre-release version of Snow Leopard?); Mike says to just use
/System/Library/Frameworks/JavaVM.framework/Headers, and explains why
that doesn't always work, and that they're aware of the problem.

So let's just hardcode that path into the setup.py file for "darwin".

Bill

Subject: Re: how to locate an installed JDK?
From: Mike Swingler <[EMAIL PROTECTED]>
Date: Wed, 8 Dec 2010 07:13:59 PST
Cc: java-dev <[EMAIL PROTECTED]>

On Dec 7, 2010, at 7:08 PM, Bill Janssen wrote:

> Now that we have /usr/libexec/java_home, is there a standard way to
> locate a JDK, with tools like "javac" and "jar" and header files like
> "jni.h"?
>
> The PyLucene project installs a tool called "JCC", which wraps a jar
> file as a Python module, but to build JCC, it has to find a Java JDK
> with header files.  Is there a standard way of doing that yet?

To find a $JAVA_HOME for the purposes of using the command line tools,
please use /usr/libexec/java_home. The man page for "java_home"
enumerates all of it's options.

As for headers, always use /System/Library/Frameworks/JavaVM.framework/Headers.

It may be necessary to install the "Java for Mac OS X 10.6 Update 3
Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
Package" from <http://connect.apple.com> under the "Java" section, to
ensure the native headers are present in the JavaVM.framework. We are
aware that this has caused some inconvenience for otherwise native
projects that make use of the JNI API, and we are leaning towards just
shipping the headers in the regular customer software update package.

Regards,
Mike Swingler
Java Engineering
Apple Inc.

On Wed, 8 Dec 2010, Bill Janssen wrote:

> Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>> Yes, two things made me do this:
>>   1. Bill Janssen said on this list that an Apple developer he talked to
>>      said that the /Developer tree is the correct one to use.
>
> I asked again, and got this answer.  Looks like I was (wrong? talking
> about a pre-release version of Snow Leopard?); Mike says to just use
> /System/Library/Frameworks/JavaVM.framework/Headers, and explains why
> that doesn't always work, and that they're aware of the problem.
>
> So let's just hardcode that path into the setup.py file for "darwin".

That would be the safer thing to do at this point, yes.

I wasn't able to build on 10.5 yesterday (my old G4 powerbook needs some
refreshing first [1]). Once I confirm 10.5 presents no problems, I'm
probably going to revert the setup.py change and prepare new release
artifacts and call for another vote.

Andi..

[1] bitrot story:
   - the svn I have installed there doesn't support the externals format used
   - updating svn via MacPorts requires a MacPorts update first
   - updating MacPorts implies a dev tools update (since it complains about
     Xcode 3.0 causing "problems")
   - installing the 3.2.2 devtools on my 10.5.8 G4 powerbook crashes the
     installer.app, consistently, while "looking for installation volumes"
   - downloading the 3.2.5 devtools took 3 hrs (3.52GB download, I was not
     able to find a Xcode-only .dmg, only Xcode+iPhone SDK, iPhone part which
     is not even supported on 10.5), download completed overnight
   - no time to continue this until tonight again

>
> Bill
>
> Subject: Re: how to locate an installed JDK?
> From: Mike Swingler <[EMAIL PROTECTED]>
> Date: Wed, 8 Dec 2010 07:13:59 PST
> Cc: java-dev <[EMAIL PROTECTED]>
>
> On Dec 7, 2010, at 7:08 PM, Bill Janssen wrote:
>
>> Now that we have /usr/libexec/java_home, is there a standard way to
>> locate a JDK, with tools like "javac" and "jar" and header files like
>> "jni.h"?
>>
>> The PyLucene project installs a tool called "JCC", which wraps a jar
>> file as a Python module, but to build JCC, it has to find a Java JDK
>> with header files.  Is there a standard way of doing that yet?
>
> To find a $JAVA_HOME for the purposes of using the command line tools,
> please use /usr/libexec/java_home. The man page for "java_home"
> enumerates all of it's options.
>
> As for headers, always use /System/Library/Frameworks/JavaVM.framework/Headers.
>
> It may be necessary to install the "Java for Mac OS X 10.6 Update 3
> Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
> Package" from <http://connect.apple.com> under the "Java" section, to
> ensure the native headers are present in the JavaVM.framework. We are
> aware that this has caused some inconvenience for otherwise native
> projects that make use of the JNI API, and we are leaning towards just
> shipping the headers in the regular customer software update package.
>
> Regards,
> Mike Swingler
> Java Engineering
> Apple Inc.
>

Hi Andi,

I think Xcode 3.1.[34] is the last one which supports10.5.  Here's a link to
get 3.1.3 (requires ADC login:)
https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=20414

I see some mention that Xcode 3.1.4 is the latest one (for 10.5), but I
couldn't locate it on Apple's site.

Hope this helps! But you probably will have to contend with another mongo
download!

Cheers,
-Adil

On Wed, Dec 8, 2010 at 10:03 AM, Andi Vajda <[EMAIL PROTECTED]> wrote:

>
> On Wed, 8 Dec 2010, Bill Janssen wrote:
>
>  Andi Vajda <[EMAIL PROTECTED]> wrote:
>>
>>  Yes, two things made me do this:
>>>  1. Bill Janssen said on this list that an Apple developer he talked to
>>>     said that the /Developer tree is the correct one to use.
>>>
>>
>> I asked again, and got this answer.  Looks like I was (wrong? talking
>> about a pre-release version of Snow Leopard?); Mike says to just use
>> /System/Library/Frameworks/JavaVM.framework/Headers, and explains why
>> that doesn't always work, and that they're aware of the problem.
>>
>> So let's just hardcode that path into the setup.py file for "darwin".
>>
>
> That would be the safer thing to do at this point, yes.
>
> I wasn't able to build on 10.5 yesterday (my old G4 powerbook needs some
> refreshing first [1]). Once I confirm 10.5 presents no problems, I'm
> probably going to revert the setup.py change and prepare new release
> artifacts and call for another vote.
>
> Andi..
>
> [1] bitrot story:
>  - the svn I have installed there doesn't support the externals format used
>  - updating svn via MacPorts requires a MacPorts update first
>  - updating MacPorts implies a dev tools update (since it complains about
>    Xcode 3.0 causing "problems")
>  - installing the 3.2.2 devtools on my 10.5.8 G4 powerbook crashes the
>    installer.app, consistently, while "looking for installation volumes"
>  - downloading the 3.2.5 devtools took 3 hrs (3.52GB download, I was not
>    able to find a Xcode-only .dmg, only Xcode+iPhone SDK, iPhone part which
>    is not even supported on 10.5), download completed overnight
>  - no time to continue this until tonight again
>
>
>
>> Bill
>>
>> Subject: Re: how to locate an installed JDK?
>> From: Mike Swingler <[EMAIL PROTECTED]>
>> Date: Wed, 8 Dec 2010 07:13:59 PST
>> Cc: java-dev <[EMAIL PROTECTED]>
>>
>> On Dec 7, 2010, at 7:08 PM, Bill Janssen wrote:
>>
>>  Now that we have /usr/libexec/java_home, is there a standard way to
>>> locate a JDK, with tools like "javac" and "jar" and header files like
>>> "jni.h"?
>>>
>>> The PyLucene project installs a tool called "JCC", which wraps a jar
>>> file as a Python module, but to build JCC, it has to find a Java JDK
>>> with header files.  Is there a standard way of doing that yet?
>>>
>>
>> To find a $JAVA_HOME for the purposes of using the command line tools,
>> please use /usr/libexec/java_home. The man page for "java_home"
>> enumerates all of it's options.
>>
>> As for headers, always use
>> /System/Library/Frameworks/JavaVM.framework/Headers.
>>
>> It may be necessary to install the "Java for Mac OS X 10.6 Update 3
>> Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
>> Package" from <http://connect.apple.com> under the "Java" section, to
>> ensure the native headers are present in the JavaVM.framework. We are
>> aware that this has caused some inconvenience for otherwise native
>> projects that make use of the JNI API, and we are leaning towards just
>> shipping the headers in the regular customer software update package.
>>
>> Regards,
>> Mike Swingler
>> Java Engineering
>> Apple Inc.
>>
>>

On Wed, 8 Dec 2010, Adil Lotia wrote:

> I think Xcode 3.1.[34] is the last one which supports10.5.  Here's a link to
> get 3.1.3 (requires ADC login:)
> https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=20414
>
> I see some mention that Xcode 3.1.4 is the latest one (for 10.5), but I
> couldn't locate it on Apple's site.

I did find it, after looking around for a bit. 993MB download.

Andi..

>
> Hope this helps! But you probably will have to contend with another mongo
> download!
>
> Cheers,
> -Adil
>
> On Wed, Dec 8, 2010 at 10:03 AM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>>
>> On Wed, 8 Dec 2010, Bill Janssen wrote:
>>
>>  Andi Vajda <[EMAIL PROTECTED]> wrote:
>>>
>>>  Yes, two things made me do this:
>>>>  1. Bill Janssen said on this list that an Apple developer he talked to
>>>>     said that the /Developer tree is the correct one to use.
>>>>
>>>
>>> I asked again, and got this answer.  Looks like I was (wrong? talking
>>> about a pre-release version of Snow Leopard?); Mike says to just use
>>> /System/Library/Frameworks/JavaVM.framework/Headers, and explains why
>>> that doesn't always work, and that they're aware of the problem.
>>>
>>> So let's just hardcode that path into the setup.py file for "darwin".
>>>
>>
>> That would be the safer thing to do at this point, yes.
>>
>> I wasn't able to build on 10.5 yesterday (my old G4 powerbook needs some
>> refreshing first [1]). Once I confirm 10.5 presents no problems, I'm
>> probably going to revert the setup.py change and prepare new release
>> artifacts and call for another vote.
>>
>> Andi..
>>
>> [1] bitrot story:
>>  - the svn I have installed there doesn't support the externals format used
>>  - updating svn via MacPorts requires a MacPorts update first
>>  - updating MacPorts implies a dev tools update (since it complains about
>>    Xcode 3.0 causing "problems")
>>  - installing the 3.2.2 devtools on my 10.5.8 G4 powerbook crashes the
>>    installer.app, consistently, while "looking for installation volumes"
>>  - downloading the 3.2.5 devtools took 3 hrs (3.52GB download, I was not
>>    able to find a Xcode-only .dmg, only Xcode+iPhone SDK, iPhone part which
>>    is not even supported on 10.5), download completed overnight
>>  - no time to continue this until tonight again
>>
>>
>>
>>> Bill
>>>
>>> Subject: Re: how to locate an installed JDK?
>>> From: Mike Swingler <[EMAIL PROTECTED]>
>>> Date: Wed, 8 Dec 2010 07:13:59 PST
>>> Cc: java-dev <[EMAIL PROTECTED]>
>>>
>>> On Dec 7, 2010, at 7:08 PM, Bill Janssen wrote:
>>>
>>>  Now that we have /usr/libexec/java_home, is there a standard way to
>>>> locate a JDK, with tools like "javac" and "jar" and header files like
>>>> "jni.h"?
>>>>
>>>> The PyLucene project installs a tool called "JCC", which wraps a jar
>>>> file as a Python module, but to build JCC, it has to find a Java JDK
>>>> with header files.  Is there a standard way of doing that yet?
>>>>
>>>
>>> To find a $JAVA_HOME for the purposes of using the command line tools,
>>> please use /usr/libexec/java_home. The man page for "java_home"
>>> enumerates all of it's options.
>>>
>>> As for headers, always use
>>> /System/Library/Frameworks/JavaVM.framework/Headers.
>>>
>>> It may be necessary to install the "Java for Mac OS X 10.6 Update 3
>>> Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
>>> Package" from <http://connect.apple.com> under the "Java" section, to
>>> ensure the native headers are present in the JavaVM.framework. We are
>>> aware that this has caused some inconvenience for otherwise native
>>> projects that make use of the JNI API, and we are leaning towards just
>>> shipping the headers in the regular customer software update package.
>>>
>>> Regards,
>>> Mike Swingler
>>> Java Engineering
>>> Apple Inc.
>>>
>>>
>

On Wed, 8 Dec 2010, Andi Vajda wrote:

>
> On Wed, 8 Dec 2010, Bill Janssen wrote:
>
>> Andi Vajda <[EMAIL PROTECTED]> wrote:
>>
>>> Yes, two things made me do this:
>>>   1. Bill Janssen said on this list that an Apple developer he talked to
>>>      said that the /Developer tree is the correct one to use.
>>
>> I asked again, and got this answer.  Looks like I was (wrong? talking
>> about a pre-release version of Snow Leopard?); Mike says to just use
>> /System/Library/Frameworks/JavaVM.framework/Headers, and explains why
>> that doesn't always work, and that they're aware of the problem.
>>
>> So let's just hardcode that path into the setup.py file for "darwin".
>
> That would be the safer thing to do at this point, yes.

I finally opted to stay with the current setting under /Developer since, if
there is one version that should work out of the box, it should be the
latest OS/Java combination. Once Apple changes it again, if they do, I can
change it back as well so that more setups work again out of the box.

Andi..

>
> I wasn't able to build on 10.5 yesterday (my old G4 powerbook needs some
> refreshing first [1]). Once I confirm 10.5 presents no problems, I'm probably
> going to revert the setup.py change and prepare new release artifacts and
> call for another vote.
>
> Andi..
>
> [1] bitrot story:
>  - the svn I have installed there doesn't support the externals format used
>  - updating svn via MacPorts requires a MacPorts update first
>  - updating MacPorts implies a dev tools update (since it complains about
>    Xcode 3.0 causing "problems")
>  - installing the 3.2.2 devtools on my 10.5.8 G4 powerbook crashes the
>    installer.app, consistently, while "looking for installation volumes"
>  - downloading the 3.2.5 devtools took 3 hrs (3.52GB download, I was not
>    able to find a Xcode-only .dmg, only Xcode+iPhone SDK, iPhone part which
>    is not even supported on 10.5), download completed overnight
>  - no time to continue this until tonight again
>
>>
>> Bill
>>
>> Subject: Re: how to locate an installed JDK?
>> From: Mike Swingler <[EMAIL PROTECTED]>
>> Date: Wed, 8 Dec 2010 07:13:59 PST
>> Cc: java-dev <[EMAIL PROTECTED]>
>>
>> On Dec 7, 2010, at 7:08 PM, Bill Janssen wrote:
>>
>>> Now that we have /usr/libexec/java_home, is there a standard way to
>>> locate a JDK, with tools like "javac" and "jar" and header files like
>>> "jni.h"?
>>>
>>> The PyLucene project installs a tool called "JCC", which wraps a jar
>>> file as a Python module, but to build JCC, it has to find a Java JDK
>>> with header files.  Is there a standard way of doing that yet?
>>
>> To find a $JAVA_HOME for the purposes of using the command line tools,
>> please use /usr/libexec/java_home. The man page for "java_home"
>> enumerates all of it's options.
>>
>> As for headers, always use
>> /System/Library/Frameworks/JavaVM.framework/Headers.
>>
>> It may be necessary to install the "Java for Mac OS X 10.6 Update 3
>> Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
>> Package" from <http://connect.apple.com> under the "Java" section, to
>> ensure the native headers are present in the JavaVM.framework. We are
>> aware that this has caused some inconvenience for otherwise native
>> projects that make use of the JNI API, and we are leaning towards just
>> shipping the headers in the regular customer software update package.
>>
>> Regards,
>> Mike Swingler
>> Java Engineering
>> Apple Inc.
>>
>

  Hi Adil,

> And for both pylucene-3.0.3-1 (rc) and pylucene-3.0.2-1 when I run the make
> tests, most of the tests run fine, but a couple of them crash Python.
> ...
> /opt/local/bin/python test/test_PhraseQuery.py
> ........
> ----------------------------------------------------------------------
> Ran 8 tests in 0.162s
>
> OK
> /opt/local/bin/python test/test_PositionIncrement.py
> Invalid memory access of location 0x0 eip=0x240ba3b
> make: *** [test] Error 1
>
> I have tested this on three of our 10.5 systems, and it is reproduceable on
> all. (The unit tests complete okay on 10.6). I have tried switching to Java
> 6 and/or Python 2.7, and I still get the same behaviour.  This crash is not
> a show-stopper for us. Pylucene is working fine for what we need.

Hmm, let me try this on my 10.5 system and report back.
If someone else can try this before I get to it, please us know what you
see.

Thanks !

Andi..

>
> Cheers,
> -Adil
>
> On Sat, Dec 4, 2010 at 10:50 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>>
>> With the recent releases of Lucene Java 2.9.4 and 3.0.3, the PyLucene
>> 2.9.4-1 and 3.0.3-1 releases closely tracking them are ready.
>>
>> Release candidates are available from:
>>
>>    http://people.apache.org/~vajda/staging_area/
>>
>> A list of changes in this release can be seen at:
>>
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_9/CHANGES
>>
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_0/CHANGES
>>
>> All versions of PyLucene are built with the same version of JCC, currently
>> version 2.7, included in these release artifacts.
>>
>> A list of Lucene Java changes can be seen at:
>> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9/CHANGES.txt
>> http://svn.apache.org/repos/asf/lucene/java/branches/lucene_3_0/CHANGES.txt
>>
>> Please vote to release these artifacts as PyLucene 2.9.4-1 and 3.0.3-1.
>>
>> Thanks !
>>
>> Andi..
>>
>> ps: the KEYS file for PyLucene release signing is at:
>>    http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
>>    http://people.apache.org/~vajda/staging_area/KEYS
>>
>> pps: here is my +1
>>
>

On Tue, 7 Dec 2010, Andi Vajda wrote:

>
> Hi Adil,
>
>> And for both pylucene-3.0.3-1 (rc) and pylucene-3.0.2-1 when I run the make
>> tests, most of the tests run fine, but a couple of them crash Python.
>> ...
>> /opt/local/bin/python test/test_PhraseQuery.py
>> ........
>> ----------------------------------------------------------------------
>> Ran 8 tests in 0.162s
>>
>> OK
>> /opt/local/bin/python test/test_PositionIncrement.py
>> Invalid memory access of location 0x0 eip=0x240ba3b
>> make: *** [test] Error 1
>>
>> I have tested this on three of our 10.5 systems, and it is reproduceable on
>> all. (The unit tests complete okay on 10.6). I have tried switching to Java
>> 6 and/or Python 2.7, and I still get the same behaviour.  This crash is not
>> a show-stopper for us. Pylucene is working fine for what we need.
>
> Hmm, let me try this on my 10.5 system and report back.
> If someone else can try this before I get to it, please us know what you see.

There was indeed a bug with using Java 1.5 where the JNI version was still
1.4. This caused some cached methods to be NULL instead of correct pointers.

I fixed this on JCC's trunk (also used by the release candidates) and I
should be preparing new release artifacts shortly.

Andi..

Wonderful!

Thank you for the quick review and fix on that one.

Cheers,
-Adil

On Thu, Dec 9, 2010 at 1:55 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:

>
> On Tue, 7 Dec 2010, Andi Vajda wrote:
>
>
>> Hi Adil,
>>
>>  And for both pylucene-3.0.3-1 (rc) and pylucene-3.0.2-1 when I run the
>>> make
>>> tests, most of the tests run fine, but a couple of them crash Python.
>>> ...
>>> /opt/local/bin/python test/test_PhraseQuery.py
>>> ........
>>> ----------------------------------------------------------------------
>>> Ran 8 tests in 0.162s
>>>
>>> OK
>>> /opt/local/bin/python test/test_PositionIncrement.py
>>> Invalid memory access of location 0x0 eip=0x240ba3b
>>> make: *** [test] Error 1
>>>
>>> I have tested this on three of our 10.5 systems, and it is reproduceable
>>> on
>>> all. (The unit tests complete okay on 10.6). I have tried switching to
>>> Java
>>> 6 and/or Python 2.7, and I still get the same behaviour.  This crash is
>>> not
>>> a show-stopper for us. Pylucene is working fine for what we need.
>>>
>>
>> Hmm, let me try this on my 10.5 system and report back.
>> If someone else can try this before I get to it, please us know what you
>> see.
>>
>
> There was indeed a bug with using Java 1.5 where the JNI version was still
> 1.4. This caused some cached methods to be NULL instead of correct pointers.
>
> I fixed this on JCC's trunk (also used by the release candidates) and I
> should be preparing new release artifacts shortly.
>
> Andi..
>

>>>>> "Adil" == Adil Lotia <[EMAIL PROTECTED]> writes:

    Adil> Hi Andi I downloaded the release candidate from your staging
    Adil> area, and will be running pylucene on OS X.  I am not sure if
    Adil> this is intentional, but was there a reason for this specific
    Adil> change?

    Adil> adil:681 lucene $ diff pylucene-3.0.2-1/jcc/setup.py
    Adil> pylucene-3.0.3-1/jcc/setup.py ...  47c47 < 'darwin':
    Adil> '/System/Library/Frameworks/JavaVM.framework/Versions/Current',
    Adil> ---
    >> 'darwin':
    Adil> '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
    Adil> ...

    Adil> I have the standard JDK installation, which is in /System.

As I understand it, Apple is preparing to move to something like the JRE
under /System, and the JDK under /Developer/...  (which sort of makes
sense).  So development tools like "jar" and "javac" will move to the
/Developer tree.

    Adil> Also, this is my system config: adil:689 lucene $
    Adil> /usr/bin/sw_vers ProductName: Mac OS X ProductVersion: 10.5.8
    Adil> BuildVersion: 9L31a

    Adil> adil:690 lucene $ python -V Python 2.6.6

    Adil> adil:691 lucene $ java -version java version "1.5.0_26"
    Adil> Java(TM) 2 Runtime Environment, Standard Edition (build
    Adil> 1.5.0_26-b03-376-9M3263) Java HotSpot(TM) Client VM (build
    Adil> 1.5.0_26-156, mixed mode, sharing) adil:692 lucene

This is a pretty non-standard Mac.  OS X 10.5.8 uses Python 2.5.1, not
2.6.6.  And the "standard" Java is 1.6.0_22, though I also usually
compile with 1.5.

Bill

On Dec 10, 2010, at 10:24, Bill Janssen <[EMAIL PROTECTED]> wrote:

> Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>> A bug was reported against the previous release artifacts that  
>> blocked the
>> release. The bug got fixed and new artifacts were uploaded, please
>> vote again.
>
> This new candidate breaks immediately on OS X 10.5.8 (standard system
> Python, standard system Java):
>
>  /tmp 18 % cd pylucene-3.0.3-1/jcc
>  /tmp/pylucene-3.0.3-1/jcc 19 % python setup.py build
>  Traceback (most recent call last):
>    File "setup.py", line 75, in <module>
>      ''' %(JDK[platform]))
>  RuntimeError:
>
>  Java JDK directory '/Developer/SDKs/MacOSX10.6.sdk/System/Library/
> Frameworks/JavaVM.framework/Versions/Current' does not exist.
>
>  Please set the environment variable JCC_JDK to the correct location  
> before
>  running setup.py.
>  %
>
> Andi, if you're going to just hardcode a JDK path in for OS X,  
> please at
> least hardcode the path advised by the Apple Java engineers (see
> previous mail), instead of one that's guaranteed to break on 10.5
> machines.  I say "guaranteed", because the Xcode for 10.5 doesn't
> include a 10.6 SDK, and if it did it wouldn't fit the installed Java.

So, it's "broken on 10.6 out of the box" vs "broken on 10.5 out of the  
box" or am I missing something else here ?

Andi..

>
> Bill

Andi Vajda <[EMAIL PROTECTED]> wrote:

> So, it's "broken on 10.6 out of the box" vs "broken on 10.5 out of the
> box" or am I missing something else here ?

(1) The patch I sent should work on both, out of the box.

(2) Alternatively, if you hard-code the path as recommended by Mike
Swingler, that should work on both (possibly requiring the additional
download of a developer package for the headers -- but that's just part
of installing the Java developer tools).

Bill

On Fri, 10 Dec 2010, Bill Janssen wrote:

> Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>> So, it's "broken on 10.6 out of the box" vs "broken on 10.5 out of the
>> box" or am I missing something else here ?
>
> (1) The patch I sent should work on both, out of the box.
>
> (2) Alternatively, if you hard-code the path as recommended by Mike
> Swingler, that should work on both (possibly requiring the additional
> download of a developer package for the headers -- but that's just part
> of installing the Java developer tools).

If I switch to Mike's recommendation then 10.6 is broken out of the box.
I don't know what developer package he's referring to. Can you please send
me the URL ?

Yes, your patch should work on both but would it not be obsoleted by the
next time Apple moves these headers around again and setup.py is reverted
back to the /System value in consequence ?

I guess the crux of the issue/disagreement here is that you somehow have the
expectation that JCC should build out of the box whereas I've always made it
clear that the "out of the box" settings in Makefile and setup.py are only
best guesses that need to be reviewed/edited and that any automation to make
it build out of the box will make it harder for people to debug when said
automation fails in obscure ways.

I really want the latest up to date Mac OS X setup to build out of the box
and it seems that the current setup.py settings are the only way unless your
patch get added ?

Andi..

Andi Vajda <[EMAIL PROTECTED]> wrote:

> If I switch to Mike's recommendation then 10.6 is broken out of the box.
> I don't know what developer package he's referring to. Can you please
> send me the URL ?

Here's what his message said:

``It may be necessary to install the "Java for Mac OS X 10.6 Update 3
Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
Package" from <http://connect.apple.com> under the "Java" section, to
ensure the native headers are present in the JavaVM.framework. We are
aware that this has caused some inconvenience for otherwise native
projects that make use of the JNI API, and we are leaning towards just
shipping the headers in the regular customer software update package.''

That's the "developer package" he's referring to.

> Yes, your patch should work on both but would it not be obsoleted by
> the next time Apple moves these headers around again and setup.py is
> reverted back to the /System value in consequence ?
>
> I really want the latest up to date Mac OS X setup to build out of the
> box

Sure, me too!

> ...and it seems that the current setup.py settings are the only way
> unless your patch get added ?

My patch soft-codes the location of the SDK, so that it works with both
10.5 and 10.6.  Though, after hearing from Swingler, I'd revise it as
follows (I just did this in Emacs, haven't tested it):

diff -u pylucene-3.0.3-1/jcc/setup.py.orig pylucene-3.0.3-1/jcc/setup.py
--- pylucene-3.0.3-1/jcc/setup.py.orig 2010-12-07 18:50:25.000000000 -0800
+++ pylucene-3.0.3-1/jcc/setup.py 2010-12-07 19:02:44.000000000 -0800
@@ -43,8 +43,23 @@
 else:
     JAVAHOME = None
 
+if platform in ("darwin"):
+    # possibly we shouldn't overwrite the platform module in the first place
+    import platform as _platform
+    import re
+    JAVAHOME = None
+    path = "/System/Library/Frameworks/JavaVM.framework/"
+    if os.path.isdir(path) and os.path.exists(os.path.join(path, "Headers", "jni.h")):
+        JAVAHOME = os.path.join(path, "Home")
+    else:
+        path = ("/Developer/SDKs/MacOSX" +
+                 re.match("[0-9]+\.[0-9]+", _platform.mac_ver()[0]).group(0) +
+                 ".sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current")
+        if os.path.exists(path):
+            JAVAHOME = path
+
 JDK = {
-    'darwin': '/Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/Current',
+    'darwin': JAVAHOME,
     'ipod': '/usr/include/gcc',
     'linux2': '/usr/lib/jvm/java-6-openjdk',
     'sunos5': '/usr/jdk/instances/jdk1.6.0',

In other words, first look for the headers where Apple says they should
be, then if they aren't there, look in the Developer SDK, but the SDK
for the current version of OS X.

Philosophically, I see little point in avoiding the use of an actual
programming language that using setup.py gives you.  Otherwise, might as
well use another Makefile.

Bill

On Fri, 10 Dec 2010, Bill Janssen wrote:

> Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>> If I switch to Mike's recommendation then 10.6 is broken out of the box.
>> I don't know what developer package he's referring to. Can you please
>> send me the URL ?
>
> Here's what his message said:
>
> ``It may be necessary to install the "Java for Mac OS X 10.6 Update 3
> Developer Package" or "Java for Mac OS X 10.5 Update 8 Developer
> Package" from <http://connect.apple.com> under the "Java" section, to
> ensure the native headers are present in the JavaVM.framework. We are
> aware that this has caused some inconvenience for otherwise native
> projects that make use of the JNI API, and we are leaning towards just
> shipping the headers in the regular customer software update package.''
>
> That's the "developer package" he's referring to.
>
>> Yes, your patch should work on both but would it not be obsoleted by
>> the next time Apple moves these headers around again and setup.py is
>> reverted back to the /System value in consequence ?
>>
>> I really want the latest up to date Mac OS X setup to build out of the
>> box
>
> Sure, me too!

No, not today. Today's out of the box on 10.6 requires the setting to be
under /Developer (without your patch). Downloading another Java package !=
out of the box. By out of the box, I mean current Xcode + whatever software
update drops in on a regular basis, like the last Java update that lost the
headers.

>> ...and it seems that the current setup.py settings are the only way
>> unless your patch get added ?
>
> My patch soft-codes the location of the SDK, so that it works with both
> 10.5 and 10.6.  Though, after hearing from Swingler, I'd revise it as
> follows (I just did this in Emacs, haven't tested it):

Ok then, provide a tested patch that works and applies out of the box to
JCC's trunk, and that puts this code into a new helper file like is done for
linux and windows (thus not adding pages of code to setup.py) and I'll
integrate it.

> In other words, first look for the headers where Apple says they should
> be, then if they aren't there, look in the Developer SDK, but the SDK
> for the current version of OS X.
>
> Philosophically, I see little point in avoiding the use of an actual
> programming language that using setup.py gives you.  Otherwise, might as
> well use another Makefile.

You're not adressing the issue here: obscure automatic failures vs simple
edits in setup.py to match your system.

Andi..

Andi Vajda <[EMAIL PROTECTED]> wrote:

> >> I really want the latest up to date Mac OS X setup to build out of the
> >> box
> >
> > Sure, me too!
>
> No, not today. Today's out of the box on 10.6 requires the setting to
> be under /Developer (without your patch).

Well, of course, that's why I sent you the patch :-).

> Downloading another Java package != out of the box. By out of the box,
> I mean current Xcode + whatever software update drops in on a regular
> basis, like the last Java update that lost the headers.

But what Apple is saying is that to install "current XCode" right now,
for Java, requires installing that extra download.  The current slogan
there is that being a Java developer on OS X requires "XCode plus
patches", apparently.

> >> ...and it seems that the current setup.py settings are the only way
> >> unless your patch get added ?

Of course, that will break on Lion when that's the latest and greatest.

> > My patch soft-codes the location of the SDK, so that it works with both
> > 10.5 and 10.6.  Though, after hearing from Swingler, I'd revise it as
> > follows (I just did this in Emacs, haven't tested it):
>
> Ok then, provide a tested patch that works and applies out of the box
> to JCC's trunk, and that puts this code into a new helper file like is
> done for linux and windows (thus not adding pages of code to setup.py)
> and I'll integrate it.

OK, will do.  Wasn't sure about your feeling about it, or I would have
done it sooner.

> > Philosophically, I see little point in avoiding the use of an actual
> > programming language that using setup.py gives you.  Otherwise, might as
> > well use another Makefile.
>
> You're not adressing the issue here: obscure automatic failures vs
> simple edits in setup.py to match your system.

Not going to touch that one... :-)

Bill

Andi Vajda <[EMAIL PROTECTED]> wrote:

> Ok then, provide a tested patch that works and applies out of the box
> to JCC's trunk, and that puts this code into a new helper file like is
> done for linux and windows (thus not adding pages of code to setup.py)
> and I'll integrate it.

Here it is.  I happened to have both old and new Snow Leopard machines
around to test it on, as well as my Leopard machines.  Works on all of
them.  This applies to the trunk of pylucene.

Bill

On Fri, 10 Dec 2010, Bill Janssen wrote:

> Andi Vajda <[EMAIL PROTECTED]> wrote:
>
>> Ok then, provide a tested patch that works and applies out of the box
>> to JCC's trunk, and that puts this code into a new helper file like is
>> done for linux and windows (thus not adding pages of code to setup.py)
>> and I'll integrate it.
>
> Here it is.  I happened to have both old and new Snow Leopard machines
> around to test it on, as well as my Leopard machines.  Works on all of
> them.  This applies to the trunk of pylucene.

Thanks, I integrated it with minor changes.
I also tested it on 10.6 and 10.5.

New release artifacts are under way...

Andi..