Hello,

I would like to open a discussion about release frequency from
lucene/solr's 3x branch. I'm not asking for votes or anything, just
ideas.

For Lucene/Solr its been a pretty long time since the users got a
feature release.
I don't consider Lucene 3.0 as a feature release either.

I think now that we have a "trunk" for unstable development, and a
"3x" stable branch, that we should think about cutting releases from
this branch much more often, for example every month or two.

I think that by doing this, we will engage the community more: because
many people won't run svn checkouts/snapshots, and many people
probably wont even look at unreleased code.

In the past it seems releases were fairly infrequent, and I'm not sure
I have the background to really understand why, but i have 3 theories:
* concerns about the actual code being stable
* the release process is too complicated
* getting someone to do the work

For stability, my argument is that our "3x" stable branch is
inherently more stable than previous trunks were, so its safe to
release more often from it. I give some very rough, very unscientific
numbers below from Lucene's JIRA, but I think the same applies to
Solr.

Based on the last 4 weeks of development (resolved issues):

Of bugfixes, about half (6) are fixes to bugs that already existed in 2.9/3.0
About 25% (3) are fixes to bugs we only introduced in trunk, and do
not affect 3x.
The other 25% (4) are fixes to things we introduced in trunk/3x

You could say this suggests 3x is very roughly "twice" as stable as
trunk, yet has about 80% of the new features/improvements (14 out of
17).

With regards to the release process: I also think if we decide to do
this, we must simplify and automate our release procedures.
To be frank, http://wiki.apache.org/lucene-java/ReleaseTodo scares the
crap out of me, and something like 'ant release' that someone can run
from the top level for both Lucene and Solr would really go a long way
towards making the release process less painful. I realize this is
difficult and cannot be fully automated but I think it can be
improved.

Finally, as far as getting someone to do the work, I can certainly
volunteer to help in the following ways:
* being RM if you are ok with a non-maven release (until LUCENE-2268
is fixed, i am uncomfortable with maven)
* improving the build to simplify the release process: (see SOLR-2002
for a start)
--
Robert Muir
[EMAIL PROTECTED]

+1 to simplify the release process / ReleaseTodo wiki, and +1 to
release a 3.1, and +1 to do frequent stable releases.

Having a stable branch gives us that freedom and we should use it!

Mike

On Thu, Sep 9, 2010 at 2:41 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I would like to open a discussion about release frequency from lucene/solr's
> 3x branch. I'm not asking for votes or anything, just ideas.
>
> For Lucene/Solr its been a pretty long time since the users got a feature
> release.
> I don't consider Lucene 3.0 as a feature release either.
>
> I think now that we have a "trunk" for unstable development, and a "3x"
> stable branch, that we should think about cutting releases from this branch
> much more often, for example every month or two.
>
> I think that by doing this, we will engage the community more: because many
> people won't run svn checkouts/snapshots, and many people probably wont even
> look at unreleased code.
>
> In the past it seems releases were fairly infrequent, and I'm not sure I
> have the background to really understand why, but i have 3 theories:
> * concerns about the actual code being stable
> * the release process is too complicated
> * getting someone to do the work
>
> For stability, my argument is that our "3x" stable branch is inherently more
> stable than previous trunks were, so its safe to release more often from it.
> I give some very rough, very unscientific numbers below from Lucene's JIRA,
> but I think the same applies to Solr.
>
> Based on the last 4 weeks of development (resolved issues):
>
> Of bugfixes, about half (6) are fixes to bugs that already existed in
> 2.9/3.0
> About 25% (3) are fixes to bugs we only introduced in trunk, and do not
> affect 3x.
> The other 25% (4) are fixes to things we introduced in trunk/3x
>
> You could say this suggests 3x is very roughly "twice" as stable as trunk,
> yet has about 80% of the new features/improvements (14 out of 17).
>
> With regards to the release process: I also think if we decide to do this,
> we must simplify and automate our release procedures.
> To be frank, http://wiki.apache.org/lucene-java/ReleaseTodo scares the crap
> out of me, and something like 'ant release' that someone can run from the
> top level for both Lucene and Solr would really go a long way towards making
> the release process less painful. I realize this is difficult and cannot be
> fully automated but I think it can be improved.
>
> Finally, as far as getting someone to do the work, I can certainly volunteer
> to help in the following ways:
> * being RM if you are ok with a non-maven release (until LUCENE-2268 is
> fixed, i am uncomfortable with maven)
> * improving the build to simplify the release process: (see SOLR-2002 for a
> start)
>
> --
> Robert Muir
> [EMAIL PROTECTED]
>

---------------------------------------------------------------------

+1 here too - I'm all for more frequent releases - not sure how much
time I can put behind that release to release, but I'm all for the idea
of it.

- Mark

On 9/9/10 5:54 PM, Michael McCandless wrote:
> +1 to simplify the release process / ReleaseTodo wiki, and +1 to
> release a 3.1, and +1 to do frequent stable releases.
>
> Having a stable branch gives us that freedom and we should use it!
>
> Mike
>
> On Thu, Sep 9, 2010 at 2:41 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> I would like to open a discussion about release frequency from lucene/solr's
>> 3x branch. I'm not asking for votes or anything, just ideas.
>>
>> For Lucene/Solr its been a pretty long time since the users got a feature
>> release.
>> I don't consider Lucene 3.0 as a feature release either.
>>
>> I think now that we have a "trunk" for unstable development, and a "3x"
>> stable branch, that we should think about cutting releases from this branch
>> much more often, for example every month or two.
>>
>> I think that by doing this, we will engage the community more: because many
>> people won't run svn checkouts/snapshots, and many people probably wont even
>> look at unreleased code.
>>
>> In the past it seems releases were fairly infrequent, and I'm not sure I
>> have the background to really understand why, but i have 3 theories:
>> * concerns about the actual code being stable
>> * the release process is too complicated
>> * getting someone to do the work
>>
>> For stability, my argument is that our "3x" stable branch is inherently more
>> stable than previous trunks were, so its safe to release more often from it.
>> I give some very rough, very unscientific numbers below from Lucene's JIRA,
>> but I think the same applies to Solr.
>>
>> Based on the last 4 weeks of development (resolved issues):
>>
>> Of bugfixes, about half (6) are fixes to bugs that already existed in
>> 2.9/3.0
>> About 25% (3) are fixes to bugs we only introduced in trunk, and do not
>> affect 3x.
>> The other 25% (4) are fixes to things we introduced in trunk/3x
>>
>> You could say this suggests 3x is very roughly "twice" as stable as trunk,
>> yet has about 80% of the new features/improvements (14 out of 17).
>>
>> With regards to the release process: I also think if we decide to do this,
>> we must simplify and automate our release procedures.
>> To be frank, http://wiki.apache.org/lucene-java/ReleaseTodo scares the crap
>> out of me, and something like 'ant release' that someone can run from the
>> top level for both Lucene and Solr would really go a long way towards making
>> the release process less painful. I realize this is difficult and cannot be
>> fully automated but I think it can be improved.
>>
>> Finally, as far as getting someone to do the work, I can certainly volunteer
>> to help in the following ways:
>> * being RM if you are ok with a non-maven release (until LUCENE-2268 is
>> fixed, i am uncomfortable with maven)
>> * improving the build to simplify the release process: (see SOLR-2002 for a
>> start)
>>
>> --
>> Robert Muir
>> [EMAIL PROTECTED]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------

+1 to more frequent releases.  I think we are already seeing 3x being pushed
forward as the stable version for users wishing to do some development of
their own.

I am more than happy to work with whoever on the maven issues as I feel that
it is really important to keep making maven releases.  Maven does have a
release plugin, similar to what you want to build for ant, perhaps that can
provide some inspiration?

Chris

On Fri, Sep 10, 2010 at 12:06 PM, Mark Miller <[EMAIL PROTECTED]> wrote:

> +1 here too - I'm all for more frequent releases - not sure how much
> time I can put behind that release to release, but I'm all for the idea
> of it.
>
> - Mark
>
> On 9/9/10 5:54 PM, Michael McCandless wrote:
> > +1 to simplify the release process / ReleaseTodo wiki, and +1 to
> > release a 3.1, and +1 to do frequent stable releases.
> >
> > Having a stable branch gives us that freedom and we should use it!
> >
> > Mike
> >
> > On Thu, Sep 9, 2010 at 2:41 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> >> Hello,
> >>
> >> I would like to open a discussion about release frequency from
> lucene/solr's
> >> 3x branch. I'm not asking for votes or anything, just ideas.
> >>
> >> For Lucene/Solr its been a pretty long time since the users got a
> feature
> >> release.
> >> I don't consider Lucene 3.0 as a feature release either.
> >>
> >> I think now that we have a "trunk" for unstable development, and a "3x"
> >> stable branch, that we should think about cutting releases from this
> branch
> >> much more often, for example every month or two.
> >>
> >> I think that by doing this, we will engage the community more: because
> many
> >> people won't run svn checkouts/snapshots, and many people probably wont
> even
> >> look at unreleased code.
> >>
> >> In the past it seems releases were fairly infrequent, and I'm not sure I
> >> have the background to really understand why, but i have 3 theories:
> >> * concerns about the actual code being stable
> >> * the release process is too complicated
> >> * getting someone to do the work
> >>
> >> For stability, my argument is that our "3x" stable branch is inherently
> more
> >> stable than previous trunks were, so its safe to release more often from
> it.
> >> I give some very rough, very unscientific numbers below from Lucene's
> JIRA,
> >> but I think the same applies to Solr.
> >>
> >> Based on the last 4 weeks of development (resolved issues):
> >>
> >> Of bugfixes, about half (6) are fixes to bugs that already existed in
> >> 2.9/3.0
> >> About 25% (3) are fixes to bugs we only introduced in trunk, and do not
> >> affect 3x.
> >> The other 25% (4) are fixes to things we introduced in trunk/3x
> >>
> >> You could say this suggests 3x is very roughly "twice" as stable as
> trunk,
> >> yet has about 80% of the new features/improvements (14 out of 17).
> >>
> >> With regards to the release process: I also think if we decide to do
> this,
> >> we must simplify and automate our release procedures.
> >> To be frank, http://wiki.apache.org/lucene-java/ReleaseTodo scares the
> crap
> >> out of me, and something like 'ant release' that someone can run from
> the
> >> top level for both Lucene and Solr would really go a long way towards
> making
> >> the release process less painful. I realize this is difficult and cannot
> be
> >> fully automated but I think it can be improved.
> >>
> >> Finally, as far as getting someone to do the work, I can certainly
> volunteer
> >> to help in the following ways:
> >> * being RM if you are ok with a non-maven release (until LUCENE-2268 is
> >> fixed, i am uncomfortable with maven)
> >> * improving the build to simplify the release process: (see SOLR-2002
> for a
> >> start)
> >>
> >> --
Chris Male | Software Developer | JTeam BV.| www.jteam.nl

My unscientific, off the cuff, sociological impression is that once we
moved forward with
the "multi-branch" development plan and created the 3x branch, a lot of
people who use to be the big proponents of regular releases got really
about the freedom involved in working on the trunk, and lost their
motivation to push for releases - because a big part of that motivation
came from the backcompat concerns and the need to churn out releasees
with deprecations so that future versions could move forward ith more
interesting APIs.  "trunk" turned into the new hot sexiness.

but like i said: that's just my unscientific impression.

: I think now that we have a "trunk" for unstable development, and a
: "3x" stable branch, that we should think about cutting releases from
: this branch much more often, for example every month or two.

I think that might be overly ambitious, particularly because we've never
really talked about how the release process for the 3x branch *should*
work (given the lucene/solr development merged) let alone start on those
changes to make it easy (that process doc that scares the crap out of you
is just for Lucene-Java, there's an equally scaray one for Solr)
I think it's going to take some work just on build/process before we can
get our first "merged" release from 3x.  Assuming we improve some
automation while we're at it, then i think it's feasible that we could
start doing releases off of it every couple of months.  it would remain
to be seen wether we sould *need* to release that often -- it will
depend on wether anything new gets committed there -- but it would
certianly be nice to be able to.

: Finally, as far as getting someone to do the work, I can certainly
: volunteer to help in the following ways:
: * being RM if you are ok with a non-maven release (until LUCENE-2268
: is fixed, i am uncomfortable with maven)

maven is such a contentious issue -- people either don't give a shit
about it, or think it's the end of the world if the jars aren't there.

In the past i've argued that enough users care about maven we should
really try to make sure we play nicely, but the more i think about it the
less i think it should be part of the release process.  

the ASF releases source code.  When we vote on a release, tha's what we
are voting on: source.  We may also distribute precompiled binary jars
via the download mirrors, or via maven, but that's not what the release is
about -- so let's get hte pom template files out of hte src tree, let's
get the maven related tasks out of the build.xml file and treat publishing
to maven as a seperate process that happens *after* the release.  We vote
on the release, we release it, and then the folks that care about maven
can publish the jars after the fact.

-Hoss

--
http://lucenerevolution.org/  ...  October 7-8, Boston
http://bit.ly/stump-hoss      ...  Stump The Chump!
---------------------------------------------------------------------

On 9/17/10 9:45 PM, Chris Hostetter wrote:
 When we vote on a release, tha's what we
> are voting on: source.  We may also distribute precompiled binary jars
> via the download mirrors, or via maven, but that's not what the release is
> about -- so let's get hte pom template files out of hte src tree, let's
> get the maven related tasks out of the build.xml file and treat publishing
> to maven as a seperate process that happens *after* the release.  We vote
> on the release, we release it, and then the folks that care about maven
> can publish the jars after the fact.
>
Amen.

- Mark

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 2:45 PM, Mark Miller <[EMAIL PROTECTED]> wrote:

> On 9/17/10 9:45 PM, Chris Hostetter wrote:
>  When we vote on a release, tha's what we
> > are voting on: source.  We may also distribute precompiled binary jars
> > via the download mirrors, or via maven, but that's not what the release
> is
> > about -- so let's get hte pom template files out of hte src tree, let's
> > get the maven related tasks out of the build.xml file and treat
> publishing
> > to maven as a seperate process that happens *after* the release.  We vote
> > on the release, we release it, and then the folks that care about maven
> > can publish the jars after the fact.
> >
>

Given how messy and problem plagued the maven releases code is currently, +1
to this idea.

--
Chris Male | Software Developer | JTeam BV.| www.jteam.nl

+1 on the ant-only policy. I've recently been futzing with Mahout and I
had not been faced with the scrofulous horror of Maven. Please keep it
out of the main source tree of solr. You can do whatever you want with
the internal Apache build process.

Chris Hostetter wrote:
> My unscientific, off the cuff, sociological impression is that once we
> moved forward with
> the "multi-branch" development plan and created the 3x branch, a lot of
> people who use to be the big proponents of regular releases got really
> about the freedom involved in working on the trunk, and lost their
> motivation to push for releases - because a big part of that motivation
> came from the backcompat concerns and the need to churn out releasees
> with deprecations so that future versions could move forward ith more
> interesting APIs.  "trunk" turned into the new hot sexiness.
>
> but like i said: that's just my unscientific impression.
>
> : I think now that we have a "trunk" for unstable development, and a
> : "3x" stable branch, that we should think about cutting releases from
> : this branch much more often, for example every month or two.
>
> I think that might be overly ambitious, particularly because we've never
> really talked about how the release process for the 3x branch *should*
> work (given the lucene/solr development merged) let alone start on those
> changes to make it easy (that process doc that scares the crap out of you
> is just for Lucene-Java, there's an equally scaray one for Solr)
>
>
> I think it's going to take some work just on build/process before we can
> get our first "merged" release from 3x.  Assuming we improve some
> automation while we're at it, then i think it's feasible that we could
> start doing releases off of it every couple of months.  it would remain
> to be seen wether we sould *need* to release that often -- it will
> depend on wether anything new gets committed there -- but it would
> certianly be nice to be able to.
>
> : Finally, as far as getting someone to do the work, I can certainly
> : volunteer to help in the following ways:
> : * being RM if you are ok with a non-maven release (until LUCENE-2268
> : is fixed, i am uncomfortable with maven)
>
> maven is such a contentious issue -- people either don't give a shit
> about it, or think it's the end of the world if the jars aren't there.
>
> In the past i've argued that enough users care about maven we should
> really try to make sure we play nicely, but the more i think about it the
> less i think it should be part of the release process.
>
> the ASF releases source code.  When we vote on a release, tha's what we
> are voting on: source.  We may also distribute precompiled binary jars
> via the download mirrors, or via maven, but that's not what the release is
> about -- so let's get hte pom template files out of hte src tree, let's
> get the maven related tasks out of the build.xml file and treat publishing
> to maven as a seperate process that happens *after* the release.  We vote
> on the release, we release it, and then the folks that care about maven
> can publish the jars after the fact.
>
>
>
>
>
> -Hoss
>
> --
> http://lucenerevolution.org/  ...  October 7-8, Boston
> http://bit.ly/stump-hoss      ...  Stump The Chump!
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>    

---------------------------------------------------------------------

Helau!

(means +1)

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Lance Norskog [mailto:[EMAIL PROTECTED]]
> Sent: Friday, September 17, 2010 10:19 PM
> To: dev@lucene.apache.org
> Subject: Re: discussion about release frequency.
>
> +1 on the ant-only policy. I've recently been futzing with Mahout and I
> had not been faced with the scrofulous horror of Maven. Please keep it out
of
> the main source tree of solr. You can do whatever you want with the
internal
> Apache build process.
>
> Chris Hostetter wrote:
> > My unscientific, off the cuff, sociological impression is that once we
> > moved forward with the "multi-branch" development plan and created the
> > 3x branch, a lot of people who use to be the big proponents of regular
> > releases got really about the freedom involved in working on the
> > trunk, and lost their motivation to push for releases - because a big
> > part of that motivation came from the backcompat concerns and the need
> > to churn out releasees with deprecations so that future versions could
> > move forward ith more interesting APIs.  "trunk" turned into the new
> > hot sexiness.
> >
> > but like i said: that's just my unscientific impression.
> >
> > : I think now that we have a "trunk" for unstable development, and a
> > : "3x" stable branch, that we should think about cutting releases from
> > : this branch much more often, for example every month or two.
> >
> > I think that might be overly ambitious, particularly because we've
> > never really talked about how the release process for the 3x branch
> > *should* work (given the lucene/solr development merged) let alone
> > start on those changes to make it easy (that process doc that scares
> > the crap out of you is just for Lucene-Java, there's an equally scaray
> > one for Solr)
> >
> >
> > I think it's going to take some work just on build/process before we
> > can get our first "merged" release from 3x.  Assuming we improve some
> > automation while we're at it, then i think it's feasible that we could
> > start doing releases off of it every couple of months.  it would
> > remain to be seen wether we sould *need* to release that often -- it
> > will depend on wether anything new gets committed there -- but it
> > would certianly be nice to be able to.
> >
> > : Finally, as far as getting someone to do the work, I can certainly
> > : volunteer to help in the following ways:
> > : * being RM if you are ok with a non-maven release (until LUCENE-2268
> > : is fixed, i am uncomfortable with maven)
> >
> > maven is such a contentious issue -- people either don't give a shit
> > about it, or think it's the end of the world if the jars aren't there.
> >
> > In the past i've argued that enough users care about maven we should
> > really try to make sure we play nicely, but the more i think about it
> > the less i think it should be part of the release process.
> >
> > the ASF releases source code.  When we vote on a release, tha's what
> > we are voting on: source.  We may also distribute precompiled binary
> > jars via the download mirrors, or via maven, but that's not what the
> > release is about -- so let's get hte pom template files out of hte src
> > tree, let's get the maven related tasks out of the build.xml file and
> > treat publishing to maven as a seperate process that happens *after*
> > the release.  We vote on the release, we release it, and then the
> > folks that care about maven can publish the jars after the fact.
> >
> >
> >
> >
> >
> > -Hoss
> >
> > --
> > http://lucenerevolution.org/  ...  October 7-8, Boston
> > http://bit.ly/stump-hoss      ...  Stump The Chump!
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For

On Sat, Sep 18, 2010 at 7:52 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Helau!

LOL - that made my day uwe :D
>
> (means +1)

same here! +1
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
>> -----Original Message-----
>> From: Lance Norskog [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, September 17, 2010 10:19 PM
>> To: dev@lucene.apache.org
>> Subject: Re: discussion about release frequency.
>>
>> +1 on the ant-only policy. I've recently been futzing with Mahout and I
>> had not been faced with the scrofulous horror of Maven. Please keep it out
> of
>> the main source tree of solr. You can do whatever you want with the
> internal
>> Apache build process.
>>
>> Chris Hostetter wrote:
>> > My unscientific, off the cuff, sociological impression is that once we
>> > moved forward with the "multi-branch" development plan and created the
>> > 3x branch, a lot of people who use to be the big proponents of regular
>> > releases got really about the freedom involved in working on the
>> > trunk, and lost their motivation to push for releases - because a big
>> > part of that motivation came from the backcompat concerns and the need
>> > to churn out releasees with deprecations so that future versions could
>> > move forward ith more interesting APIs.  "trunk" turned into the new
>> > hot sexiness.
>> >
>> > but like i said: that's just my unscientific impression.
>> >
>> > : I think now that we have a "trunk" for unstable development, and a
>> > : "3x" stable branch, that we should think about cutting releases from
>> > : this branch much more often, for example every month or two.
>> >
>> > I think that might be overly ambitious, particularly because we've
>> > never really talked about how the release process for the 3x branch
>> > *should* work (given the lucene/solr development merged) let alone
>> > start on those changes to make it easy (that process doc that scares
>> > the crap out of you is just for Lucene-Java, there's an equally scaray
>> > one for Solr)
>> >
>> >
>> > I think it's going to take some work just on build/process before we
>> > can get our first "merged" release from 3x.  Assuming we improve some
>> > automation while we're at it, then i think it's feasible that we could
>> > start doing releases off of it every couple of months.  it would
>> > remain to be seen wether we sould *need* to release that often -- it
>> > will depend on wether anything new gets committed there -- but it
>> > would certianly be nice to be able to.
>> >
>> > : Finally, as far as getting someone to do the work, I can certainly
>> > : volunteer to help in the following ways:
>> > : * being RM if you are ok with a non-maven release (until LUCENE-2268
>> > : is fixed, i am uncomfortable with maven)
>> >
>> > maven is such a contentious issue -- people either don't give a shit
>> > about it, or think it's the end of the world if the jars aren't there.
>> >
>> > In the past i've argued that enough users care about maven we should
>> > really try to make sure we play nicely, but the more i think about it
>> > the less i think it should be part of the release process.
>> >
>> > the ASF releases source code.  When we vote on a release, tha's what
>> > we are voting on: source.  We may also distribute precompiled binary
>> > jars via the download mirrors, or via maven, but that's not what the
>> > release is about -- so let's get hte pom template files out of hte src
>> > tree, let's get the maven related tasks out of the build.xml file and
>> > treat publishing to maven as a seperate process that happens *after*
>> > the release.  We vote on the release, we release it, and then the
>> > folks that care about maven can publish the jars after the fact.
>> >
>> >
>> >
>> >
>> >
>> > -Hoss
>> >
>> >

> : Finally, as far as getting someone to do the work, I can certainly
> : volunteer to help in the following ways:
> : * being RM if you are ok with a non-maven release (until LUCENE-2268
> : is fixed, i am uncomfortable with maven)
>
> In the past i've argued that enough users care about maven we should
> really try to make sure we play nicely, but the more i think about it the
> less i think it should be part of the release process.

+1 - release managers shouldn't have to be in the love-Maven camp, which seems to be mighty small here in Lucene/Solr dev-land.  User-land, however, is likely a very different story.

> the ASF releases source code.  When we vote on a release, tha's what we
> are voting on: source.  We may also distribute precompiled binary jars
> via the download mirrors, or via maven, but that's not what the release is
> about -- so let's get hte pom template files out of hte src tree, let's
> get the maven related tasks out of the build.xml file and treat publishing
> to maven as a seperate process that happens *after* the release.  We vote
> on the release, we release it, and then the folks that care about maven
> can publish the jars after the fact.

If what you mean is that maven artifact production tools should not be in the Lucene/Solr source tree: -1000.  Requiring these to be hosted elsewhere would very likely kill Lucene/Solr maven compatibility, and I don't think that's your intention.

Maven artifact production under Lucene's/Solr's Ant build suffers from the same problem as releasing generally: lack of automation.  Too much manual intervention is required to keep the .pom templates in sync, etc.  LUCENE-2268 would afford confidence in the produced artifacts, but it would do nothing to address the current production process problems.

Steve

On Sat, Sep 18, 2010 at 7:59 AM, Steven A Rowe <[EMAIL PROTECTED]> wrote:

>
> Maven artifact production under Lucene's/Solr's Ant build suffers from the
> same problem as releasing generally: lack of automation.  Too much manual
> intervention is required to keep the .pom templates in sync, etc.
>  LUCENE-2268 would afford confidence in the produced artifacts, but it would
> do nothing to address the current production process problems.
>

How do other projects handle this? especially ones that "leave it to
interested users to do the maven releasing", since that sounds like what we
want to do.

as far as LUCENE-2268 not helping much, you might be right, I don't
understand maven at all... but the fact we currently "support" this but have
absolutely no way to test it is wrong.
the other day i was actually looking at ant's xml validation task:
http://ant.apache.org/manual/Tasks/xmlvalidate.html thinking the build could
at least make sure someone doesnt forget a </bar> in these pom.xml files...
even this would be an improvement.

--
Robert Muir
[EMAIL PROTECTED]

On Fri, Sep 17, 2010 at 9:45 PM, Chris Hostetter
<hossman_lucene@fucit.org>wrote:

>
> My unscientific, off the cuff, sociological impression is that once we
> moved forward with
> the "multi-branch" development plan and created the 3x branch, a lot of
> people who use to be the big proponents of regular releases got really
> about the freedom involved in working on the trunk, and lost their
> motivation to push for releases - because a big part of that motivation
> came from the backcompat concerns and the need to churn out releasees
> with deprecations so that future versions could move forward ith more
> interesting APIs.  "trunk" turned into the new hot sexiness.
>

Maybe, but deprecations shouldn't be the primary motivation for releasing
anyway.
> I think it's going to take some work just on build/process before we can
> get our first "merged" release from 3x.  Assuming we improve some
> automation while we're at it, then i think it's feasible that we could
> start doing releases off of it every couple of months.  it would remain
> to be seen wether we sould *need* to release that often -- it will
> depend on wether anything new gets committed there -- but it would
> certianly be nice to be able to.
>

I can't argue with this, I think the first release will be painful, but
thats exactly what I am proposing here, gearing ourselves up for more rapid
stable releases.

And as far as whether we *need* to release often, i'd like to start looking
at whether we *should*.
For a realistic example, Solr 3.x got autosuggest functionality. This isn't
really a disruptive thing, its not going to introduce a lot of deprecations,
etc.
do we *need* to release because of that? no. *should* we release? I think
yes.

To a lot of users, this is a major search engine feature, and they might
consider it more "major" than something we consider "major" like flexible
indexing.
With stable releases, the user can put this feature into production more
easily because all the "sexy" stuff is in trunk, not causing them
upgrade-hell.
This results in a faster feedback process for us, too.

I think we should try to avoid massive yearly releases with a ton of changes
at once.

--
Robert Muir
[EMAIL PROTECTED]

I don't get what is being proposed...  are you guys really suggesting
we drop the ant generate-maven-artifacts target?  and let people who
use maven sort it out for themselves?  Making sure the generated poms
are valid is another question
On Fri, Sep 17, 2010 at 8:44 PM, Chris Male <[EMAIL PROTECTED]> wrote:
>
> On Sat, Sep 18, 2010 at 2:45 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>
>> On 9/17/10 9:45 PM, Chris Hostetter wrote:
>>  When we vote on a release, tha's what we
>> > are voting on: source.  We may also distribute precompiled binary jars
>> > via the download mirrors, or via maven, but that's not what the release
>> > is
>> > about -- so let's get hte pom template files out of hte src tree, let's
>> > get the maven related tasks out of the build.xml file and treat
>> > publishing
>> > to maven as a seperate process that happens *after* the release.  We
>> > vote
>> > on the release, we release it, and then the folks that care about maven
>> > can publish the jars after the fact.
>> >
>
> Given how messy and problem plagued the maven releases code is currently, +1
> to this idea.
>
>
> --
> Chris Male | Software Developer | JTeam BV.| www.jteam.nl
>

---------------------------------------------------------------------

I don't think we really care if the tools remain in Lucene - that will
allow some guy that cares about Maven to still make it all happen.

I think we are more saying, lets drop it from part of the official
release process. When we release, we don't worry about Maven as part of
the release process. We can leave the Maven support tools we have - but
it will be up to someone to step up and handle the Maven part outside of
the official release process. So we drop it from the release todo's and
what not (moving it to another page about how to create the maven
artifacts).

Then those that release and don't want to deal with Maven do not have
to. It won't fall on the RM to think about Maven by default. Mavens
supporters will have to step in.

- Mark

On 9/18/10 12:02 PM, Ryan McKinley wrote:
> I don't get what is being proposed...  are you guys really suggesting
> we drop the ant generate-maven-artifacts target?  and let people who
> use maven sort it out for themselves?  Making sure the generated poms
> are valid is another question
>
>

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 1:13 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
> I think we are more saying, lets drop it from part of the official
> release process.

To clarify - it never really was part of the official release process,
(just as the referenced wiki pages and much on them never were either)
- it was sort of on a release-by-release basis.  We've never
officially set release policy to be anything other than what is
strictly required by the ASF.  But further clarification on what we
*want* to do isn't a bad thing...

-Yonik
http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8

---------------------------------------------------------------------

On 9/18/10 1:31 PM, Yonik Seeley wrote:
> On Sat, Sep 18, 2010 at 1:13 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>> I think we are more saying, lets drop it from part of the official
>> release process.
>
> To clarify - it never really was part of the official release process,
> (just as the referenced wiki pages and much on them never were either)
> - it was sort of on a release-by-release basis.  We've never
> officially set release policy to be anything other than what is
> strictly required by the ASF.  But further clarification on what we
> *want* to do isn't a bad thing...
>
> -Yonik
> http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8
>
Well, you have always claimed that as de jure, I think defacto is that
it's part of the release. And the defacto is to follow the 'release to
do' best as makes sense (I'm not sure the Solr release to do wiki always
makes much sense). I've been waiting for the day that you release Lucene
and drop all consideration for Maven as you have said you would likely
do - but I think most of us feel it's pretty much on the list and this
general agreement will free us of our conscious. I was ready to follow
your coat tails to freedom, but this way lets me off easier I think.

- Mark

---------------------------------------------------------------------

Just to be a bit more clear: I'm all for ditching the maven stuff from
the src tree as well, and having everything happen post release. But I
can see how that could be a pain in the ass if we did that quickly. I'm
much more concerned that Maven get out of the release process than the
maven poms/build.xml support being ripped out soon.

- Mark

On 9/18/10 1:13 PM, Mark Miller wrote:
> I don't think we really care if the tools remain in Lucene - that will
> allow some guy that cares about Maven to still make it all happen.
>
> I think we are more saying, lets drop it from part of the official
> release process. When we release, we don't worry about Maven as part of
> the release process. We can leave the Maven support tools we have - but
> it will be up to someone to step up and handle the Maven part outside of
> the official release process. So we drop it from the release todo's and
> what not (moving it to another page about how to create the maven
> artifacts).
>
> Then those that release and don't want to deal with Maven do not have
> to. It won't fall on the RM to think about Maven by default. Mavens
> supporters will have to step in.
>
> - Mark

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 2:49 PM, Mark Miller <[EMAIL PROTECTED]> wrote:

>
> Well, you have always claimed that as de jure, I think defacto is that
> it's part of the release. And the defacto is to follow the 'release to
> do' best as makes sense (I'm not sure the Solr release to do wiki always
> makes much sense). I've been waiting for the day that you release Lucene
> and drop all consideration for Maven as you have said you would likely
> do - but I think most of us feel it's pretty much on the list and this
> general agreement will free us of our conscious. I was ready to follow
> your coat tails to freedom, but this way lets me off easier I think.
>
>
Just my opinion: (personally i do not use maven, nor understand it).

If maven support is beneficial to bringing more devs to lucene, we should
consider what we can do.
But at the same time, perhaps Makefiles would bring more devs, too.

My problem with releasing with maven is that i could not honestly even +1 my
own release artifacts, because i don't know what the hell is going on with
the maven artifacts.

There has to be a way to let the "maven experts" take care of this stuff
somehow, if its really going to be beneficial.

--
Robert Muir
[EMAIL PROTECTED]

> I had not been faced with the scrofulous horror of Maven

Nice... Is this phrase copyrighted or can I use it extenuously without
paying royalties (eg, open sourced).  :)

In other words, I could not agree more.

On Sat, Sep 18, 2010 at 1:19 AM, Lance Norskog <[EMAIL PROTECTED]> wrote:
> +1 on the ant-only policy. I've recently been futzing with Mahout and I had
> not been faced with the scrofulous horror of Maven. Please keep it out of
> the main source tree of solr. You can do whatever you want with the internal
> Apache build process.
>
> Chris Hostetter wrote:
>>
>> My unscientific, off the cuff, sociological impression is that once we
>> moved forward with
>> the "multi-branch" development plan and created the 3x branch, a lot of
>> people who use to be the big proponents of regular releases got really
>> about the freedom involved in working on the trunk, and lost their
>> motivation to push for releases - because a big part of that motivation
>> came from the backcompat concerns and the need to churn out releasees
>> with deprecations so that future versions could move forward ith more
>> interesting APIs.  "trunk" turned into the new hot sexiness.
>>
>> but like i said: that's just my unscientific impression.
>>
>> : I think now that we have a "trunk" for unstable development, and a
>> : "3x" stable branch, that we should think about cutting releases from
>> : this branch much more often, for example every month or two.
>>
>> I think that might be overly ambitious, particularly because we've never
>> really talked about how the release process for the 3x branch *should*
>> work (given the lucene/solr development merged) let alone start on those
>> changes to make it easy (that process doc that scares the crap out of you
>> is just for Lucene-Java, there's an equally scaray one for Solr)
>>
>>
>> I think it's going to take some work just on build/process before we can
>> get our first "merged" release from 3x.  Assuming we improve some
>> automation while we're at it, then i think it's feasible that we could
>> start doing releases off of it every couple of months.  it would remain
>> to be seen wether we sould *need* to release that often -- it will
>> depend on wether anything new gets committed there -- but it would
>> certianly be nice to be able to.
>>
>> : Finally, as far as getting someone to do the work, I can certainly
>> : volunteer to help in the following ways:
>> : * being RM if you are ok with a non-maven release (until LUCENE-2268
>> : is fixed, i am uncomfortable with maven)
>>
>> maven is such a contentious issue -- people either don't give a shit
>> about it, or think it's the end of the world if the jars aren't there.
>>
>> In the past i've argued that enough users care about maven we should
>> really try to make sure we play nicely, but the more i think about it the
>> less i think it should be part of the release process.
>>
>> the ASF releases source code.  When we vote on a release, tha's what we
>> are voting on: source.  We may also distribute precompiled binary jars
>> via the download mirrors, or via maven, but that's not what the release is
>> about -- so let's get hte pom template files out of hte src tree, let's
>> get the maven related tasks out of the build.xml file and treat publishing
>> to maven as a seperate process that happens *after* the release.  We vote
>> on the release, we release it, and then the folks that care about maven
>> can publish the jars after the fact.
>>
>>
>>
>>
>>
>> -Hoss
>>
>> --
>> http://lucenerevolution.org/  ...  October 7-8, Boston
>> http://bit.ly/stump-hoss      ...  Stump The Chump!
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org

On Sat, Sep 18, 2010 at 11:59 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>
> On Sat, Sep 18, 2010 at 2:49 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>
>> Well, you have always claimed that as de jure, I think defacto is that
>> it's part of the release. And the defacto is to follow the 'release to
>> do' best as makes sense (I'm not sure the Solr release to do wiki always
>> makes much sense). I've been waiting for the day that you release Lucene
>> and drop all consideration for Maven as you have said you would likely
>> do - but I think most of us feel it's pretty much on the list and this
>> general agreement will free us of our conscious. I was ready to follow
>> your coat tails to freedom, but this way lets me off easier I think.
>>
>
> Just my opinion: (personally i do not use maven, nor understand it).
> If maven support is beneficial to bringing more devs to lucene, we should
> consider what we can do.
> But at the same time, perhaps Makefiles would bring more devs, too.
> My problem with releasing with maven is that i could not honestly even +1 my
> own release artifacts, because i don't know what the hell is going on with
> the maven artifacts.
> There has to be a way to let the "maven experts" take care of this stuff
> somehow, if its really going to be beneficial.

As a maven user (not an expert by any means), the maven stuff in
3.x/trunk is actually pretty good.  Running:
  ant generate-maven-artifacts -Dmaven.dist.dir=maven -Dversion=4.0.rxxx
makes a folder (maven) with everything it needs.  This is *very* easy
for maven apps to test against.

What are the deploy steps that we are talking about dropping/changing?
- - - -

As an aside, I still think it is worth changing our dev builds from
"-dev.jar" to "-SNAPSHOT.jar" so that the daily builds are
automatically valid SNAPSHOT builds that are easy for maven/ivy users
to work with.  (LUCENE-2493)  As is, maven users have to checkout and
build with a special version to test/use a dev build -- since this is
more work then many people want to deal with, we find problems with
the maven pom files *after* the official release.
ryan

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 3:36 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

> As a maven user (not an expert by any means), the maven stuff in
> 3.x/trunk is actually pretty good.  Running:
>  ant generate-maven-artifacts -Dmaven.dist.dir=maven -Dversion=4.0.rxxx
> makes a folder (maven) with everything it needs.  This is *very* easy
> for maven apps to test against.
>

which maven apps test these though? I have no way to know if things are
correct.
see LUCENE-2268
>
> What are the deploy steps that we are talking about dropping/changing?
>
>
anything having to do with maven at all.
i dont understand why we (apache) need to be involved? why cant someone who
really knows maven do this, and do it right?

since i have been around, it seems the "maven" is wrong in nearly every
release[1] including even bugfix releases.
if i am going to be the one making artifacts, i want them to be right.

[1]:
Lucene/Solr 3.x, 4.0: SOLR-2041, SOLR-2055
Solr 1.4.1: SOLR-1977
Solr 1.4: SOLR-981
Lucene 2.9.1, 3.0: LUCENE-2107
Lucene 2.9.0:  LUCENE-1927
Lucene 2.4: LUCENE-1525

--
Robert Muir
[EMAIL PROTECTED]

Last time I checked, maven artifacts have to be uploaded by a representative of the project.  I.e. a committer.  Farming this out seems like a no-go (for good reasons). - Steve

From: Robert Muir [mailto:[EMAIL PROTECTED]]
Sent: Saturday, September 18, 2010 3:51 PM
To: dev@lucene.apache.org
Subject: Re: discussion about release frequency.
On Sat, Sep 18, 2010 at 3:36 PM, Ryan McKinley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
As a maven user (not an expert by any means), the maven stuff in
3.x/trunk is actually pretty good.  Running:
 ant generate-maven-artifacts -Dmaven.dist.dir=maven -Dversion=4.0.rxxx
makes a folder (maven) with everything it needs.  This is *very* easy
for maven apps to test against.

which maven apps test these though? I have no way to know if things are correct.
see LUCENE-2268
What are the deploy steps that we are talking about dropping/changing?

anything having to do with maven at all.
i dont understand why we (apache) need to be involved? why cant someone who really knows maven do this, and do it right?

since i have been around, it seems the "maven" is wrong in nearly every release[1] including even bugfix releases.
if i am going to be the one making artifacts, i want them to be right.

[1]:
Lucene/Solr 3.x, 4.0: SOLR-2041, SOLR-2055
Solr 1.4.1: SOLR-1977
Solr 1.4: SOLR-981
Lucene 2.9.1, 3.0: LUCENE-2107
Lucene 2.9.0:  LUCENE-1927
Lucene 2.4: LUCENE-1525

--
Robert Muir
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

On Sat, 18 Sep 2010, Robert Muir wrote:

> Just my opinion: (personally i do not use maven, nor understand it).
>
> If maven support is beneficial to bringing more devs to lucene, we should
> consider what we can do.
> But at the same time, perhaps Makefiles would bring more devs, too.
>
> My problem with releasing with maven is that i could not honestly even +1 my
> own release artifacts, because i don't know what the hell is going on with
> the maven artifacts.
>
> There has to be a way to let the "maven experts" take care of this stuff
> somehow, if its really going to be beneficial.

My - possibly flawed - understanding is that Maven brings in more users, not
more devs. A dev needs to know how to build the software package(s) s/he's
working on.

With PyLucene, for example, I've consistently refused to release binary
packages as there are just way too many possible combinations of
Python/Java/gcc/OS versions to do a thorough job.

Building PyLucene isn't exactly trivial on some platforms such as Windows,
for example. I'm sure this has cost the project some users as the barrier to
entry is a bit higher than it could be.

On the other hand, the users that remain at least have some understanding
about how to build and debug a C++ python extension and that's a very good
thing because a user is more likely to become a dev if they understand how
to rebuild the software package they want to contribute a bug fix to, for
example.

That being said, it is rather easy to build Lucene binaries using ant, so
Maven at first blush doesn't seem to bring much. Where it becomes more
useful is that in a situation where many dependencies are required before
a particular project of interest that depends on Lucene can be built, a
working Maven install takes care of the chore or bringing in all these
dependencies.

For example, I recently did an experimental build of Tika for Python using
JCC. Tika's Maven tricks pulled in a very large number - dozens - of
dependencies that had I had to download and build them all by myself
manually, I would maybe not have spent that time with Tika to begin with.

Of course, Lucene is quite the opposite. Besides a JDK (and possibly ICU),
it depends on nothing else. The need for Maven support is less obvious there
too.

The question about Maven support is then about how hard do we want to work
to get more users, balancing that with how easy do we want to make it for
them to become devs, contributors.

I agree with the consensus that seems to be forming. Lucene has plenty of
users already, many of which are hopefully Maven savvy. They should help
maintain and improve Maven support for Lucene.

+1 to keep the Maven code in the source
+1 to not release Maven artifacts

Andi..

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 3:57 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:

> Last time I checked, maven artifacts have to be uploaded by a
> representative of the project.  I.e. a committer.  Farming this out seems
> like a no-go (for good reasons). - Steve
>
>
>

Well, (again completely ignorant of maven) it seems this itself is a
separate concern?

I'm not sure how complicated actually upload artifacts is, although just
browsing other projects, getting maven releases out isn't exactly a walk in
the park.

but theres two steps right?
1. generating the actual artifacts and verifying they are correct
2. uploading the artifacts (e.g. commit or whatever it takes to do that)

Why exactly does it require a committer to upload maven artifacts (they must
be to an apache.org location?)

--
Robert Muir
[EMAIL PROTECTED]

On Sat, Sep 18, 2010 at 3:57 PM, Andi Vajda <[EMAIL PROTECTED]> wrote:

>
> For example, I recently did an experimental build of Tika for Python using
> JCC. Tika's Maven tricks pulled in a very large number - dozens - of
> dependencies that had I had to download and build them all by myself
> manually, I would maybe not have spent that time with Tika to begin with.
>
> Of course, Lucene is quite the opposite. Besides a JDK (and possibly ICU),
> it depends on nothing else. The need for Maven support is less obvious there
> too.
>
>
Well, this is of interest to us too I think. Because (though its not flushed
out) we are talking about possibly Lucene + Solr releases at the same time.

Solr has Tika integration, and even apart from that brings in other
dependencies too, much more than any lucene modules do.

--
Robert Muir
[EMAIL PROTECTED]

Hi Robert,

I can help a little here. Check out this guide:

http://maven.apache.org/guides/mini/guide-central-repository-upload.html

The long and the short of it is that there are several canonical Maven repos that are sync'ed to Ibiblio and Maven central. Apache has one (through repository.apache.org), and other organizations have em' too. In Tika, we simply inherit from the top level Apache POM and the Maven release plugin takes care of the rest. I'm not sure how to do this with Ant, but am sure it can be done (and it might be aided by Ivy).

If you are just doing it when you release, which doesn't happen that often, then you could consider just creating a Ticket and uploading the release jars through the Sonatype JIRA process, located here:

http://nexus.sonatype.org/oss-repository-hosting.html

Guide is here:

https://docs.sonatype.org/display/repository/sonatype+oss+maven+repository+usage+guide

Hope that helps!

Cheers,
Chris

On 9/18/10 1:08 PM, "Robert Muir" <[EMAIL PROTECTED]> wrote:
On Sat, Sep 18, 2010 at 3:57 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
Last time I checked, maven artifacts have to be uploaded by a representative of the project.  I.e. a committer.  Farming this out seems like a no-go (for good reasons). - Steve
Well, (again completely ignorant of maven) it seems this itself is a separate concern?

I'm not sure how complicated actually upload artifacts is, although just browsing other projects, getting maven releases out isn't exactly a walk in the park.

but theres two steps right?
1. generating the actual artifacts and verifying they are correct
2. uploading the artifacts (e.g. commit or whatever it takes to do that)

Why exactly does it require a committer to upload maven artifacts (they must be to an apache.org <http://apache.org>  location?)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

> the maven stuff in 3.x/trunk is actually pretty good

I've heard that about every release of Maven, and any time I've tried
to use it, it doesn't quite work as expected, and given what it does
should be fairly trivial, the fact that there bugs/issues, and it's
been released to me has meant I don't want to use it.  It's like a
toaster that also plays videos, I just want a toaster, thanks.

On Sat, Sep 18, 2010 at 3:36 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:
> On Sat, Sep 18, 2010 at 11:59 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>
>>
>> On Sat, Sep 18, 2010 at 2:49 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>>
>>> Well, you have always claimed that as de jure, I think defacto is that
>>> it's part of the release. And the defacto is to follow the 'release to
>>> do' best as makes sense (I'm not sure the Solr release to do wiki always
>>> makes much sense). I've been waiting for the day that you release Lucene
>>> and drop all consideration for Maven as you have said you would likely
>>> do - but I think most of us feel it's pretty much on the list and this
>>> general agreement will free us of our conscious. I was ready to follow
>>> your coat tails to freedom, but this way lets me off easier I think.
>>>
>>
>> Just my opinion: (personally i do not use maven, nor understand it).
>> If maven support is beneficial to bringing more devs to lucene, we should
>> consider what we can do.
>> But at the same time, perhaps Makefiles would bring more devs, too.
>> My problem with releasing with maven is that i could not honestly even +1 my
>> own release artifacts, because i don't know what the hell is going on with
>> the maven artifacts.
>> There has to be a way to let the "maven experts" take care of this stuff
>> somehow, if its really going to be beneficial.
>
> As a maven user (not an expert by any means), the maven stuff in
> 3.x/trunk is actually pretty good.  Running:
>  ant generate-maven-artifacts -Dmaven.dist.dir=maven -Dversion=4.0.rxxx
> makes a folder (maven) with everything it needs.  This is *very* easy
> for maven apps to test against.
>
> What are the deploy steps that we are talking about dropping/changing?
>
>
> - - - -
>
> As an aside, I still think it is worth changing our dev builds from
> "-dev.jar" to "-SNAPSHOT.jar" so that the daily builds are
> automatically valid SNAPSHOT builds that are easy for maven/ivy users
> to work with.  (LUCENE-2493)  As is, maven users have to checkout and
> build with a special version to test/use a dev build -- since this is
> more work then many people want to deal with, we find problems with
> the maven pom files *after* the official release.
>
>
> ryan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 4:18 PM, Mattmann, Chris A (388J) <
[EMAIL PROTECTED]> wrote:

>  Hi Robert,
>
> I can help a little here. Check out this guide:
>
> http://maven.apache.org/guides/mini/guide-central-repository-upload.html
>
> The long and the short of it is that there are several canonical Maven
> repos that are sync’ed to Ibiblio and Maven central. Apache has one (through
> repository.apache.org), and other organizations have em’ too. In Tika, we
> simply inherit from the top level Apache POM and the Maven release plugin
> takes care of the rest. I’m not sure how to do this with Ant, but am sure it
> can be done (and it might be aided by Ivy).
>
> If you are just doing it when you release, which doesn’t happen that often,
> then you could consider just creating a Ticket and uploading the release
> jars through the Sonatype JIRA process, located here:
>
> http://nexus.sonatype.org/oss-repository-hosting.html
>
> Guide is here:
>
>
> https://docs.sonatype.org/display/repository/sonatype+oss+maven+repository+usage+guide
>
> Hope that helps!
>
>
Thank you Chris. Actually this does help a lot, more than you think.
I tried to digest these guides and it only reinforced my opinion that I
shouldn't be involved with maven at all.

I don't mean to criticise the maven project by any means, but as a software
developer there are times when you look at something and you say 'this is
way too complicated, it just simply isn't compatible with my brain'. I know
some people prefer ant, some people prefer maven, some people write their
own python scripts and still use emacs, and I'm not trying to say I have
preference over any given system.

I cannot in good conscience sign with my key, nor vote over any maven
artifacts. I noticed these guides only mentioned how to upload (which itself
seems extremely complex). But nowhere do i see 'how do you test that your
artifacts are correct'. And thats really the main problem I have with our
maven support.

I don't care how easy it is to actually upload things (via plugin X or a
JIRA issue). I want to know things are correct before uploading them
anywhere.
So, I think my original statement still stands. I'm happy to volunteer as RM
as long as we realize this means no maven involved. If someone else wants to
take on the 'maven' role, thats fantastic!

If its not acceptable to do a release without being a maven expert, I am
fine with that too. I can contribute in other ways such as improving
automation.

--
Robert Muir
[EMAIL PROTECTED]

Note, I am *not* suggesting that maven is the best tool etc etc...
that is not a discussion worth having here.

I am saying that the maven artifacts (pom files etc) generate in
3.x/trunk work well if you are using maven/ivy.  I don't see why we
want to drop that from the release?
On Sat, Sep 18, 2010 at 1:45 PM, Jason Rutherglen
<[EMAIL PROTECTED]> wrote:
>> the maven stuff in 3.x/trunk is actually pretty good
>
> I've heard that about every release of Maven, and any time I've tried
> to use it, it doesn't quite work as expected, and given what it does
> should be fairly trivial, the fact that there bugs/issues, and it's
> been released to me has meant I don't want to use it.  It's like a
> toaster that also plays videos, I just want a toaster, thanks.
>
> On Sat, Sep 18, 2010 at 3:36 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:
>> On Sat, Sep 18, 2010 at 11:59 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>> On Sat, Sep 18, 2010 at 2:49 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>>>
>>>> Well, you have always claimed that as de jure, I think defacto is that
>>>> it's part of the release. And the defacto is to follow the 'release to
>>>> do' best as makes sense (I'm not sure the Solr release to do wiki always
>>>> makes much sense). I've been waiting for the day that you release Lucene
>>>> and drop all consideration for Maven as you have said you would likely
>>>> do - but I think most of us feel it's pretty much on the list and this
>>>> general agreement will free us of our conscious. I was ready to follow
>>>> your coat tails to freedom, but this way lets me off easier I think.
>>>>
>>>
>>> Just my opinion: (personally i do not use maven, nor understand it).
>>> If maven support is beneficial to bringing more devs to lucene, we should
>>> consider what we can do.
>>> But at the same time, perhaps Makefiles would bring more devs, too.
>>> My problem with releasing with maven is that i could not honestly even +1 my
>>> own release artifacts, because i don't know what the hell is going on with
>>> the maven artifacts.
>>> There has to be a way to let the "maven experts" take care of this stuff
>>> somehow, if its really going to be beneficial.
>>
>> As a maven user (not an expert by any means), the maven stuff in
>> 3.x/trunk is actually pretty good.  Running:
>>  ant generate-maven-artifacts -Dmaven.dist.dir=maven -Dversion=4.0.rxxx
>> makes a folder (maven) with everything it needs.  This is *very* easy
>> for maven apps to test against.
>>
>> What are the deploy steps that we are talking about dropping/changing?
>>
>>
>> - - - -
>>
>> As an aside, I still think it is worth changing our dev builds from
>> "-dev.jar" to "-SNAPSHOT.jar" so that the daily builds are
>> automatically valid SNAPSHOT builds that are easy for maven/ivy users
>> to work with.  (LUCENE-2493)  As is, maven users have to checkout and
>> build with a special version to test/use a dev build -- since this is
>> more work then many people want to deal with, we find problems with
>> the maven pom files *after* the official release.
>>
>>
>> ryan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------

On 9/18/2010 at 3:37 PM, Ryan McKinley wrote:
> As an aside, I still think it is worth changing our dev builds from
> "-dev.jar" to "-SNAPSHOT.jar" so that the daily builds are
> automatically valid SNAPSHOT builds that are easy for maven/ivy users
> to work with.  (LUCENE-2493)  As is, maven users have to checkout and
> build with a special version to test/use a dev build -- since this is
> more work then many people want to deal with, we find problems with
> the maven pom files *after* the official release.

Shortly after LUCENE-2493 was created, I looked into the possibility of having Maven treat *-dev.jar files as snapshots.  As far as I could tell, this is not a configurable item: snapshots are named *-SNAPSHOT.jar - it's hard-coded in lots of places in the Maven code.  I like Maven, but inflexibility like this makes it hard to get pre-existing projects to play nice with it...

Steve

> I cannot in good conscience sign with my key, nor vote over any maven
> artifacts. I noticed these guides only mentioned how to upload (which itself
> seems extremely complex). But nowhere do i see 'how do you test that your
> artifacts are correct'. And thats really the main problem I have with our
> maven support.

I understand what you are worried about... and think we can avoid it.
How about:

1. Keep the "generate-maven-artifacts" in the release.  This just
copies the "official" jar files to a special directory structure (same
keys etc)
2. The RM just makes a zip or copies that folder somewhere.  (perhaps
to their own ~people folder)
3. Someone else get that into the repo

I think it is important to *try* to have the "official" jar files in
the maven repositories -- we have scripts that get that mostly right.
In the past when we have errors it is just the pom files that get
edited (these are the files that say what the other files/dependencies
are).  If we remove the "generate-maven-artifacts" task, then without
writing lots of new scripts (uggg) the jar files in the maven repos
will be a different build.

Does that sound OK?

ryan

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 5:09 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

> > I cannot in good conscience sign with my key, nor vote over any maven
> > artifacts. I noticed these guides only mentioned how to upload (which
> itself
> > seems extremely complex). But nowhere do i see 'how do you test that your
> > artifacts are correct'. And thats really the main problem I have with our
> > maven support.
>
> I understand what you are worried about... and think we can avoid it.
> How about:
>
> 1. Keep the "generate-maven-artifacts" in the release.  This just
> copies the "official" jar files to a special directory structure (same
> keys etc)
> 2. The RM just makes a zip or copies that folder somewhere.  (perhaps
> to their own ~people folder)
> 3. Someone else get that into the repo
>
> I think it is important to *try* to have the "official" jar files in
> the maven repositories -- we have scripts that get that mostly right.
> In the past when we have errors it is just the pom files that get
> edited (these are the files that say what the other files/dependencies
> are).  If we remove the "generate-maven-artifacts" task, then without
> writing lots of new scripts (uggg) the jar files in the maven repos
> will be a different build.
>
> Does that sound OK?
>
>
it sounds like it only solves 'part 2: uploading'.
i want to solve 'part 1: verifying artifacts are correct before
signing/uploading at all'.

to me, maven is nothing more than a contrib with no unit tests.

it needs tests so we know it is working.
--
Robert Muir
[EMAIL PROTECTED]

I can see leaving the build support around for those committers that
like and want to support Maven - I'm sure it wouldn't be easy to rip it
out short term. But I still think it should be completely outside of the
RM's cares (as Yonik claims it is now - I'd like to see a bit of
consensus on it though). The RM does the Lucene release - someone steps
up and does the Maven stuff or they don't.

P.S. Every time I have had to do the Maven stuff, there has been issues
with uploading the files (permissions are always a little off), or other
little things that some dude's maven repo wathcing script pings me about
- even when i follow the release todo to a T. It's annoying and a hassle
to handle these situations - for Solr 1.4.1 I took forever too fix the
Maven repo - first I said: someone who cares about maven please step up
- no one did. Eventually I went and got it done because I felt
responsibility as the RM. Personally, I think all those little issues
should be on some that cares about supporting Maven - it's separate from
releasing the Lucene/Solr bits.
- Mark

---------------------------------------------------------------------

>
> it sounds like it only solves 'part 2: uploading'.
> i want to solve 'part 1: verifying artifacts are correct before
> signing/uploading at all'.

The "artifacts" are the identical .jar files put into a special
directory structure.  You have already verified they are correct --
this is what I don't want to loose.

I agree the RM does not need to be responsible for saying the pom
files are correct -- this can be checked by whoever actually deploys
them.
> to me, maven is nothing more than a contrib with no unit tests.
> it needs tests so we know it is working.
>

The only way to 'test' this would be to have something that uses maven
try to load them -- I would happily add that, but have avoided that
since it opens the toxic maven hate hard.   LUCENE-2493 would make it
pretty easy for others to know if stuff is valid on dev builds.

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 2:22 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
> I can see leaving the build support around for those committers that
> like and want to support Maven - I'm sure it wouldn't be easy to rip it
> out short term. But I still think it should be completely outside of the
> RM's cares (as Yonik claims it is now - I'd like to see a bit of
> consensus on it though). The RM does the Lucene release - someone steps
> up and does the Maven stuff or they don't.
>

I agree with this -- let us make them two tasks.  However can the RM
run the "generate-maven-artifacts" task and put the results somewhere
for someone else to deploy?

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 5:29 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

> On Sat, Sep 18, 2010 at 2:22 PM, Mark Miller <[EMAIL PROTECTED]>
> wrote:
> > I can see leaving the build support around for those committers that
> > like and want to support Maven - I'm sure it wouldn't be easy to rip it
> > out short term. But I still think it should be completely outside of the
> > RM's cares (as Yonik claims it is now - I'd like to see a bit of
> > consensus on it though). The RM does the Lucene release - someone steps
> > up and does the Maven stuff or they don't.
> >
>
> I agree with this -- let us make them two tasks.  However can the RM
> run the "generate-maven-artifacts" task and put the results somewhere
> for someone else to deploy?
>
>
I won't speak for anyone else, but the .pom files are signed by the RM's
key.
I checked to be sure, for example:
http://people.apache.org/~markrmiller/staging-area/rc1/maven/org/apache/solr/solr-core/

as stated earlier, i dont want to sign anything that is incorrect.
I know the true purpose of signing, but i'm not willing to put my release
key on these things, unless there is automated testing.

is maven incompatible with unit testing?

--
Robert Muir
[EMAIL PROTECTED]

On Sat, Sep 18, 2010 at 2:32 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>
> On Sat, Sep 18, 2010 at 5:29 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:
>>
>> On Sat, Sep 18, 2010 at 2:22 PM, Mark Miller <[EMAIL PROTECTED]>
>> wrote:
>> > I can see leaving the build support around for those committers that
>> > like and want to support Maven - I'm sure it wouldn't be easy to rip it
>> > out short term. But I still think it should be completely outside of the
>> > RM's cares (as Yonik claims it is now - I'd like to see a bit of
>> > consensus on it though). The RM does the Lucene release - someone steps
>> > up and does the Maven stuff or they don't.
>> >
>>
>> I agree with this -- let us make them two tasks.  However can the RM
>> run the "generate-maven-artifacts" task and put the results somewhere
>> for someone else to deploy?
>>
>
> I won't speak for anyone else, but the .pom files are signed by the RM's
> key.
> I checked to be sure, for example:
> http://people.apache.org/~markrmiller/staging-area/rc1/maven/org/apache/solr/solr-core/
> as stated earlier, i dont want to sign anything that is incorrect.
> I know the true purpose of signing, but i'm not willing to put my release
> key on these things, unless there is automated testing.
> is maven incompatible with unit testing?

To automatically test if the maven files work, we would need to run
maven...  that seems like a non-starter for many people.

We could easily add a maven test project -- but for it to be 'real' it
needs to use maven.  I'll happily volunteer to write it.

If you are worried about the signing part...  can we keep the
maven.dist folder out of the signing task?

ryan

---------------------------------------------------------------------

On Sat, Sep 18, 2010 at 6:12 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

>
> To automatically test if the maven files work, we would need to run
> maven...  that seems like a non-starter for many people.
>
>
why is this a problem? I don't mind having maven installed and setup if i
can run 'ant test-maven' and do verifications on these things.
even if that involves simply forking a 'mvn' process externally.
> We could easily add a maven test project -- but for it to be 'real' it
> needs to use maven.  I'll happily volunteer to write it.
>
>
sounds great, i thought this was the idea here:
https://issues.apache.org/jira/browse/LUCENE-2268

I mean, even during normal development, if there was an easy single command
i could run to verify i'm not breaking maven support, this would help,
especially when refactoring / moving things around / etc.
I wouldn't mind running this at all, to try to keep maven functional. I also
wouldn't mind debugging or even trying to figure out bits of maven until i
make it pass.

my problem is just that without any automated verification like this: maven
is just a big scary piece of code with no tests, and I cannot touch it!

--
Robert Muir
[EMAIL PROTECTED]

On Sat, Sep 18, 2010 at 3:22 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>
> On Sat, Sep 18, 2010 at 6:12 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:
>>
>> To automatically test if the maven files work, we would need to run
>> maven...  that seems like a non-starter for many people.
>>
>
> why is this a problem? I don't mind having maven installed and setup if i
> can run 'ant test-maven' and do verifications on these things.
> even if that involves simply forking a 'mvn' process externally.
>

You sound so reasonable.  There is lots of evidence that when people
hear the word "maven" they freak out and say "no.  no. no. nooooo!!!"
>>
>> We could easily add a maven test project -- but for it to be 'real' it
>> needs to use maven.  I'll happily volunteer to write it.
>>
>
> sounds great, i thought this was the idea
> here: https://issues.apache.org/jira/browse/LUCENE-2268
> I mean, even during normal development, if there was an easy single command
> i could run to verify i'm not breaking maven support, this would help,
> especially when refactoring / moving things around / etc.
> I wouldn't mind running this at all, to try to keep maven functional. I also
> wouldn't mind debugging or even trying to figure out bits of maven until i
> make it pass.
> my problem is just that without any automated verification like this: maven
> is just a big scary piece of code with no tests, and I cannot touch it!

I'll take a crack at LUCENE-2268 so that I am not that guy saying "you
need to do XXXX" when clearly you don't (really) care if XXXX happens
or not :)

ps.  sorry the 'release discussion' got hijaked by another maven discussion!

ryan

---------------------------------------------------------------------

It's good to see what appears to be a maven-positive resolution to this thread.  I've gotten to this thread late, but FWIW, I've agreed with basically everything Ryan has said, as I am a maven user too.

What would be *really cool* is Hudson automatically uploading the artifacts of each build to an apache maven snapshot repo (which I don't think would require any public/private key blessing).  This used to be the case in SOLR-586 (thanks Grant!) but it has since broke.  I'll file a bug report.

~ David Smiley
 Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book

-----Original Message-----
From: Ryan McKinley [mailto:[EMAIL PROTECTED]]
Sent: Saturday, September 18, 2010 7:20 PM
To: dev@lucene.apache.org
Subject: Re: discussion about release frequency.

On Sat, Sep 18, 2010 at 3:22 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>
> On Sat, Sep 18, 2010 at 6:12 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:
>>
>> To automatically test if the maven files work, we would need to run
>> maven...  that seems like a non-starter for many people.
>>
>
> why is this a problem? I don't mind having maven installed and setup if i
> can run 'ant test-maven' and do verifications on these things.
> even if that involves simply forking a 'mvn' process externally.
>

You sound so reasonable.  There is lots of evidence that when people
hear the word "maven" they freak out and say "no.  no. no. nooooo!!!"
>>
>> We could easily add a maven test project -- but for it to be 'real' it
>> needs to use maven.  I'll happily volunteer to write it.
>>
>
> sounds great, i thought this was the idea
> here: https://issues.apache.org/jira/browse/LUCENE-2268
> I mean, even during normal development, if there was an easy single command
> i could run to verify i'm not breaking maven support, this would help,
> especially when refactoring / moving things around / etc.
> I wouldn't mind running this at all, to try to keep maven functional. I also
> wouldn't mind debugging or even trying to figure out bits of maven until i
> make it pass.
> my problem is just that without any automated verification like this: maven
> is just a big scary piece of code with no tests, and I cannot touch it!

I'll take a crack at LUCENE-2268 so that I am not that guy saying "you
need to do XXXX" when clearly you don't (really) care if XXXX happens
or not :)

ps.  sorry the 'release discussion' got hijaked by another maven discussion!

ryan

---------------------------------------------------------------------
---------------------------------------------------------------------

On Sun, Sep 19, 2010 at 1:37 PM, Smiley, David W. <[EMAIL PROTECTED]> wrote:
>
>
> What would be *really cool* is Hudson automatically uploading the artifacts
> of each build to an apache maven snapshot repo (which I don't think would
> require any public/private key blessing).  This used to be the case in
> SOLR-586 (thanks Grant!) but it has since broke.  I'll file a bug report.
>

but even this should not happen until the maven feature itself has tests.
otherwise, maven users might issue bug reports constantly about these
nightly artifacts being wrong.

its better not to upload them at all at the moment, since the pom files are
untested.

--
Robert Muir
[EMAIL PROTECTED]

A little late to the party, but...

On Sep 18, 2010, at 5:09 PM, Ryan McKinley wrote:

>> I cannot in good conscience sign with my key, nor vote over any maven
>> artifacts. I noticed these guides only mentioned how to upload (which itself
>> seems extremely complex). But nowhere do i see 'how do you test that your
>> artifacts are correct'. And thats really the main problem I have with our
>> maven support.
>
> I understand what you are worried about... and think we can avoid it.
> How about:
>
> 1. Keep the "generate-maven-artifacts" in the release.  This just
> copies the "official" jar files to a special directory structure (same
> keys etc)

OK, I get that a lot of committers here don't like Maven and I don't think Lucene should switch to a Maven build and it's a pain to do complex things in, but I use it all the time for Lucene/Solr (for none complex things) and I know of a lot of people in user land who use it as well b/c it makes the common things _users_ do really easy.  

And, as much as Hoss restarted this thread by saying the PMC releases only source, it simply is not what users expect.  That's why we sign all the artifacts.  They are the RM saying I verify this and the PMC then votes on all the artifacts and it's why we push them all up for distribution.  Of course, we are only required to release source, but you show me a project that does only that at the ASF and I'll show you a project w/ very few users.

At any rate, the big problem w/ Maven and Lucene is not that generate-maven-artifacts doesn't work, it's that the POM templates aren't kept in sync.  However, I think we now have a solution for that thanks to Steve and Robert's work to make it easier to bring Lucene into IntelliJ.  In other words, that process does much of what is needed for Maven, so it should be relatively straightforward to have it automatically generate the templates, too.  In fact, it would be just as easy for that project to simply produce POM files (which are well understood and have a published spec) instead of creating the IntelliJ project files, which are not well understood and not publicly spec'd and subject to change w/ every release and simply have IntelliJ suck in the POM file since IntelliJ supports that very, very well.

Then, to automatically test Maven, we simply need to do a few things:
1. Generate the templates
2. Build the Maven artifacts and "install" them (this is a Maven concept that copies them to your local repository, usually in ~/.mvn/repository, but it can be in other places and it should be clean)
3. Generate a "test" pom that includes, as dependencies all the Lucene Maven artifacts and maybe even compiles a small source tree with it

If that last step passes, you know everything is right.  However, short of #2 and #3, as long as the POM's are being generated accurately, I think I would feel comfortable releasing them, whereas I agree, now, with Robert, that we probably shouldn't be releasing them now.

(BTW, I love the "Maven is Magic" (and really any "It's magic, therefore I don't like it") reasoning for not liking it, whereby everyone complains that b/c Maven hides a bunch of details from you (i.e. it's "magic"), therefore you don't like it.  At the same time, I'm sure said person doesn't understand every last detail of, oh, I don't know: the CPU, RAM, the Compiler, the JDK, etc. and yet they have no problem using that.  In other words, we deal with abstractions all the time.  It's fine if you don't get the abstraction or don't personally find it useful, but that doesn't make the abstraction bad.)

-Grant
---------------------------------------------------------------------

On Mon, Sep 20, 2010 at 8:23 AM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

> At any rate, the big problem w/ Maven and Lucene is not that
> generate-maven-artifacts doesn't work, it's that the POM templates aren't
> kept in sync.  However, I think we now have a solution for that thanks to
> Steve and Robert's work to make it easier to bring Lucene into IntelliJ.  In
> other words, that process does much of what is needed for Maven, so it
> should be relatively straightforward to have it automatically generate the
> templates, too.  In fact, it would be just as easy for that project to
> simply produce POM files (which are well understood and have a published
> spec) instead of creating the IntelliJ project files, which are not well
> understood and not publicly spec'd and subject to change w/ every release
> and simply have IntelliJ suck in the POM file since IntelliJ supports that
> very, very well.
>
>
So are you saying, instead of generating IntelliJ configuration, we generate
poms, and then we have a route, via maven, for users to automatically set up
their IntelliJ (and also eclipse?) IDEs?

If so this sounds great to me. Because it would be nice to make the IDE
configuration easier, not just for IntelliJ.
> Then, to automatically test Maven, we simply need to do a few things:
> 1. Generate the templates
> 2. Build the Maven artifacts and "install" them (this is a Maven concept
> that copies them to your local repository, usually in ~/.mvn/repository, but
> it can be in other places and it should be clean)
> 3. Generate a "test" pom that includes, as dependencies all the Lucene
> Maven artifacts and maybe even compiles a small source tree with it
>
>
+1. this would resolve all my concerns about maven, because we have a way to
test that it stands a chance of working *before release*.

I hope you don't think I am picking on maven here, I'm equally disturbed
about the demo application, and i think it should have a basic unit test too
that indexes stuff, fires itself up in jetty, and runs a search.

Like maven, i know some people don't necessarily like the demo, but as long
as we are going to ship it, I want tests so that we dont find its completely
nonfunctional after the release. Unlike maven, i think i stand a chance of
actually being able to write the test for this one though.
--
Robert Muir
[EMAIL PROTECTED]

On Sep 20, 2010, at 8:44 AM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 8:23 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> At any rate, the big problem w/ Maven and Lucene is not that generate-maven-artifacts doesn't work, it's that the POM templates aren't kept in sync.  However, I think we now have a solution for that thanks to Steve and Robert's work to make it easier to bring Lucene into IntelliJ.  In other words, that process does much of what is needed for Maven, so it should be relatively straightforward to have it automatically generate the templates, too.  In fact, it would be just as easy for that project to simply produce POM files (which are well understood and have a published spec) instead of creating the IntelliJ project files, which are not well understood and not publicly spec'd and subject to change w/ every release and simply have IntelliJ suck in the POM file since IntelliJ supports that very, very well.
>
>
> So are you saying, instead of generating IntelliJ configuration, we generate poms, and then we have a route, via maven, for users to automatically set up their IntelliJ (and also eclipse?) IDEs?
>
> If so this sounds great to me. Because it would be nice to make the IDE configuration easier, not just for IntelliJ.

Yes.  I know for a fact IntelliJ can read the POMs.  I use it all the time.  Go check out Mahout and point IntelliJ at it's POM.  You will be up and compiling  (in your IDE) in less than 2 minutes give or take.  I imagine Eclipse has similar support.

>  
> Then, to automatically test Maven, we simply need to do a few things:
> 1. Generate the templates
> 2. Build the Maven artifacts and "install" them (this is a Maven concept that copies them to your local repository, usually in ~/.mvn/repository, but it can be in other places and it should be clean)
> 3. Generate a "test" pom that includes, as dependencies all the Lucene Maven artifacts and maybe even compiles a small source tree with it
>
>
> +1. this would resolve all my concerns about maven, because we have a way to test that it stands a chance of working *before release*.
>
> I hope you don't think I am picking on maven here, I'm equally disturbed about the demo application, and i think it should have a basic unit test too that indexes stuff, fires itself up in jetty, and runs a search.

I totally understand it.  I'm not some Maven fanboi (especially as the person who used it to put together the Mahout release, initially).  I know it's warts, believe me, as I have lived the pain.  That being said, for _most_ users (i.e. not necessarily us committers) who are simply using Lucene/Solr within a much broader environment of dependencies, having the JARs available in the Maven repo w/ correct POM files is a very good thing that makes it so much easier for them to do their day to day work and I would hate to see that go away, especially since it is something we have supported for quite some time, albeit with varying levels of success.

>
> Like maven, i know some people don't necessarily like the demo, but as long as we are going to ship it, I want tests so that we dont find its completely nonfunctional after the release. Unlike maven, i think i stand a chance of actually being able to write the test for this one though.

I've been wanting to do those Maven tests for a while now, but simply can't find the time relative to my other priorities.  I guess if the community is saying that if someone doesn't step up, it's going to be dropped, I'll step up.  I can likely commit to it before the next release.

-Grant
---------------------------------------------------------------------

> (BTW, I love the "Maven is Magic" (and really any "It's magic, therefore I don't like it") reasoning for not liking it, whereby everyone complains that b/c Maven hides a bunch of details from you (i.e. it's "magic"), therefore you don't like it.  At the same time, I'm sure said person doesn't understand every last detail of, oh, I don't know: the CPU, RAM, the Compiler, the JDK, etc. and yet they have no problem using that.  In other words, we deal with abstractions all the time.  It's fine if you don't get the abstraction or don't personally find it useful, but that doesn't make the abstraction bad.)
>
> -Grant

Maven is not bad because it's magic - magic is frigging great - I want
my software to be magic - it's bad because every 5 line program from
some open source code/project that I have tried to build with it has
gone on an absurd downloading spree that takes forever because it's
getting many tiny files. This downloading spree never corresponds to the
size of the code base I am working with, and always manages to surprise
by the amount of time it can slurp up.

That's enough for me right there - I've heard others talk of other non
magical things that sound scary, but I won't dig any deeper into this
absurdity. Either I *really* don't like Maven, or no one knows how to
properly set it up - which makes me still not like it. When the magic is
absurd, it loses a little of its magic.

Finally, there is a difference between releasing source code, releasing
signed jars, and signed maven files, and *just* releasing signed jars.
Dropping maven doesn't get you back down to releasing source code. I
still think Maven should be a downstream issue.

- Mark

---------------------------------------------------------------------

On Sep 20, 2010, at 8:58 AM, Grant Ingersoll wrote:

>
> On Sep 20, 2010, at 8:44 AM, Robert Muir wrote:
>
>>
>>
>> On Mon, Sep 20, 2010 at 8:23 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>> At any rate, the big problem w/ Maven and Lucene is not that generate-maven-artifacts doesn't work, it's that the POM templates aren't kept in sync.  However, I think we now have a solution for that thanks to Steve and Robert's work to make it easier to bring Lucene into IntelliJ.  In other words, that process does much of what is needed for Maven, so it should be relatively straightforward to have it automatically generate the templates, too.  In fact, it would be just as easy for that project to simply produce POM files (which are well understood and have a published spec) instead of creating the IntelliJ project files, which are not well understood and not publicly spec'd and subject to change w/ every release and simply have IntelliJ suck in the POM file since IntelliJ supports that very, very well.
>>
>>
>> So are you saying, instead of generating IntelliJ configuration, we generate poms, and then we have a route, via maven, for users to automatically set up their IntelliJ (and also eclipse?) IDEs?
>>
>> If so this sounds great to me. Because it would be nice to make the IDE configuration easier, not just for IntelliJ.
>
> Yes.  I know for a fact IntelliJ can read the POMs.  I use it all the time.  Go check out Mahout and point IntelliJ at it's POM.  You will be up and compiling  (in your IDE) in less than 2 minutes give or take.  I imagine Eclipse has similar support.

I should correct myself here.  While all of the above is true, it likely still won't work for Lucene b/c the source trees aren't in line w/ Maven conventions.  Thus, we will probably still need to output IntelliJ format.  I do, however, think it isn't much of a leap to also output a POM file.

-Grant
---------------------------------------------------------------------

> I hope you don't think I am picking on maven here, I'm equally disturbed
> about the demo application, and i think it should have a basic unit test too
> that indexes stuff, fires itself up in jetty, and runs a search.

The solr sample app is tested -- i don't know anything about lucene demo stuff.

Most of the solrj tests run from the example schema via jetty and embedded.

ryan

---------------------------------------------------------------------

On Sep 20, 2010, at 9:00 AM, Mark Miller wrote:

>
>> (BTW, I love the "Maven is Magic" (and really any "It's magic, therefore I don't like it") reasoning for not liking it, whereby everyone complains that b/c Maven hides a bunch of details from you (i.e. it's "magic"), therefore you don't like it.  At the same time, I'm sure said person doesn't understand every last detail of, oh, I don't know: the CPU, RAM, the Compiler, the JDK, etc. and yet they have no problem using that.  In other words, we deal with abstractions all the time.  It's fine if you don't get the abstraction or don't personally find it useful, but that doesn't make the abstraction bad.)
>>
>> -Grant
>
> Maven is not bad because it's magic - magic is frigging great - I want
> my software to be magic - it's bad because every 5 line program from
> some open source code/project that I have tried to build with it has
> gone on an absurd downloading spree that takes forever because it's
> getting many tiny files. This downloading spree never corresponds to the
> size of the code base I am working with, and always manages to surprise
> by the amount of time it can slurp up.

Agreed, but over time, it is lessened by the fact that you already have most common files/jars and furthermore, you only have one copy of them instead of one under every source tree.  I think, over time, you actually end up downloading less than with other approaches and that even includes the downloads one gets when Maven upgrades itself.  

I do, agree, though, that Maven makes you drink the Kool-aid and it doesn't play well with other conventions (although it isn't horrible when it comes to Ant, either).  There are plenty of days I hate Maven for what it assumes, but there are also many days when I love the fact that the POM describes my project in one clear, fairly concise, "validatable" way.

>
> I
> still think Maven should be a downstream issue.

I don't see how it can be.  You have to be a committer to push it to the ASF repository for syndication on iBiblio, etc.  That being said, we really aren't that far from a process that we can have confidence in.

-Grant
---------------------------------------------------------------------

My 2c...

Maven is pretty much incompatible with some of the standards of release engineering, namely repeatable builds.  It tries to do pretty much the same job that apt does under debian and ubuntu and is therefore not terribly useful in that environment.  All the mavenistas I've talked to have no good answer to that, except perhaps for setting up local repositories to do what you need.

Another approach that might be worth considering is using ivy instead of maven per se.  There is a lot more control that way, and most of the engineers familiar with both approaches prefer ivy, if it's an option.

Karl
-----Original Message-----
From: ext Grant Ingersoll [mailto:[EMAIL PROTECTED]]
Sent: Monday, September 20, 2010 9:22 AM
To: dev@lucene.apache.org
Subject: Re: discussion about release frequency.
On Sep 20, 2010, at 9:00 AM, Mark Miller wrote:

>
>> (BTW, I love the "Maven is Magic" (and really any "It's magic, therefore I don't like it") reasoning for not liking it, whereby everyone complains that b/c Maven hides a bunch of details from you (i.e. it's "magic"), therefore you don't like it.  At the same time, I'm sure said person doesn't understand every last detail of, oh, I don't know: the CPU, RAM, the Compiler, the JDK, etc. and yet they have no problem using that.  In other words, we deal with abstractions all the time.  It's fine if you don't get the abstraction or don't personally find it useful, but that doesn't make the abstraction bad.)
>>
>> -Grant
>
> Maven is not bad because it's magic - magic is frigging great - I want
> my software to be magic - it's bad because every 5 line program from
> some open source code/project that I have tried to build with it has
> gone on an absurd downloading spree that takes forever because it's
> getting many tiny files. This downloading spree never corresponds to the
> size of the code base I am working with, and always manages to surprise
> by the amount of time it can slurp up.

Agreed, but over time, it is lessened by the fact that you already have most common files/jars and furthermore, you only have one copy of them instead of one under every source tree.  I think, over time, you actually end up downloading less than with other approaches and that even includes the downloads one gets when Maven upgrades itself.  

I do, agree, though, that Maven makes you drink the Kool-aid and it doesn't play well with other conventions (although it isn't horrible when it comes to Ant, either).  There are plenty of days I hate Maven for what it assumes, but there are also many days when I love the fact that the POM describes my project in one clear, fairly concise, "validatable" way.

>
> I
> still think Maven should be a downstream issue.

I don't see how it can be.  You have to be a committer to push it to the ASF repository for syndication on iBiblio, etc.  That being said, we really aren't that far from a process that we can have confidence in.

-Grant
---------------------------------------------------------------------
---------------------------------------------------------------------

On Mon, Sep 20, 2010 at 9:00 AM, Mark Miller <[EMAIL PROTECTED]> wrote:
> I still think Maven should be a downstream issue.

+1

Maven has never been a required part of our releases, and I don't
think we should change that.

We should also keep in mind that there's nothing really official about
a "release manager".
There's no reason the person(s) that signed the normal release need to
be the same person that signs the maven stuff (but it should be a PMC
member if it's hosted by the ASF).

If there are people around during a release that want to handle the
maven stuff, that seems fine.  It does *not* have to be the release
manager.  It seems fine to make reasonable accommodations if some are
working on making maven artifacts available at roughly the same... but
if not,  it should not hold up the release.

-Yonik
http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8

---------------------------------------------------------------------

On 2010-09-20 15:21, Grant Ingersoll wrote:
> I do, agree, though, that Maven makes you drink the Kool-aid and it doesn't play well with other conventions (although it isn't horrible when it comes to Ant, either).  There are plenty of days I hate Maven for what it assumes, but there are also many days when I love the fact that the POM describes my project in one clear, fairly concise, "validatable" way.

We took the middle road in Nutch - we switched to ant+ivy to manage
dependencies. This way we get single copies of all deps, and build.xml
is still recognizable and useful. Of coure, this doesn't solve the
publishing part of Maven functionality (yet).

--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com
---------------------------------------------------------------------

On Sep 20, 2010, at 9:55 AM, Yonik Seeley wrote:

> On Mon, Sep 20, 2010 at 9:00 AM, Mark Miller <[EMAIL PROTECTED]> wrote:
>> I still think Maven should be a downstream issue.
>
> +1
>
> Maven has never been a required part of our releases, and I don't
> think we should change that.
>
> We should also keep in mind that there's nothing really official about
> a "release manager".
> There's no reason the person(s) that signed the normal release need to
> be the same person that signs the maven stuff (but it should be a PMC
> member if it's hosted by the ASF).
>
> If there are people around during a release that want to handle the
> maven stuff, that seems fine.  It does *not* have to be the release
> manager.  It seems fine to make reasonable accommodations if some are
> working on making maven artifacts available at roughly the same... but
> if not,  it should not hold up the release.

I completely disagree.  It's either a first class citizen or it's not and by moving it out, you're guaranteeing it will not be consistently right.   I think it would be interesting to take a poll as to who uses Maven in the user community.

-Grant
---------------------------------------------------------------------

On 9/20/2010 at 8:24 AM, Grant Ingersoll wrote:
> At any rate, the big problem w/ Maven and Lucene is not that generate-
> maven-artifacts doesn't work, it's that the POM templates aren't kept in
> sync.  However, I think we now have a solution for that thanks to Steve
> and Robert's work to make it easier to bring Lucene into IntelliJ.  In
> other words, that process does much of what is needed for Maven, so it
> should be relatively straightforward to have it automatically generate the
> templates, too.  In fact, it would be just as easy for that project to
> simply produce POM files (which are well understood and have a published
> spec) instead of creating the IntelliJ project files, which are not well
> understood and not publicly spec'd and subject to change w/ every release
> and simply have IntelliJ suck in the POM file since IntelliJ supports that
> very, very well.

Unfortunately, LUCENE-2611 does not automatically generate IntelliJ setup files - they are static, just like the POM template files.  I think it's possible, using an Ant BuildListener-extending class, to do automatic generation, but I haven't attempted it yet.  I'll open an issue.

Steve

On Mon, Sep 20, 2010 at 10:18 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> On Sep 20, 2010, at 9:55 AM, Yonik Seeley wrote:
>
>> On Mon, Sep 20, 2010 at 9:00 AM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>> I still think Maven should be a downstream issue.
>>
>> +1
>>
>> Maven has never been a required part of our releases, and I don't
>> think we should change that.
>>
>> We should also keep in mind that there's nothing really official about
>> a "release manager".
>> There's no reason the person(s) that signed the normal release need to
>> be the same person that signs the maven stuff (but it should be a PMC
>> member if it's hosted by the ASF).
>>
>> If there are people around during a release that want to handle the
>> maven stuff, that seems fine.  It does *not* have to be the release
>> manager.  It seems fine to make reasonable accommodations if some are
>> working on making maven artifacts available at roughly the same... but
>> if not,  it should not hold up the release.
>
> I completely disagree.

With what part?  Do you mean to say you wish to make maven a required
part of our releases?
If so, perhaps you should call a vote?

>  It's either a first class citizen or it's not and by moving it out

It is not a first class citizen.  Apparently the last Solr release
went out w/o working maven support.

But it's not quite so black and white either... I see no reason to
*remove* maven related stuff from ant (and it's good if people improve
it), and I've even applied patches to the maven stuff when supplied by
others.

-Yonik

---------------------------------------------------------------------

>
> I completely disagree.  It's either a first class citizen or it's not and by moving it out, you're guaranteeing it will not be consistently right.   I think it would be interesting to take a poll as to who uses Maven in the user community.
>

For sure... though I also suspect the majority of lucene maven users
are not on the mailing list.  As a maven user, people can just depend
on 'h2 full text' or 'hibernate search' and never worry about what
version of lucene they are using (or even that it is lucene at all)

ryan

---------------------------------------------------------------------

On Sep 20, 2010, at 10:28 AM, Steven A Rowe wrote:

> On 9/20/2010 at 8:24 AM, Grant Ingersoll wrote:
>> At any rate, the big problem w/ Maven and Lucene is not that generate-
>> maven-artifacts doesn't work, it's that the POM templates aren't kept in
>> sync.  However, I think we now have a solution for that thanks to Steve
>> and Robert's work to make it easier to bring Lucene into IntelliJ.  In
>> other words, that process does much of what is needed for Maven, so it
>> should be relatively straightforward to have it automatically generate the
>> templates, too.  In fact, it would be just as easy for that project to
>> simply produce POM files (which are well understood and have a published
>> spec) instead of creating the IntelliJ project files, which are not well
>> understood and not publicly spec'd and subject to change w/ every release
>> and simply have IntelliJ suck in the POM file since IntelliJ supports that
>> very, very well.
>
> Unfortunately, LUCENE-2611 does not automatically generate IntelliJ setup files - they are static, just like the POM template files.

Hmm, hadn't looked that closely.  I'd say this is going to suffer the same fate of the POM template files then and would thus be against including it.

>  I think it's possible, using an Ant BuildListener-extending class, to do automatic generation, but I haven't attempted it yet.  I'll open an issue.
Cool.
---------------------------------------------------------------------

On 9/20/2010 at 11:15 AM, Grant Ingersoll wrote:
>> Unfortunately, LUCENE-2611 does not automatically generate IntelliJ
>> setup files - they are static, just like the POM template files.
>
> Hmm, hadn't looked that closely.  I'd say this is going to suffer the same
> fate of the POM template files then and would thus be against including
> it.

It's not quite as bad as the POM template files, since IntelliJ can be told to find all dependencies in a directory, rather than explicitly naming every dependency, and LUCENE-2611 uses that facility just about everywhere (I think the only exception is the JUnit jar test dependency, since the other stuff in the same directory shouldn't necessarily be depended on during testing).

So the IntelliJ project files in LUCENE-2611 would continue to work without manual intervention in the face of upgraded and/or additional dependencies, but would require manual effort to sync up with structural changes.  While I don't agree that this is a deal-breaker, since the manual intervention required would be fairly minimal, I agree that auto-generation would be a lot more useful than the current static approach.

My thought process was that setting this up manually would provide a benchmark for auto-generation; the auto-generated version should not be less functional than the manually generated one.

Steve

On Mon, Sep 20, 2010 at 10:29 AM, Yonik Seeley
<[EMAIL PROTECTED]>wrote:

>
> With what part?  Do you mean to say you wish to make maven a required
> part of our releases?
> If so, perhaps you should call a vote?
>
>
It sounds like maybe we should, since the situation seems ambiguous at the
moment, and there is a lot of different opinions/disagreement.

Though we should make it clear what we are voting on:
1. having the maven stuff in our source code.
2. maven being part of the release process.

As far as #1 goes, to me its just like any other code. We have some other
code already with no tests and I don't think thats reason to remove it from
the source tree, when it can be improved by adding tests.

However, I don't think its acceptable that the release process (#2) is
currently undefined wrt maven. I think we need to get this nailed down, i
think we need a well-defined release process.

--
Robert Muir
[EMAIL PROTECTED]

On Mon, Sep 20, 2010 at 12:01 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>
> On Mon, Sep 20, 2010 at 10:29 AM, Yonik Seeley <[EMAIL PROTECTED]>
> wrote:
>>
>> With what part?  Do you mean to say you wish to make maven a required
>> part of our releases?
>> If so, perhaps you should call a vote?
>>
>
> It sounds like maybe we should

I'm not sure it would be useful yet.  There is consensus that the
process needs to improve.  The only concrete 'vote' i could imagine
now is to drop maven.
ryan

---------------------------------------------------------------------

On Mon, Sep 20, 2010 at 12:29 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

>
> I'm not sure it would be useful yet.  There is consensus that the
> process needs to improve.  The only concrete 'vote' i could imagine
> now is to drop maven.
>
>
I completely agree the process needs to improve, but at the end of the day,
if we are planning to support maven officially in releases, i think we
should vote on it becoming part of the actual release process.

So maybe its premature to vote on this part, but at the same time, I have
concerns about what it would take to 'fully support' maven.

For example, if we have to reorganize our source tree to what it wants
(src/main/java, src/main/test), and rename our artifacts to what it wants
(-SNAPSHOT, etc), this is pretty important. what else might maven 'require'.

its also my understanding that in the past, when maven is upgraded (e.g.
Maven 2), it might require you to modify your project in ways such as this
to fit its new "needs".

>From what I know of maven, its quite inflexible about such things, and I
want to know what i'm getting into before we claim to 'make maven first
class citizen'.

--
Robert Muir
[EMAIL PROTECTED]

On 9/20/2010 at 12:35 PM, Robert Muir wrote:
> So maybe its premature to vote on this part, but at the same
> time, I have concerns about what it would take to 'fully
> support' maven.
>
> For example, if we have to reorganize our source tree to
> what it wants (src/main/java, src/main/test), and rename our
> artifacts to what it wants (-SNAPSHOT, etc), this is pretty
> important. what else might maven 'require'.

Producing maven artifacts does not now and will not require source tree reorg.  If the build itself were converted from Ant to Maven, some reorg would likely be required, but there are way too many dead bodies that would have to be trampled on before Lucene/Solr Ant->Maven build conversion could happen...

The -SNAPSHOT.jar renaming thing is because that is the only suffix Maven recognizes as a snapshot, for which special handling is required, so that the latest version is always acquired from the Maven repository by local builds depending on Lucene/Solr artifacts.

> its also my understanding that in the past, when maven is
> upgraded (e.g. Maven 2), it might require you to modify your
> project in ways such as this to fit its new "needs".

AFAICT, Maven 3.0 is fairly close to release (in 3rd beta release).  One of the goals of this release is maximizing backward compatibility, so I don't think this is anywhere near as much of a concern as it was last time around (1->2).  Here are detailed compatibility notes:

<https://cwiki.apache.org/MAVEN/maven-3x-compatibility-notes.html>

Nothing major there, and in a non-Maven build that just wants to produce Maven artifacts, probably no concerns at all (IMHO).

Steve

If somebody reorders the directory structure, I will shout “revert revert revert” J

 

We can only “fully” support maven by switching to maven, but most of the core committers don’t want this (including me). In my opinion, the approach we had was fine, to simply create the jar files as we do for the binary release, but add some (hopefully) automatically generated pom files to it.

 

One thing I don’t like in this release process (as it currently works) is non-repeatable maven artifact generation. With maven, it’s impossible to regenerate the JAR files with the *same* MD5, even the MD5’s of the jar files in the binary release zip are different than the maven ones. If repeatability is not possible, at least the JAR files in the –bin.zip should be identical to the maven released ones!

 

Uwe

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

 <http://www.thetaphi.de/> http://www.thetaphi.de

eMail: [EMAIL PROTECTED]

 

From: Robert Muir [mailto:[EMAIL PROTECTED]]
Sent: Monday, September 20, 2010 9:35 AM
To: dev@lucene.apache.org
Subject: Re: discussion about release frequency.

 

 

On Mon, Sep 20, 2010 at 12:29 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote:

 

I'm not sure it would be useful yet.  There is consensus that the
process needs to improve.  The only concrete 'vote' i could imagine
now is to drop maven.

 

I completely agree the process needs to improve, but at the end of the day, if we are planning to support maven officially in releases, i think we should vote on it becoming part of the actual release process.

 

So maybe its premature to vote on this part, but at the same time, I have concerns about what it would take to 'fully support' maven.

 

For example, if we have to reorganize our source tree to what it wants (src/main/java, src/main/test), and rename our artifacts to what it wants (-SNAPSHOT, etc), this is pretty important. what else might maven 'require'.

 

its also my understanding that in the past, when maven is upgraded (e.g. Maven 2), it might require you to modify your project in ways such as this to fit its new "needs".

 

>From what I know of maven, its quite inflexible about such things, and I want to know what i'm getting into before we claim to 'make maven first class citizen'.

 

--
Robert Muir
[EMAIL PROTECTED]

On Mon, Sep 20, 2010 at 12:54 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:

> If somebody reorders the directory structure, I will shout “revert revert
> revert” J
>

I wouldn't shout "revert revert revert" if by renaming stuff from src/java
to src/main/java etc, Grant's idea would work, in that we still use ant for
our build, but we have some way to automagically generate IDE configuration
files for eclipse, idea, netbeans, emacs, whatever, via some maven tool.

If this was the benefit, and the tradeoff being more difficult merging, and
having to ignore some path segments on existing patches, I might consider it
worth the cost.

but again, i have serious questions about maven in general. for example,
what if I wanted to add/modify a contrib that depends on a library that is
not "mavenized"?   Is it my responsibility to "mavenize" that dependency,
too? Does it make the release artifact invalid? is it a valid reason against
adding that contrib, since its dependencies are not all mavenized?

the fact that maven acts like a computer virus, but requires special things
of its hosts, means that i am pretty hesitant to vote for "full support of
it" without knowing exactly what the tradeoffs are.

--
Robert Muir
[EMAIL PROTECTED]

“but again, i have serious questions about maven in general.”

Maybe you just need to drink the Maven Koolaid.  Unless they have something stronger… ;-)

Karl
From: ext Robert Muir [mailto:[EMAIL PROTECTED]]
Sent: Monday, September 20, 2010 1:08 PM
To: dev@lucene.apache.org
Subject: Re: discussion about release frequency.
On Mon, Sep 20, 2010 at 12:54 PM, Uwe Schindler <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
If somebody reorders the directory structure, I will shout “revert revert revert” ☺

I wouldn't shout "revert revert revert" if by renaming stuff from src/java to src/main/java etc, Grant's idea would work, in that we still use ant for our build, but we have some way to automagically generate IDE configuration files for eclipse, idea, netbeans, emacs, whatever, via some maven tool.

If this was the benefit, and the tradeoff being more difficult merging, and having to ignore some path segments on existing patches, I might consider it worth the cost.

but again, i have serious questions about maven in general. for example, what if I wanted to add/modify a contrib that depends on a library that is not "mavenized"?   Is it my responsibility to "mavenize" that dependency, too? Does it make the release artifact invalid? is it a valid reason against adding that contrib, since its dependencies are not all mavenized?

the fact that maven acts like a computer virus, but requires special things of its hosts, means that i am pretty hesitant to vote for "full support of it" without knowing exactly what the tradeoffs are.

--
Robert Muir
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

On Sep 20, 2010, at 1:07 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 12:54 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> If somebody reorders the directory structure, I will shout “revert revert revert” J
>
>
> I wouldn't shout "revert revert revert" if by renaming stuff from src/java to src/main/java etc, Grant's idea would work, in that we still use ant for our build, but we have some way to automagically generate IDE configuration files for eclipse, idea, netbeans, emacs, whatever, via some maven tool.
>
> If this was the benefit, and the tradeoff being more difficult merging, and having to ignore some path segments on existing patches, I might consider it worth the cost.
>
> but again, i have serious questions about maven in general. for example, what if I wanted to add/modify a contrib that depends on a library that is not "mavenized"?   Is it my responsibility to "mavenize" that dependency, too? Does it make the release artifact invalid? is it a valid reason against adding that contrib, since its dependencies are not all mavenized?

Typically, this is done by adding the library in question to the release, renamed appropriately.  For instance, in Solr, we had a trunk based version of Commons CSV at one point, so we put it up w/ the Solr artifacts and had the POM reflect that.  But yeah, it can be a pain.

>
> the fact that maven acts like a computer virus, but requires special things of its hosts, means that i am pretty hesitant to vote for "full support of it" without knowing exactly what the tradeoffs are.

I'm not saying we have to support it, but, in my view, it's pretty hard to take back a feature, admittedly only for some, that we have supported for a long time.

---------------------------------------------------------------------

On Mon, Sep 20, 2010 at 2:31 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> Typically, this is done by adding the library in question to the release,
> renamed appropriately.  For instance, in Solr, we had a trunk based version
> of Commons CSV at one point, so we put it up w/ the Solr artifacts and had
> the POM reflect that.  But yeah, it can be a pain.
>

I don't understand this, if I, as a lucene committer, can arbitrarily
publish commons CSV artifacts under maven, without being a commons CSV
committer, then why does someone have to be a lucene committer to publish
maven artifacts?!

Furthermore, if this is possible, then why does lucene itself have to
support maven, if someone else (e.g. hibernate) can simply download our jar
files and do the same?
> I'm not saying we have to support it, but, in my view, it's pretty hard to
> take back a feature, admittedly only for some, that we have supported for a
> long time.
>
>
I'm not sure we supported it, it seems to be a broken feature in nearly
every release.

--
Robert Muir
[EMAIL PROTECTED]

On Sep 20, 2010, at 2:37 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 2:31 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> Typically, this is done by adding the library in question to the release, renamed appropriately.  For instance, in Solr, we had a trunk based version of Commons CSV at one point, so we put it up w/ the Solr artifacts and had the POM reflect that.  But yeah, it can be a pain.
>
> I don't understand this, if I, as a lucene committer, can arbitrarily publish commons CSV artifacts under maven, without being a commons CSV committer, then why does someone have to be a lucene committer to publish maven artifacts?!

It's under the Solr area, not the commons CSV area.  

>
> Furthermore, if this is possible, then why does lucene itself have to support maven, if someone else (e.g. hibernate) can simply download our jar files and do the same?
>
>
> I'm not saying we have to support it, but, in my view, it's pretty hard to take back a feature, admittedly only for some, that we have supported for a long time.
>
>
> I'm not sure we supported it, it seems to be a broken feature in nearly every release.

Nah, some times some pieces are broken, but the core one always works, AFAICT.  ;-)
---------------------------------------------------------------------

On Mon, Sep 20, 2010 at 2:43 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> It's under the Solr area, not the commons CSV area.
>

sure, but this doesn't answer the question. if other projects can do this
same trick, why do we need to do any maven at all? we can just let those
that want maven support, provide it themselves. Ultimately this would
probably mean they do a better job of it anyway, since they care about it
working for their project to work.

--
Robert Muir
[EMAIL PROTECTED]

On Sep 20, 2010, at 2:47 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 2:43 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> It's under the Solr area, not the commons CSV area.
>
> sure, but this doesn't answer the question. if other projects can do this same trick, why do we need to do any maven at all? we can just let those that want maven support, provide it themselves. Ultimately this would probably mean they do a better job of it anyway, since they care about it working for their project to work.

Not following.  Joe Schmoe w/ project X doesn't have the right to go publish artifacts at org.apache.lucene.XXX in the iBiblio repository.  And, in many cases, we may not have the right to publish others, but for Apache projects, we can.  Otherwise, in the past, I've often asked the dependency authors to produce them.  Most people will if it means they are getting a wider distribution.  In practice, it rarely is an issue.

-Grant

On Mon, Sep 20, 2010 at 3:30 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> Not following.  Joe Schmoe w/ project X doesn't have the right to go
> publish artifacts at org.apache.lucene.XXX in the iBiblio repository.  And,
> in many cases, we may not have the right to publish others, but for Apache
> projects, we can.  Otherwise, in the past, I've often asked the dependency
> authors to produce them.  Most people will if it means they are getting a
> wider distribution.  In practice, it rarely is an issue.
>
>
right but why cant joe shmoe make joe.schmoe.luceneMaven.XXX in the iBiblio
repository?

At the end of the day, I'm trying to figure out if we can push maven
"downstream" as others have suggested, and it sounds like we can.

--
Robert Muir
[EMAIL PROTECTED]

On 9/20/10 3:36 PM, Robert Muir wrote:
>
> right but why cant joe shmoe make joe.schmoe.luceneMaven.XXX in the
> iBiblio repository?
>

That sounds enticing - someone else can step up to be the authority.

---------------------------------------------------------------------

On Sep 20, 2010, at 3:36 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 3:30 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> Not following.  Joe Schmoe w/ project X doesn't have the right to go publish artifacts at org.apache.lucene.XXX in the iBiblio repository.  And, in many cases, we may not have the right to publish others, but for Apache projects, we can.  Otherwise, in the past, I've often asked the dependency authors to produce them.  Most people will if it means they are getting a wider distribution.  In practice, it rarely is an issue.
>
>
> right but why cant joe shmoe make joe.schmoe.luceneMaven.XXX in the iBiblio repository?
>
> At the end of the day, I'm trying to figure out if we can push maven "downstream" as others have suggested, and it sounds like we can.
>

Why don't we just leave this as this:

Those of us who want Maven supported as part of the release need to get our stuff together by the next release or else it will be dropped.  That means making sure the artifacts are correct and easily testable/reproducible.  If we can't do that, then I agree, it should be a downstream effort, at least until we all realize how many people actually use it and then we revisit it at the next release.

-Grant

On Mon, Sep 20, 2010 at 3:46 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> Why don't we just leave this as this:
>
> Those of us who want Maven supported as part of the release need to get our
> stuff together by the next release or else it will be dropped.  That means
> making sure the artifacts are correct and easily testable/reproducible.  If
> we can't do that, then I agree, it should be a downstream effort, at least
> until we all realize how many people actually use it and then we revisit it
> at the next release.
>
>
But I'm not sure this is the best solution? If we can push this downstream,
so that the release manager has less to worry about (even with testable
artifacts etc, the publication etc), why wouldn't we do that instead?

--
Robert Muir
[EMAIL PROTECTED]

On Sep 20, 2010, at 3:49 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 3:46 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> Why don't we just leave this as this:
>
> Those of us who want Maven supported as part of the release need to get our stuff together by the next release or else it will be dropped.  That means making sure the artifacts are correct and easily testable/reproducible.  If we can't do that, then I agree, it should be a downstream effort, at least until we all realize how many people actually use it and then we revisit it at the next release.
>
>
> But I'm not sure this is the best solution? If we can push this downstream, so that the release manager has less to worry about (even with testable artifacts etc, the publication etc), why wouldn't we do that instead?
>

Because it's not authoritative.  How would our users know which one is the official one?  By publishing it under the ASF one with our signatures we are saying this is our official version.  We would never claim that the Solr Commons CSV one is the official Commons jar, it's just the official one that Solr officially uses.  It's a big difference.   Besides, it's not like the iBiblio repo is open to anyone.  You have to apply and you have to have authority to write to it.  For the ASF, there is a whole sync process whereby iBiblio syncs with an ASF version.  In other words, we are the only ones who can publish it to the same space where it is currently published.

-Grant

On Mon, Sep 20, 2010 at 4:11 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> Because it's not authoritative.  How would our users know which one is the
> official one?  By publishing it under the ASF one with our signatures we are
> saying this is our official version.  We would never claim that the Solr
> Commons CSV one is the official Commons jar, it's just the official one that
> Solr officially uses.  It's a big difference.   Besides, it's not like the
> iBiblio repo is open to anyone.  You have to apply and you have to have
> authority to write to it.  For the ASF, there is a whole sync process
> whereby iBiblio syncs with an ASF version.  In other words, we are the only
> ones who can publish it to the same space where it is currently published.
>
>
This "authoratitiveness" comes with a significant cost, that is the
complexity of maven in our release process.  I'm not convinced its worth
this cost, and before we decide to have maven as part of the release, i'd
like for there to be an actual vote.

Sorry to change my tone, but I was under the impression we needed a lucene
committer to do all this releasing work to support maven, it seems that this
is not the case, and other options are available.

--
Robert Muir
[EMAIL PROTECTED]

On Sep 20, 2010, at 4:15 PM, Robert Muir wrote:

>
>
> On Mon, Sep 20, 2010 at 4:11 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> Because it's not authoritative.  How would our users know which one is the official one?  By publishing it under the ASF one with our signatures we are saying this is our official version.  We would never claim that the Solr Commons CSV one is the official Commons jar, it's just the official one that Solr officially uses.  It's a big difference.   Besides, it's not like the iBiblio repo is open to anyone.  You have to apply and you have to have authority to write to it.  For the ASF, there is a whole sync process whereby iBiblio syncs with an ASF version.  In other words, we are the only ones who can publish it to the same space where it is currently published.
>
>
> This "authoratitiveness" comes with a significant cost, that is the complexity of maven in our release process.  I'm not convinced its worth this cost, and before we decide to have maven as part of the release, i'd like for there to be an actual vote.

I agree.  But, like I said, if those who want it step up and make it fully supported, then there is no more cost than uploading a few extra artifacts, then what's the extra cost?  As usual in open source, why don't we just leave it those who do the work?  If no one steps up and fixes it, then it doesn't get included.

>
> Sorry to change my tone, but I was under the impression we needed a lucene committer to do all this releasing work to support maven, it seems that this is not the case, and other options are available.
>

I'm sorry, I don't see the other options.  I think it does need to be done by a Lucene committer to be an official Lucene artifact.  OK, well, I suppose some other ASF person could do it, but short of a benevolent volunteer to do so, I don't think there are other options.

-Grant

On Mon, Sep 20, 2010 at 4:20 PM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> I'm sorry, I don't see the other options.  I think it does need to be done
> by a Lucene committer to be an official Lucene artifact.  OK, well, I
> suppose some other ASF person could do it, but short of a benevolent
> volunteer to do so, I don't think there are other options.
>
>
I will quote Ryan here: The "artifacts" are the identical .jar files put
into a special directory structure.

Therefore, if we release without maven, the jar files are signed by our
release key. this is authoritative enough, maven does check signatures
correct?

I'm not buying the authoritative argument, it seems like any old joker can
take our signed jars and put them in maven themselves, without us having to
do any work.

--
Robert Muir
[EMAIL PROTECTED]

: > to be seen wether we sould *need* to release that often -- it will
: > depend on wether anything new gets committed there -- but it would
...
: And as far as whether we *need* to release often, i'd like to start looking
: at whether we *should*.
: For a realistic example, Solr 3.x got autosuggest functionality. This isn't
: really a disruptive thing, its not going to introduce a lot of deprecations,
: etc.
: do we *need* to release because of that? no. *should* we release? I think
: yes.

Agreed, my point was merely that even if we get to the point where we are
capable fo releasing off the 3x breanch every month (because we have the
process/tools down pat) that doesn't mean we should.  we should release
when we have features that we think are worth releasing, and are stable
enough to support moving forward along that major branch.

For example: we probably shouldn't bother having a release if the only
thing commited to that branch since the previous release are to fix some
typoes in javadocs, or because new tests were added -- those changes are
good, and worth having, but too much proliferation of minor versions for
things that don't impact the users can be distracting and confusion, and
makes it hard to recognize when a release is worth upgrading too (it's a
girl who cried wolf thing).

The other situation to ocnsider is when a feature is commited which works
correctly, and has good tests and documentation, but people have
questions/reservations about wether the API is really what we want -- just
because the calender says it's time for the quarterly release, and the
code works, doesn't mean we should just blindly release.  (and yes, i
realize that for the 3x branch, these API decisions should probably be
hashed out on trunk before the feature is backported to 3x ... it's not
hte best example)

In any case, i'm really just trying to bring things back to the point i
was getting at in my last mail: the decision to release should be made
based on state of the code, not the calender; but it would be awesome if
hte process was automated enough that we could release as often as we
thought the state of the code warranted it.

: I think we should try to avoid massive yearly releases with a ton of changes
: at once.

No argument there ... in the past i think this was really a combination of
(a) "concerns about API/back-compat" and (b) "the release process is such
a nightmare let's get a few more features in before we release so we don't
have to do it again too soon" ... the 3x branch makes (a) a smaller
concern, if we can make (b) go away we can pretty much release whenever we
want.
-Hoss

--
http://lucenerevolution.org/  ...  October 7-8, Boston
http://bit.ly/stump-hoss      ...  Stump The Chump!
---------------------------------------------------------------------

> I'm not buying the authoritative argument, it seems like any old joker can
> take our signed jars and put them in maven themselves, without us having to
> do any work.

The "standard" places, (apache / ibiblio / sonatype) require a project
representatives to post artifacts to the repository.  If the artifacts
are hosted at "hotsteamylucene.ru" yes, anyone could post anything.  I
hope we agree it is worth making it easy for people to use lucene via
maven, ivy, or whatever.  We need to make sure this is not a pain, and
maybe split out parts so that the RM does not necessarily need to do
everything.

My understanding is that you would feel OK about this if there was
some automated test that used the artifacts, and that the RM need not
know anything about maven internal -- just copy a directory to
somewhere in apache infrastructure, I don't think this is too far off.

ryan

---------------------------------------------------------------------

On Tue, Sep 21, 2010 at 7:40 PM, Chris Hostetter
<hossman_lucene@fucit.org>wrote:

> For example: we probably shouldn't bother having a release if the only
> thing commited to that branch since the previous release are to fix some
> typoes in javadocs, or because new tests were added -- those changes are
> good, and worth having, but too much proliferation of minor versions for
> things that don't impact the users can be distracting and confusion, and
> makes it hard to recognize when a release is worth upgrading too (it's a
> girl who cried wolf thing).
>

i completely agree with you. I didn't mean to give the impression by "every
month or two" that we should actually have anything remotely resembling a
schedule driven by arbitrary dates. I meant here to suggest a very rough
idea of the sort of frequency that I think might actually work, and to bring
up the point that what we might consider minor features can be viewed by
users as major.

--
Robert Muir
[EMAIL PROTECTED]

From a users perspective, I would say that each one of FieldCollapse, AutoSuggest and SpatialSearch would justify a minor release (once stable).

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

On 22. sep. 2010, at 03.05, Robert Muir wrote:

>
>
> On Tue, Sep 21, 2010 at 7:40 PM, Chris Hostetter <hossman_lucene@fucit.org> wrote:
> For example: we probably shouldn't bother having a release if the only
> thing commited to that branch since the previous release are to fix some
> typoes in javadocs, or because new tests were added -- those changes are
> good, and worth having, but too much proliferation of minor versions for
> things that don't impact the users can be distracting and confusion, and
> makes it hard to recognize when a release is worth upgrading too (it's a
> girl who cried wolf thing).
>
> i completely agree with you. I didn't mean to give the impression by "every month or two" that we should actually have anything remotely resembling a schedule driven by arbitrary dates. I meant here to suggest a very rough idea of the sort of frequency that I think might actually work, and to bring up the point that what we might consider minor features can be viewed by users as major.
>
> --
> Robert Muir
> [EMAIL PROTECTED]
---------------------------------------------------------------------

On Sep 20, 2010, at 4:23 PM, Robert Muir wrote:
>
> I'm not buying the authoritative argument, it seems like any old joker can take our signed jars and put them in maven themselves, without us having to do any work.

No, they can't.  They can put them in their own repo, but they can't put them in the official ones that are by default baked into Maven (the iBiblio one).

At the end of the day, I don't see what the big deal is.  For those of us who want Maven, we need to step up and make it work seamlessly.  Why should you care if we do that?  Given it gets fixed, it won't effect you.  I understand your concern now since it is broken, so I stand by the thought that if we (the Maven campers) can't fix it by the next release it can be dropped for that release.

-Grant

On Wed, Sep 22, 2010 at 8:08 AM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> At the end of the day, I don't see what the big deal is.  For those of us
> who want Maven, we need to step up and make it work seamlessly.  Why should
> you care if we do that?  Given it gets fixed, it won't effect you.  I
> understand your concern now since it is broken, so I stand by the thought
> that if we (the Maven campers) can't fix it by the next release it can be
> dropped for that release.
>
>
To me, the real problem is there does not seem to be consensus that maven is
part of the release process.

For example, you feel it is "defacto part of the release", and that calling
it optional is removing a feature.
Others have mentioned they feel it was always optional, and to be "part of
releasing" needs a vote/some sort of consensus

I already said that personally, if there is a way to verify that it works,
*my* concerns go away. And if i was moving stuff around, or refactoring the
build, or adding/upgrading a dependency, i don't mind at all hacking on some
pom.xml in a minor way until something like 'ant test-maven' succeeds.  But
currently it is the onus of the RM to *eyeball all these xml files* and
determine they are correct.

Still, this is just my personal opinion, and doesn't speak for everyone
else. So what i was trying to do was to get all the options out on the
table: such as maven maintained downstream, maven part of the release, maven
not part of the release, etc.

--
Robert Muir
[EMAIL PROTECTED]

On Sep 22, 2010, at 8:26 AM, Robert Muir wrote:

>
>
> On Wed, Sep 22, 2010 at 8:08 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> At the end of the day, I don't see what the big deal is.  For those of us who want Maven, we need to step up and make it work seamlessly.  Why should you care if we do that?  Given it gets fixed, it won't effect you.  I understand your concern now since it is broken, so I stand by the thought that if we (the Maven campers) can't fix it by the next release it can be dropped for that release.
>
>
> To me, the real problem is there does not seem to be consensus that maven is part of the release process.
>
> For example, you feel it is "defacto part of the release", and that calling it optional is removing a feature.
> Others have mentioned they feel it was always optional, and to be "part of releasing" needs a vote/some sort of consensus

Does an Indic Analyzer have to be part of the release?  I have no need for it and it is optional for a lot (likely most) of people.  It's a pain for me as I have no way of knowing that it is really working b/c I don't speak the language.  Besides, I don't know if the Unit Tests are comprehensive so there could be things broken in it.  Can I choose to not include it if I am the RM?  Same goes for Armenian and all of the other languages I don't speak.  Same goes for the Berkeley DB stuff, the Surround Query Parser, Instantiated Index and a whole slew of other "features" we release that I, as an RM, have no use for.  My life as an RM would be a whole lot easier if I simply packaged up core and Solr as source and put them up for download.   Heck, why bother releasing the docs?  Some of it isn't always correct and I don't have a way to test that automatically either.  Shall I continue?

We've been providing Maven releases since 2005.  That's Lucene 1.2!!!!!!!!!  That's pretty much longer than everyone here has been a committer other than Hatcher and Otis.  So, yeah, I do think it is removing a pretty big feature.

>
> I already said that personally, if there is a way to verify that it works, *my* concerns go away. And if i was moving stuff around, or refactoring the build, or adding/upgrading a dependency, i don't mind at all hacking on some pom.xml in a minor way until something like 'ant test-maven' succeeds.  But currently it is the onus of the RM to *eyeball all these xml files* and determine they are correct.

Again, for the 5th or 6th time, this is why I said the onus is on those who want Maven to fix it.   I don't expect you to care about it but that doesn't mean you should prevent it from being in the release.   I totally agree that it needs to be fixed, but don't act like it is any different from the myriad of other things that we release, some of which are broken as well.  Besides the main stuff that is broken w/ Maven is not the core pieces, but contribs.

>
> Still, this is just my personal opinion, and doesn't speak for everyone else. So what i was trying to do was to get all the options out on the table: such as maven maintained downstream, maven part of the release, maven not part of the release, etc.

I get your reasoning, but unfortunately, because you don't understand how Maven works you don't realize why maintaining it downstream is not an option.  Sorry.

-Grant
---------------------------------------------------------------------

> I get your reasoning, but unfortunately, because you don't understand how Maven works you don't realize why maintaining it downstream is not an option.  Sorry.
>

Seems very doable to me. A couple dudes get together and become the
authority for Lucene/Solr Maven themselves. Apply to Mavens whatever
process, make a website, throw a party, make maven artifacts. In a short
time you will be known as the source of Maven artifacts for Lucene/Solr,
sending out our signed jars to the masses. We don't need to do it. No
one has said anything about the Maven process that locks us into doing it.

Comparing Maven support to code features is insane IMO. We are in the
business of code in my opinion. Code and tagging code as a release - at
most putting the release on the mirrors.

Again, I'm still of the opinion Maven should be downstream.

- Mark

---------------------------------------------------------------------

On Sep 22, 2010, at 8:56 AM, Mark Miller wrote:

>
>> I get your reasoning, but unfortunately, because you don't understand how Maven works you don't realize why maintaining it downstream is not an option.  Sorry.
>>
>
> Seems very doable to me. A couple dudes get together and become the
> authority for Lucene/Solr Maven themselves. Apply to Mavens whatever
> process, make a website, throw a party, make maven artifacts. In a short
> time you will be known as the source of Maven artifacts for Lucene/Solr,
> sending out our signed jars to the masses. We don't need to do it. No
> one has said anything about the Maven process that locks us into doing it.
>

Like I said, you don't get how it works.  It's obvious by the comment.

> Comparing Maven support to code features is insane IMO. We are in the
> business of code in my opinion. Code and tagging code as a release - at
> most putting the release on the mirrors.

So, if the large majority of people get Lucene through Maven would you feel the same way?
---------------------------------------------------------------------

On 9/22/10 9:00 AM, Grant Ingersoll wrote:
>
> On Sep 22, 2010, at 8:56 AM, Mark Miller wrote:
>
>>
>>> I get your reasoning, but unfortunately, because you don't understand how Maven works you don't realize why maintaining it downstream is not an option.  Sorry.
>>>
>>
>> Seems very doable to me. A couple dudes get together and become the
>> authority for Lucene/Solr Maven themselves. Apply to Mavens whatever
>> process, make a website, throw a party, make maven artifacts. In a short
>> time you will be known as the source of Maven artifacts for Lucene/Solr,
>> sending out our signed jars to the masses. We don't need to do it. No
>> one has said anything about the Maven process that locks us into doing it.
>>
>
> Like I said, you don't get how it works.  It's obvious by the comment.

No, I looked - that works. Your too concerned with iBiblio.

>
>> Comparing Maven support to code features is insane IMO. We are in the
>> business of code in my opinion. Code and tagging code as a release - at
>> most putting the release on the mirrors.
>
> So, if the large majority of people get Lucene through Maven would you feel the same way?

That would be great! There would be enough people into Maven to actually
handle the support downstream!

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------

On Wed, Sep 22, 2010 at 8:47 AM, Grant Ingersoll <[EMAIL PROTECTED]>wrote:

>
> Does an Indic Analyzer have to be part of the release?  I have no need for
> it and it is optional for a lot (likely most) of people.  It's a pain for me
> as I have no way of knowing that it is really working b/c I don't speak the
> language.  Besides, I don't know if the Unit Tests are comprehensive so
> there could be things broken in it.  Can I choose to not include it if I am
> the RM?  Same goes for Armenian and all of the other languages I don't
> speak.  Same goes for the Berkeley DB stuff, the Surround Query Parser,
> Instantiated Index and a whole slew of other "features" we release that I,
> as an RM, have no use for.  My life as an RM would be a whole lot easier if
> I simply packaged up core and Solr as source and put them up for download.
> Heck, why bother releasing the docs?  Some of it isn't always correct and I
> don't have a way to test that automatically either.  Shall I continue?
>

your comparison has some flaws, while its true there are likely bugs in all
this stuff (and the tests are not 100%), they do in fact have tests.

for the javadocs, it too has tests. when you run the javadocs target, it
verifies several things: and at the end outputs warnings and errors.

maven has nothing.
>
> Again, for the 5th or 6th time, this is why I said the onus is on those who
> want Maven to fix it.   I don't expect you to care about it but that doesn't
> mean you should prevent it from being in the release.   I totally agree that
> it needs to be fixed, but don't act like it is any different from the myriad
> of other things that we release, some of which are broken as well.  Besides
> the main stuff that is broken w/ Maven is not the core pieces, but contribs.
>

this is still a serious problem to me, especially if we are going the path
of modularizing lucene, then it will become even more serious in the future.

i'm not trying to 'prevent' maven from being part of the release. I'm trying
to get resolution on this maven issue, for which consensus does not exist.

I get your reasoning, but unfortunately, because you don't understand how
> Maven works you don't realize why maintaining it downstream is not an
> option.  Sorry.
>

i admit i dont understand how maven works, but it does seem to be a viable
alternative. Just because you don't personally like that alternative,
doesn't mean that its completely invalid.
--
Robert Muir
[EMAIL PROTECTED]

On Sep 22, 2010, at 9:02 AM, Mark Miller wrote:

> On 9/22/10 9:00 AM, Grant Ingersoll wrote:
>>
>> On Sep 22, 2010, at 8:56 AM, Mark Miller wrote:
>>
>>>
>>>> I get your reasoning, but unfortunately, because you don't understand how Maven works you don't realize why maintaining it downstream is not an option.  Sorry.
>>>>
>>>
>>> Seems very doable to me. A couple dudes get together and become the
>>> authority for Lucene/Solr Maven themselves. Apply to Mavens whatever
>>> process, make a website, throw a party, make maven artifacts. In a short
>>> time you will be known as the source of Maven artifacts for Lucene/Solr,
>>> sending out our signed jars to the masses. We don't need to do it. No
>>> one has said anything about the Maven process that locks us into doing it.
>>>
>>
>> Like I said, you don't get how it works.  It's obvious by the comment.
>
> No, I looked - that works. Your too concerned with iBiblio.

iBiblio is how people get artifacts w/o having to configure anything.  It is the defacto repository.  We've been publishing to iBiblio for 5 years.  To all of a sudden stop b/c a few devs who don't know about Maven and don't use Maven don't think it should be supported despite the fact that those who do want it are willing to do the work is stupid.

>
>>
>>> Comparing Maven support to code features is insane IMO. We are in the
>>> business of code in my opinion. Code and tagging code as a release - at
>>> most putting the release on the mirrors.
>>
>> So, if the large majority of people get Lucene through Maven would you feel the same way?
>
> That would be great! There would be enough people into Maven to actually
> handle the support downstream!

I really don't get all this venom towards Maven.  No one is asking you to do anything about it.  Ryan and I will fix it.  Or we won't.  Either way it doesn't effect you, so why do you care so much as to say I can't work on it and have it in a release?
---------------------------------------------------------------------

On Wed, Sep 22, 2010 at 8:47 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> We've been providing Maven releases since 2005.  That's Lucene 1.2!!!!!!!!!  That's pretty much longer than everyone here has been a committer other than Hatcher and Otis.  So, yeah, I do think it is removing a pretty big feature.

I really wanted to exit this discussion - but it's not true *we've*
been providing maven releases since xxx.

Doug's release of 1.9 didn't have maven.
My release of 2.1 didn't have maven (though it did have a discussion on it):
http://www.mail-archive.com/java-dev@lucene.apache.org/msg08528.html

AFAIK, people who cared about maven did the work after the fact or something.

-Yonik

---------------------------------------------------------------------