What should the next version of Solr be?

Options:
- have a Solr 1.5 with a lucene 2.9.x
- have a Solr 1.5 with a lucene 3.x, with weaker back compat given all
of the removed lucene deprecations from 2.9->3.0
- have a Solr 2.0 with a lucene 3.x

-Yonik
http://www.lucidimagination.com

On Thu, Nov 19, 2009 at 3:31 PM, Yonik Seeley
<[EMAIL PROTECTED]> wrote:
> What should the next version of Solr be?
>
> Options:
> - have a Solr 1.5 with a lucene 2.9.x
> - have a Solr 1.5 with a lucene 3.x, with weaker back compat given all
> of the removed lucene deprecations from 2.9->3.0
> - have a Solr 2.0 with a lucene 3.x

IMHO, Solr 2.0 given the removed deprecations in Lucene 3.0 and the
typical mindset that major versions indicate BC breaks

This can then also be used to remove deprecated Solr bits as well

Just my €0.0.2

Paul

Yonik Seeley wrote:
> What should the next version of Solr be?
>
> Options:
> - have a Solr 1.5 with a lucene 2.9.x
> - have a Solr 1.5 with a lucene 3.x, with weaker back compat given all
> of the removed lucene deprecations from 2.9->3.0
> - have a Solr 2.0 with a lucene 3.x
>
> -Yonik
> http://www.lucidimagination.com
>  
+1 for 2.0 with 3.x - not going to be easy keeping devs from taking
advantage of new Lucene features for another whole release period. And
1.5 with 3.x doesn't make a lot of sense if its going to be weaker back
compat - 2.0 makes sense in that case.

--
- Mark

http://www.lucidimagination.com

Since Solr is dependent on Lucene I agree that there should be a major
version number bump in Solr whenever there is one in Lucene:

Solr 2.x with Lucene 3.x

On Thu, Nov 19, 2009 at 11:11 AM, Mark Miller <[EMAIL PROTECTED]> wrote:

> Yonik Seeley wrote:
> > What should the next version of Solr be?
> >
> > Options:
> > - have a Solr 1.5 with a lucene 2.9.x
> > - have a Solr 1.5 with a lucene 3.x, with weaker back compat given all
> > of the removed lucene deprecations from 2.9->3.0
> > - have a Solr 2.0 with a lucene 3.x
> >
> > -Yonik
> > http://www.lucidimagination.com
> >
> +1 for 2.0 with 3.x - not going to be easy keeping devs from taking
> advantage of new Lucene features for another whole release period. And
> 1.5 with 3.x doesn't make a lot of sense if its going to be weaker back
> compat - 2.0 makes sense in that case.
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>

Unfortunately I accidentally started this thread on java-dev.
FWIW, I agree with Ryan's ranking below:

> In general, I wonder where the solr back-compatibility contract applies (and to what degree).  For solr, I would rank > the importance as:
> #1 - the URL API syntax.  Client query parameters should change as little as possible
> #2 - configuration
> #3 - java APIs

And it depends on exactly what the Java API is of course... common
analysis components like filter factories probably being the most
important of the Java APIs.

-Yonik
http://www.lucidimagination.com

Gun to my head the ranking makes sense - but I don't think it has any  
practical application. Plugin back compat is important and independnt  
of the urls.

I think solrj back compat is important too - I can understand  
experimental, but it's still important.

- Mark

http://www.lucidimagination.com (mobile)

On Nov 19, 2009, at 8:22 PM, Yonik Seeley <[EMAIL PROTECTED]>  
wrote:

> Unfortunately I accidentally started this thread on java-dev.
> FWIW, I agree with Ryan's ranking below:
>
>> In general, I wonder where the solr back-compatibility contract  
>> applies (and to what degree).  For solr, I would rank > the  
>> importance as:
>> #1 - the URL API syntax.  Client query parameters should change as  
>> little as possible
>> #2 - configuration
>> #3 - java APIs
>
> And it depends on exactly what the Java API is of course... common
> analysis components like filter factories probably being the most
> important of the Java APIs.
>
> -Yonik
> http://www.lucidimagination.com

Hey Yonik,

My personal experience with this is if you jump directly to 2.0, you'll have people wondering where 1.5, 1.6-->1.9 is in the CM system, and this would create some confusion unless it is documented well. This may warrant rethinking the tag structure a bit in SVN, or perhaps even the regular trunk structure at some point down the road. What's the SOLR versioning scheme by the way? Is it:

M.n

Where M is the major version #
Where n is the minor version #

In this type of scheme, all n's in a series are expected to be at least backwards compatible with n-1 through 0-9, but all M+1's aren't necessarily compatible with M or M-1. We've used this approach in Nutch and Tika land for a while and it's been pretty successful.

If this versioning scheme isn't documented somewhere, I'd be happy to throw up a page on the Wiki. Let me know and thanks. With the right documentation, the move from 1.4->2.0 will introduce less confusion, IMHO.

Cheers,
Chris

On 11/19/09 6:31 AM, "Yonik Seeley" <[EMAIL PROTECTED]> wrote:

What should the next version of Solr be?

Options:
- have a Solr 1.5 with a lucene 2.9.x
- have a Solr 1.5 with a lucene 3.x, with weaker back compat given all
of the removed lucene deprecations from 2.9->3.0
- have a Solr 2.0 with a lucene 3.x

-Yonik
http://www.lucidimagination.com
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Ryan McKinley wrote:
>>>
>>> In general, I wonder where the solr back-compatibility contract
>>> applies (and to what degree).  For solr, I would rank the importance
>>> as:
>>> #1 - the URL API syntax.  Client query parameters should change as
>>> little as possible
>>> #2 - configuration
>>> #3 - java APIs
>> Someone else would likely rank it differently - not everyone using Solr
>> even uses HTTP with it. Someone heavily involved in custom plugins might
>> care more about that than config. As a dev, I just plainly rank them all
>> as important and treat them on a case by case basis.
>
> I think it is fair to suggest that people will have the most
> stable/consistent/seamless upgrade path if you stick to the HTTP API
> (and by extension most of the solrj API)
>
> I am not suggesting that the java APIs are not important and that
> back-compatibly is not important.  Solr has a some APIs with a clear
> purpose, place, and intended use -- we need to take these very
> seriously.  We also have lots of APIs that are half baked and loosy
> goosy.  If a developer is working on the edges, i think it is fair to
> expect more hickups in the upgrade path.
I agree with all of that - I just didn't understand starting a Solr back
compat policy by ranking importance. Does that mean I don't have to
strive for back compat with configuration with the same intensity as the
URL API? I think the current policy of just doing our best and taking
the situations as they come case-by-case works pretty well. When a
backcompat issue comes up with configuration, should I be thinking,
well, its not a URL API syntax, so ... I'd just consider the impact on
users and do my best right? The same should apply to each of the cats.

Lucene has a back compat policy, but the reality is not that policy -
its what I said above - we evaluate the impact on users and do our best,
case-by-case.  The policy isn't really totally in line with the reality.
And Solr has appeared to me at least, to work in the same manner.
>
>
>>>
>>> With that in mind, i think 'solr 1.5 with lucene 3.x' makes the most
>>> sense.  Unless we see making serious changes to solr that would
>>> warrent a major release bump.
>> What is a serious change that would warrant a bump in your opinion?
>
> for example:
> - config overhaul.  detangle the XML from the components.  perhaps
> using spring.
> - major URL request changes.  perhaps we change things to be more
> RESTful -- perhaps let jersey take care of the URL/request building
> https://jersey.dev.java.net/
> - perhaps OSGi support/control/configuration
Okay - was just curious :)
>
>
>>>
>>> Lucene has an explict back-compatibility contract:
>>> http://wiki.apache.org/lucene-java/BackwardsCompatibility
>>>
>>> I don't know if solr has one...  if we make one, I would like it to
>>> focus on the URL syntax+configuration
>> Its not nice to give people plugins and then not worry about back compat
>> for them :)
>
> i want to be nice.  I just think that a different back compatibility
> contract applies for solr then for lucene.  It seems reasonable to
> consider the HTTP API, configs, and java API independently.
>
> From my perspective, saying "solr 1.5 uses lucene 3.0" implies
> everything a plugin developer using lucene APIs needs to know about
> the changes.
>
> To be clear, I am not against bumping to solr 2.0 -- I just have high
> aspirations (yet little time) for what a 2.0 bump could mean for solr.
Lucene works that way too though - case-by-case we determine the affect
on users. Sometimes we say, eh, this won't affect that many, and that
determines a break. I really don't think Lucene operates any different
than Solr in the respect - other than its a bit more conservative (for
good reason I think) - but it certainly doesn't treat everything the
same across the board in terms of back compat. But neither do we
prioritize everything in terms of importance - case-by-case we consider
the impact. Or we just plain screw up and don't notice till too late ;)

switching back to solr-dev... sorry for spinning off that thread...
>>>
>>> What is a serious change that would warrant a bump in your opinion?
>>
>> for example:
>> - config overhaul.  detangle the XML from the components.  perhaps  
>> using
>> spring.
> This is already done. No components read config from xml anymore  
> SOLR-1198

nice work!  I had not seen this.

To be clear, I am not against bumping to solr 2.0 -- I just have high
>> aspirations (yet little time) for what a 2.0 bump could mean for  
>> solr.
>

By the way, I don't disagree with this at all - I don't think now is  
the time to decide this. We don't even know what's going to happen. I  
just think it's interesting to see what comes out of the discussion. I  
did think it was an odd time to have it though - makes more sense when  
we see what actually happens. And *when* we actually end up releasing.

On Nov 19, 2009, at 9:31 AM, Yonik Seeley wrote:

> What should the next version of Solr be?
>
> Options:
>
> - have a Solr 2.0 with a lucene 3.x

+1.  This gives us a chance to remove some deprecated stuff, too.

Grant Ingersoll wrote:
> On Nov 19, 2009, at 9:31 AM, Yonik Seeley wrote:
>
>  
>> What should the next version of Solr be?
>>
>> Options:
>>
>> - have a Solr 2.0 with a lucene 3.x
>>    
>
> +1.  This gives us a chance to remove some deprecated stuff, too.
>
>
>  
What would be the current development branch of Solr 2.0 , if that were
branched already that is ?

In practical terms, calling a release 2.0 means it will never finish.
"One last feature! No, really!" happens with 1.x. A Solr 2.0 will be
killed by "Let's rewrite this!"

http://en.wikipedia.org/wiki/Second-system_effect

On Mon, Nov 23, 2009 at 2:32 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>
> On Nov 19, 2009, at 9:31 AM, Yonik Seeley wrote:
>
>> What should the next version of Solr be?
>>
>> Options:
>>
>> - have a Solr 2.0 with a lucene 3.x
>
> +1.  This gives us a chance to remove some deprecated stuff, too.
>
>

--
Lance Norskog
[EMAIL PROTECTED]

Is that a proposal to never leave 1.x :) I guess numbers do allow it ...

Lance Norskog wrote:
> In practical terms, calling a release 2.0 means it will never finish.
> "One last feature! No, really!" happens with 1.x. A Solr 2.0 will be
> killed by "Let's rewrite this!"
>
> http://en.wikipedia.org/wiki/Second-system_effect
>
> On Mon, Nov 23, 2009 at 2:32 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>  
>> On Nov 19, 2009, at 9:31 AM, Yonik Seeley wrote:
>>
>>    
>>> What should the next version of Solr be?
>>>
>>> Options:
>>>
>>> - have a Solr 2.0 with a lucene 3.x
>>>      
>> +1.  This gives us a chance to remove some deprecated stuff, too.
>>
>>
>>    
>
>
>
>

On Mon, Nov 23, 2009 at 11:56 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
> Is that a proposal to never leave 1.x :) I guess numbers do allow it ...
>
> Lance Norskog wrote:
>> In practical terms, calling a release 2.0 means it will never finish.
>> "One last feature! No, really!" happens with 1.x. A Solr 2.0 will be
>> killed by "Let's rewrite this!"
>>
>> http://en.wikipedia.org/wiki/Second-system_effect

Well, Solr 1.4 is is almost a 2.0 to me already, so a short leap to me ;)
>>
>> On Mon, Nov 23, 2009 at 2:32 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
>>
>>> On Nov 19, 2009, at 9:31 AM, Yonik Seeley wrote:
>>>
>>>
>>>> What should the next version of Solr be?
>>>>
>>>> Options:
>>>>
>>>> - have a Solr 2.0 with a lucene 3.x
>>>>
>>> +1.  This gives us a chance to remove some deprecated stuff, too.
>>>
>>>
>>>
>>
>>
>>
>>
>
>

: regular trunk structure at some point down the road. What's the SOLR
: versioning scheme by the way? Is it:

that's part of the problem (and the reason why comments about the "back
compat" commitments for Solr have come up in this thread) ... Solr is a
young enough project that we've never made any "formal" specification of
what it means to rev a major version, or for that matter if/when we might
have a point release (ie: 1.4.1) ... and we've never really specified what
it means for Solr to be backwards compatible ... we've done a pretty good
job of making config syntax, schema declarations, and URL params backwards
compatible, and whenever feasible we've tried to keep "plugin" APIs back
compatible -- but we have (knowingly) broken them on occasion with the
understanding that our first priority is "user level" back compatibility
and performance, and that plugin writers are probably willing to put up
with needing to make small code changes to upgrade if it means good
performance wins.

that said: i (and i'm guressing several other people) would fight
pretty hard against making major internal API chandes that would require
plugin developers to rewrite serious chunks of code w/o doing a major
version bump.

The point being: it's all been very informat up to now -- and that's
probably for the best.  "policies" should evolve over time based on real
world situations that come up, and we're still in the process of doing
that.
-Hoss

: What should the next version of Solr be?

personally, the way the question is phrased bothers me -- it feels like
the cart leading the horse.

I think the better questions to ask are...

  Q1. should we actively try to upgrade to Lucene 3.x, or should we wait
for some demonstrated need/advantage (ie: new functionality, improved
performance)  

  Q2. If/when Solr trunk is switched to use Lucene 3.x, what should the
subsequent Solr release be called? -- ie should a major version bump in
the Lucene dependency trigger a major version bump in the Solr version?

My opinions would be...

  A1. keep using 2.9.x until 3.x offers something that makes it worth
switching for (but we can always start removing usage of deprecated APIs
that don't require changign our own APIs)

  A2. when we do finally move to Lucene 3.0, it would probably make sense
to bump the Solr version number to 2.0 since we will likely be
changing/breaking a lot of plugin APIs in non trivial ways.  If we somehoe
manage it w/o breaking a lot of existing APIs, then i see no reason to
bump our major version number.

-Hoss

Hey Hoss,

>
> : regular trunk structure at some point down the road. What's the SOLR
> : versioning scheme by the way? Is it:
>
> that's part of the problem (and the reason why comments about the "back
> compat" commitments for Solr have come up in this thread) ... Solr is a
> young enough project that we've never made any "formal" specification of
> what it means to rev a major version, or for that matter if/when we might
> have a point release (ie: 1.4.1) ... and we've never really specified what
> it means for Solr to be backwards compatible ... we've done a pretty good
> job of making config syntax, schema declarations, and URL params backwards
> compatible, and whenever feasible we've tried to keep "plugin" APIs back
> compatible -- but we have (knowingly) broken them on occasion with the
> understanding that our first priority is "user level" back compatibility
> and performance, and that plugin writers are probably willing to put up
> with needing to make small code changes to upgrade if it means good
> performance wins.

Fair enough.

> that said: i (and i'm guressing several other people) would fight
> pretty hard against making major internal API chandes that would require
> plugin developers to rewrite serious chunks of code w/o doing a major
> version bump.

You can add me to that list :)

>
> The point being: it's all been very informat up to now -- and that's
> probably for the best.  "policies" should evolve over time based on real
> world situations that come up, and we're still in the process of doing
> that.

Agreed, but now that the elephant has been identified in the room, let's
come up with a policy then. What are your thoughts on my proposal (specified
earlier in this thread)?

Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Just to toss in my two cents...

I'd have to agree with Hoss here. In terms of versioning, I see no  
reason that a major version bump in a dependency should cause a major  
version bump in Solr - unless said bump causes major changes. I  
haven't really looked at what's planned for Lucene 3.x yet, but if  
there are some major api breaking changes coming, then perhaps the  
next couple 1.x revisions should be taken to start cleaning up and  
preparing for a major version bump. So, I would agree that, unless  
there's a really compelling reason to switch to Lucene 3.x, it might  
be best let a little dust settle on 2.9.

-Colin

On Nov 23, 2009, at 6:48 PM, Chris Hostetter wrote:

>
> : What should the next version of Solr be?
>
> personally, the way the question is phrased bothers me -- it feels  
> like
> the cart leading the horse.
>
> I think the better questions to ask are...
>
>  Q1. should we actively try to upgrade to Lucene 3.x, or should we  
> wait
> for some demonstrated need/advantage (ie: new functionality, improved
> performance)
>
>  Q2. If/when Solr trunk is switched to use Lucene 3.x, what should the
> subsequent Solr release be called? -- ie should a major version bump  
> in
> the Lucene dependency trigger a major version bump in the Solr  
> version?
>
> My opinions would be...
>
>  A1. keep using 2.9.x until 3.x offers something that makes it worth
> switching for (but we can always start removing usage of deprecated  
> APIs
> that don't require changign our own APIs)
>
>  A2. when we do finally move to Lucene 3.0, it would probably make  
> sense
> to bump the Solr version number to 2.0 since we will likely be
> changing/breaking a lot of plugin APIs in non trivial ways.  If we  
> somehoe
> manage it w/o breaking a lot of existing APIs, then i see no reason to
> bump our major version number.
>
> -Hoss
>
>

Active Media Architects, Inc.
World Class Design, Programming & Strategy - Since 1998
http://www.activema.com

1-888-392-4567 toll free
1-586-445-1000 local
1-586-445-2247 fax

On Nov 24, 2009, at 9:07 AM, Colin Hynes wrote:

> Just to toss in my two cents...
>
> I'd have to agree with Hoss here. In terms of versioning, I see no reason that a major version bump in a dependency should cause a major version bump in Solr - unless said bump causes major changes.

It's got some pretty big changes.  

> I haven't really looked at what's planned for Lucene 3.x yet, but if there are some major api breaking changes coming, then perhaps the next couple 1.x revisions should be taken to start cleaning up and preparing for a major version bump. So, I would agree that, unless there's a really compelling reason to switch to Lucene 3.x, it might be best let a little dust settle on 2.9.
I think we are going to want to support flexible indexing within a pretty reasonable time frame after it is available.  Also note, there are some fairly big Solr changes in the pipeline at this point:
1. Spatial support, which will change/add some significant general purpose capabilities to Solr.
2. Significant new distributed capabilities for both indexing and searching
3. And of course a fair number of other things.  

Agreed, though, on the fact that we should cleanup in prep for 2.x, so we may want to shoot for a 1.9 first.

-Grant

On Nov 24, 2009, at 9:22 AM, Grant Ingersoll wrote:
>
> On Nov 24, 2009, at 9:07 AM, Colin Hynes wrote:
>
>> Just to toss in my two cents...
>>
>> I'd have to agree with Hoss here. In terms of versioning, I see no  
>> reason that a major version bump in a dependency should cause a  
>> major version bump in Solr - unless said bump causes major changes.
>
> It's got some pretty big changes.
>
>> I haven't really looked at what's planned for Lucene 3.x yet, but  
>> if there are some major api breaking changes coming, then perhaps  
>> the next couple 1.x revisions should be taken to start cleaning up  
>> and preparing for a major version bump. So, I would agree that,  
>> unless there's a really compelling reason to switch to Lucene 3.x,  
>> it might be best let a little dust settle on 2.9.
>
>
> I think we are going to want to support flexible indexing within a  
> pretty reasonable time frame after it is available.  Also note,  
> there are some fairly big Solr changes in the pipeline at this point:
> 1. Spatial support, which will change/add some significant general  
> purpose capabilities to Solr.
> 2. Significant new distributed capabilities for both indexing and  
> searching
> 3. And of course a fair number of other things.

Ah yes, this is definitely some major stuff.

> Agreed, though, on the fact that we should cleanup in prep for 2.x,  
> so we may want to shoot for a 1.9 first.

Sounds like a plan. :D

-Colin
Active Media Architects, Inc.
World Class Design, Programming & Strategy - Since 1998
http://www.activema.com

1-888-392-4567 toll free
1-586-445-1000 local
1-586-445-2247 fax

: > The point being: it's all been very informat up to now -- and that's
: > probably for the best.  "policies" should evolve over time based on real
: > world situations that come up, and we're still in the process of doing
: > that.
:
: Agreed, but now that the elephant has been identified in the room, let's
: come up with a policy then. What are your thoughts on my proposal (specified
: earlier in this thread)?

The elephant has been identified, and he's in the room, but he's in the
far corner and he's not bothering anybody.

my personal preference (at the moment) is to leave things undefined for
now ... i'd rather see us get to a point where we say "whoa, here is an
actaul in the flesh cross roads where it feels wrong to release w/o
bumping the major version" and then to try and write down a policy ahead
of time anticipating what that cross road is and how we we'll want to deal
with it.

if it's unspecified, it can be specified later when it actaully becomes an
issue -- if it's specified, then people will feel like we are beholden to
the specification, even if it doesn't wind up making as much sense in
practice.

(yonik: you're anarchist ways clearly rubbed off on me at apachecon)

-Hoss

On Nov 25, 2009, at 11:30 AM, Chris Hostetter wrote:

>
> : > The point being: it's all been very informat up to now -- and  
> that's
> : > probably for the best.  "policies" should evolve over time based  
> on real
> : > world situations that come up, and we're still in the process of  
> doing
> : > that.
> :
> : Agreed, but now that the elephant has been identified in the room,  
> let's
> : come up with a policy then. What are your thoughts on my proposal  
> (specified
> : earlier in this thread)?
>
> The elephant has been identified, and he's in the room, but he's in  
> the
> far corner and he's not bothering anybody.
>
> my personal preference (at the moment) is to leave things undefined  
> for
> now ... i'd rather see us get to a point where we say "whoa, here is  
> an
> actaul in the flesh cross roads where it feels wrong to release w/o
> bumping the major version" and then to try and write down a policy  
> ahead
> of time anticipating what that cross road is and how we we'll want  
> to deal
> with it.
>
> if it's unspecified, it can be specified later when it actaully  
> becomes an
> issue -- if it's specified, then people will feel like we are  
> beholden to
> the specification, even if it doesn't wind up making as much sense in
> practice.
>
> (yonik: you're anarchist ways clearly rubbed off on me at apachecon)
>

I agree with keeping it informal (for now)

I agree with Mark that yes, we should do what we can to make the best  
choices about changes that affect compatibility.  For sure.

The thing about the 1.5/1.9/2.0 question that makes me uncomfortable  
is that it is applying the same 'official' rules from lucene to solr.  
I am not sure how well that maps in practice.  What would a 1.9  
release mean in solr?  (I don't really want an answer, I just don't  
think it means the same thing in solr vs lucene)

Again, i have nothing against calling the next release 2.0 -- I just  
think that 1.5 is also fine and keeps the door open for 2.0 to be a  
more fundamental change in solr (that may or may not happen)

ryan

>
> What would a 1.9 release mean in solr?

Dooh -- after hitting send, i realized it would just mean:
Whatever we would do for the next release, but say 'after this, old  
APIs won't be supported'

ryan

: > What would a 1.9 release mean in solr?
:
: Dooh -- after hitting send, i realized it would just mean:
: Whatever we would do for the next release, but say 'after this, old APIs won't
: be supported'

but even that is still a vague statement: are we talking about the
internal/plugin Java APIs, or to the user/HTTP request APIs?

this is why we've never tried to suggest that Solr has the same back
compat policy/concerns as Lucene -- because changes in the Solr Java APIs
aren't as big of a deal between versions as they are in Lucene -- it's
changes in the user level API that we should be the most worried about,
because 95% of the people using Solr don't give a shit about what the java
internals look like.

like i said: i'd rather we just write the code we think we need to write,
and change the code we think w need to change, and when we decide to
release it, we should then ask ourselves "what name/number should we put
on this release to convey how significant the changes are" ... becuase all
the version number really is, is a marketing tool for conveying
information to our users.

-Hoss

Hi Guys,

The process should be as formal as the community dictates, but I can't help but make the observation that increments in version numbers that are strange to those with some knowledge of artifact versioning will only be stranger to those without (i.e., adopters/users of SOLR).

To me, to jump from 1.5 to 2.0 should include at least 5x the differences (improvements, bug fixes, new features, etc.) between SOLR 1.3 and 1.4, or better yet 6x the differences between 1.4 and 2.0. Is this going to be the case? If not, the 0.6 increment in version numbers is arbitrary (why not release the next SOLR as SOLR 10.0 then?). You could argue that if SOLR 2.0 includes a new major version of one of its principal components (Lucene-java), then this warrants those 6-fold differences. Or, if SOLR 2.0 is an entirely different branch of development and then I would expect to see if developed as such and then watch trunk development go on (1.5, 1.6, 1.7) as incremental improvements to 1.4 until the 2.0 branch is ready to be merged back into the 1.x trunk.

On the other hand, I've seen that SOLR is not Lucene and there is a strong sentiment in the SOLR community to that point, so if that's the case, shouldn't the release cycles (and versioning) be a bit more loosely coupled? Insulation is the key. For example with some of the GeoSOLR work going on, I know Grant is looking at how to implement code in SOLR land (e.g., CartesianTierFilter) that exist in Lucene contrib/spatial already, but perhaps SOLR has slightly different use cases or needs for this code, or improvements to make outside of the Lucene release cycle/process. This is just one example: I'm sure there's more.

I'm -1 for writing up a formal 50 page document on versioning and SOLR releases, but a few paragraphs and a formula for how to calculate releases on the SOLR wiki - that's probably going to be something valuable to the community, and more importantly, something that's needed to understand and guage the SOLR software product, so +1, and my encouragement, for that.

My 2 cents,
Chris

On 11/25/09 11:54 AM, "Ryan McKinley" <[EMAIL PROTECTED]> wrote:

On Nov 25, 2009, at 11:30 AM, Chris Hostetter wrote:

>
> : > The point being: it's all been very informat up to now -- and
> that's
> : > probably for the best.  "policies" should evolve over time based
> on real
> : > world situations that come up, and we're still in the process of
> doing
> : > that.
> :
> : Agreed, but now that the elephant has been identified in the room,
> let's
> : come up with a policy then. What are your thoughts on my proposal
> (specified
> : earlier in this thread)?
>
> The elephant has been identified, and he's in the room, but he's in
> the
> far corner and he's not bothering anybody.
>
> my personal preference (at the moment) is to leave things undefined
> for
> now ... i'd rather see us get to a point where we say "whoa, here is
> an
> actaul in the flesh cross roads where it feels wrong to release w/o
> bumping the major version" and then to try and write down a policy
> ahead
> of time anticipating what that cross road is and how we we'll want
> to deal
> with it.
>
> if it's unspecified, it can be specified later when it actaully
> becomes an
> issue -- if it's specified, then people will feel like we are
> beholden to
> the specification, even if it doesn't wind up making as much sense in
> practice.
>
> (yonik: you're anarchist ways clearly rubbed off on me at apachecon)
>

I agree with keeping it informal (for now)

I agree with Mark that yes, we should do what we can to make the best
choices about changes that affect compatibility.  For sure.

The thing about the 1.5/1.9/2.0 question that makes me uncomfortable
is that it is applying the same 'official' rules from lucene to solr.
I am not sure how well that maps in practice.  What would a 1.9
release mean in solr?  (I don't really want an answer, I just don't
think it means the same thing in solr vs lucene)

Again, i have nothing against calling the next release 2.0 -- I just
think that 1.5 is also fine and keeps the door open for 2.0 to be a
more fundamental change in solr (that may or may not happen)

ryan

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Nov 25, 2009, at 3:09 PM, Chris Hostetter wrote:
>
> : > What would a 1.9 release mean in solr?
> :
> : Dooh -- after hitting send, i realized it would just mean:
> : Whatever we would do for the next release, but say 'after this,  
> old APIs won't
> : be supported'
>
> but even that is still a vague statement: are we talking about the
> internal/plugin Java APIs, or to the user/HTTP request APIs?
>
> this is why we've never tried to suggest that Solr has the same back
> compat policy/concerns as Lucene -- because changes in the Solr Java  
> APIs
> aren't as big of a deal between versions as they are in Lucene -- it's
> changes in the user level API that we should be the most worried  
> about,
> because 95% of the people using Solr don't give a shit about what  
> the java
> internals look like.

In my experience software that has multiple APIs like Solr, will drop  
support for old stuff from either or both depending on what they're  
depreciating, so it would really just be dependent on what's getting  
cut. Having said that, I do think the much larger concern is for the  
user-facing API given that this is where the majority sit.

> like i said: i'd rather we just write the code we think we need to  
> write,
> and change the code we think w need to change, and when we decide to
> release it, we should then ask ourselves "what name/number should we  
> put
> on this release to convey how significant the changes are" ...  
> becuase all
> the version number really is, is a marketing tool for conveying
> information to our users.
>
> -Hoss

The versioning could certainly be done on a release by release basis.  
However, I would offer a word of caution about strange jumps in  
version numbers, as this can just serve to confuse the user base  
(someone mentioned this in a previous message that I can't find at the  
moment).
As an example, take a look at many of the major software companies and  
how badly they've handled versioning jumping between numbers, years,  
code names, etcetera. Which brings me to it being used as a marketing  
tool... It's likely these companies felt the same way, and imho  
they've failed to do it effectively (this comes from years of speaking  
to confused customers).
Versioning is really intended as a tool for identifying a particular  
incarnation of an application and hinting at the level of changes.  
While most users will not have the knowledge to really understand all  
the details they do understand that 2.0 > 1.9 > 1.5 > 1.4 and that  
each of these will have differences and that a jump in the first  
number usually indicates significant changes.
Personally, I think it's a bad idea to think of them as only a  
marketing tool.

Just some thoughts, take them as you will.

-Colin
Active Media Architects, Inc.
World Class Design, Programming & Strategy - Since 1998
http://www.activema.com

1-888-392-4567 toll free
1-586-445-1000 local
1-586-445-2247 fax