Any objections to fixing top level "ant test" to simply run tests...?

Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------

On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
> Any objections to fixing top level "ant test" to simply run tests...?
>
> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...

+1

Steve

---------------------------------------------------------------------

+1, we have caged the rat, we should be able to have a simple precommit check.

also top-level 'ant clean' shouldn't call clean-jars.

This *totally messes up* my IDE just because I like to run tests from
the command-line.

On Thu, Jul 19, 2012 at 12:40 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
> On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
>> Any objections to fixing top level "ant test" to simply run tests...?
>>
>> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...
>
> +1
>
> Steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

--
lucidimagination.com

---------------------------------------------------------------------

> also top-level 'ant clean' shouldn't call clean-jars.

I think it should restore pristine state so it should remove those
jars. I specifically changed it to do so because otherwise the
sequence "ant clean test" depends on your history and current folder
contents. I don't like it and there were other people that expressed
confusion about it, especially when a new JAR is resolved and the old
one stays in place.

> This *totally messes up* my IDE just because I like to run tests from
> the command-line.

I use the same thing and it doesn't do anything for me, really.
They're replaced right away with identical JARs and Eclipse refreshes
itself. What does it do for you?

Dawid

---------------------------------------------------------------------

On Thu, Jul 19, 2012 at 1:54 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:

>
> I use the same thing and it doesn't do anything for me, really.
> They're replaced right away with identical JARs and Eclipse refreshes
> itself. What does it do for you?
>

it breaks my ide completely and causes the project to fail. clean
calls clean-jars
and now jars that my project depends on are gone.

often times i just call clean before i rerun a single test or
something, its just a habit
because ive been bit before when fixing bugs in X but running the test
in Y where Y depends on X
that ant/javac/whatever didnt figure out shit needed to be recompiled.

so its really frustrating at the moment.
If its really mandatory to keep this, we should make this a system
property (defaulted to off)

--
lucidimagination.com

---------------------------------------------------------------------

> it breaks my ide completely and causes the project to fail. clean

Are you talking about Eclipse? I don't know, I've just done the above
and nothing breaks for me.

> often times i just call clean before i rerun a single test or
> something, its just a habit

That's a good habit. And exactly the same reason for syncing the jars
used for compilation and execution with the ivy files -- so that you
don't run with stuff you're not supposed to run with?

> so its really frustrating at the moment.

I agree we should change it if it breaks IDEs, it doesn't do it for me
so I didn't think it's the case.

> If its really mandatory to keep this, we should make this a system
> property (defaulted to off)

Backtracking a bit -- ideally we should use a classpath built directly
from those ivy files... then it wouldn't matter. But then things would
get complicated for other IDEs anyway (because not all of them support
ivy as far as I can tell?).

D.

On Thu, Jul 19, 2012 at 2:11 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
>> it breaks my ide completely and causes the project to fail. clean
>
> Are you talking about Eclipse? I don't know, I've just done the above
> and nothing breaks for me.

The editor appears to work at first:
But it definitely breaks as soon as it needs to compile anything
(which happens constantly as you type), or god forbid refresh the
project for any reason (as those jars are now deleted).

>
>> often times i just call clean before i rerun a single test or
>> something, its just a habit
>
> That's a good habit. And exactly the same reason for syncing the jars
> used for compilation and execution with the ivy files -- so that you
> don't run with stuff you're not supposed to run with?

I dont svn up arbitrarily or often for any reason.
when i am working on something, i work with that revision until i'm happy.
then i svn up and merge any changes from trunk (when i do svn up, i clean-jars).

I've never had any issues with stale jars... on the other hand my IDE is broken.

> Backtracking a bit -- ideally we should use a classpath built directly
> from those ivy files... then it wouldn't matter. But then things would
> get complicated for other IDEs anyway (because not all of them support
> ivy as far as I can tell?).

I agree with what you say there.

My point is that its now totally broken:
1. i dont like top level targets that do different shit than the ones
under lucene/ and solr/. This makes the build totally inconsistent and
confusing.
2. i dont like targets like test calling unrelated targets (add
test-all or precommit if you want this), or clean calling unrelated
shit for me. add 'clean-all' if you want this!
3. I don't like targets that delete anything except whats in my build/
directory. The other stuff could be important: to me or to my IDE.

So please, please, please, lets remove this dependency.

--
lucidimagination.com

---------------------------------------------------------------------

> shit for me. add 'clean-all' if you want this!

We see it differently, to me a 'clean' should restore pretty much
checkout state. No need for aliasing targets to do a clean/ cleaner/
cleanest clean. What's broken is not clean, it's the fact that we need
to copy and sync those damn jars.

> So please, please, please, lets remove this dependency.

Sure, remove it. I can live with "git clean -xfd".

Dawid

---------------------------------------------------------------------

On Thu, Jul 19, 2012 at 2:25 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
>> shit for me. add 'clean-all' if you want this!
>
> We see it differently, to me a 'clean' should restore pretty much
> checkout state. No need for aliasing targets to do a clean/ cleaner/
> cleanest clean. What's broken is not clean, it's the fact that we need
> to copy and sync those damn jars.

I guess what gets me is i get used to a certain compile-test-debug cycle:

1. run tests
2. look at which ones failed and pick the "easiest" to understand.
3. hack on things until i think its fixed
4. run that test again, if it passes GOTO 1
5. if it fails, run ant clean # clean here because i dont trust the
compiler i(e.g. changed a static value or whatever), then run it
again.
6. if it still fails, get another beer and pull my hair out and go
back to hacking on things, then back to 1

(optional) for some changes, once tests pass:
7. ant clean this checkout and another one # clean here to be safe:
because benchmarks are painfully slow
8. run benchmarks (luceneutil, whatever)
9. depending on results, make more changes and go back to test lifecycle above.

once i am happy with that:
10. run ant clean javadocs-lint # clean here because otherwise i might
have stale javadocs from classes i deleted
11. if this fails, fix javadocs warnings or any other brokenness, GOTO 1

before i commit or upload patch or whatever:
12. run validation or whatever, in no way does it need to be involved
in anything above.
13. search for nocommits, svn diff and review the patch, look for stupid stuff.

as you see 'ant clean' sits in here in at least 2 places, sometimes 3,
each time inside a debugging loop.
so its annoying if i also have to fix my IDE all these times too
(which includes recrawling the entire tree each time for 'ant eclipse'
PLUS refreshing my workspace which usually forces a painful eclipse
rebuild) while I'm trying to debug various things.

its also unnecessary to crawl the entire source tree in any of those
debugging loops doing validation checking things like licenses
when its unrelated to what I'm doing, I check that a single time when
I'm ready to commit.

--
lucidimagination.com

---------------------------------------------------------------------

: so its really frustrating at the moment.
: If its really mandatory to keep this, we should make this a system
: property (defaulted to off)

this seems like the sanest solution to satisy most people -- but i don't
think it should default to "off" ... "ant clean" should clean up all the
jars by default.  novice contributors who aren't overly familiar with the
build system and may not even be aware of when/how we are using ivy should be
able to run "svn update && ant clean test" w/o needing to understand that
*sometimes* you have to run "ant clean-jars"  or go out of their way to
set a system property.
-Hoss

---------------------------------------------------------------------

> as you see 'ant clean' sits in here in at least 2 places, sometimes 3,
> each time inside a debugging loop.
> so its annoying if i also have to fix my IDE all these times too

Like I said -- it is unacceptable to be breaking IDEs, revert it, I
wasn't aware it would do it.

I still think my point holds (that the IDE could be configured to work
directly with ivy cache since these jars should be immutable) but I
don't have time to investigate right now and I agree with you that it
must be super annoying.

D.

---------------------------------------------------------------------

I don't know if "depends" clauses can be evaluated at the same time as
properties? This would indicate that it's the case but I never used
it:

http://ant.apache.org/manual/properties.html#if+unless

Dawid

---------------------------------------------------------------------

+1, no clean-jars please until we use ivy:cachepath everywhere and supply "ant eclipse" to setup IvyDE for me. I am currently investigating using IvyDE Eclipse plugin to automatically resolve from inside Eclipse automatically. I use it in another project. You just add the ivy.xml files to your build classpath and it automatically does everything for you. No lib folders needed.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Robert Muir [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, July 19, 2012 6:46 PM
> To: [EMAIL PROTECTED]
> Subject: Re: top level "ant test" shouldn't validate?
>
> +1, we have caged the rat, we should be able to have a simple precommit
> check.
>
> also top-level 'ant clean' shouldn't call clean-jars.
>
> This *totally messes up* my IDE just because I like to run tests from the
> command-line.
>
> On Thu, Jul 19, 2012 at 12:40 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
> > On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
> >> Any objections to fixing top level "ant test" to simply run tests...?
> >>
> >> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint,
> ...
> >
> > +1
> >
> > Steve
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > additional commands, e-mail: [EMAIL PROTECTED]
> >
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

On Thu, Jul 19, 2012 at 2:44 PM, Chris Hostetter
<[EMAIL PROTECTED]> wrote:
>
> : so its really frustrating at the moment.
> : If its really mandatory to keep this, we should make this a system
> : property (defaulted to off)
>
> this seems like the sanest solution to satisy most people -- but i don't
> think it should default to "off" ... "ant clean" should clean up all the
> jars by default.  novice contributors who aren't overly familiar with the
> build system and may not even be aware of when/how we are using ivy should be
> able to run "svn update && ant clean test" w/o needing to understand that
> *sometimes* you have to run "ant clean-jars"  or go out of their way to
> set a system property.

How does this help novice users? Novice users are just as likely to go
to lucene/ or solr, and only run ant clean test from there. I imagine
solr contributors don't generally run the lucene tests... why would
they if they arent changing it? If they do this, no clean-jars will
happen when they run clean.

But working on lucene its a must to run all of them, because you might
break something in solr. This is why i got used to running this stuff
from the top level.

--
lucidimagination.com

---------------------------------------------------------------------

: I don't know if "depends" clauses can be evaluated at the same time as
: properties? This would indicate that it's the case but I never used
: it:
:
: http://ant.apache.org/manual/properties.html#if+unless

if/unless on targets can absolutely be based on properties specified at
runtime or in build.user.properties ... this is how the clover stuff use
to work.

-Hoss

---------------------------------------------------------------------

We are using that all the time in lots of tasks.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Thursday, July 19, 2012 8:48 PM
> To: [EMAIL PROTECTED]
> Subject: Re: top level "ant test" shouldn't validate?
>
> I don't know if "depends" clauses can be evaluated at the same time as
> properties? This would indicate that it's the case but I never used
> it:
>
> http://ant.apache.org/manual/properties.html#if+unless
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

Exactly! I fixed clover last weekend and now you can do "ant
-Drun.clover=true clean test" without clover installed :-) It's all coming
from IVY :-)

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Chris Hostetter [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, July 19, 2012 8:51 PM
> To: [EMAIL PROTECTED]
> Subject: Re: top level "ant test" shouldn't validate?
>
> : I don't know if "depends" clauses can be evaluated at the same time as
> : properties? This would indicate that it's the case but I never used
> : it:
> :
> : http://ant.apache.org/manual/properties.html#if+unless
>
> if/unless on targets can absolutely be based on properties specified at
runtime
> or in build.user.properties ... this is how the clover stuff use to work.
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

I have no problem adding clean.should.not.break.my.ide.nor.be.slow

but i would like to default this to true, because its more annoying to
break the IDEs of new contributors than to hand-hold people that are
constantly svn-upping without paying attention.

On Thu, Jul 19, 2012 at 2:51 PM, Chris Hostetter
<[EMAIL PROTECTED]> wrote:
> : I don't know if "depends" clauses can be evaluated at the same time as
> : properties? This would indicate that it's the case but I never used
> : it:
> :
> : http://ant.apache.org/manual/properties.html#if+unless
>
> if/unless on targets can absolutely be based on properties specified at
> runtime or in build.user.properties ... this is how the clover stuff use
> to work.
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

--
lucidimagination.com

---------------------------------------------------------------------

> We are using that all the time in lots of tasks.

I was thinking about the new "dynamic" properties that are resolved/
computed at runtime, not about "present/not present" properties. So
something like this:

<target ... if="${onOrOffProperty}">

This has the advantage that you can specify a boolean value. So:

ant -Dclover.enabled=false

wouldn't run this target:

  <target name="clover.setup" if="clover.enabled">

and I think it currently does. As far as I can tell this feature isn't
used at all in Lucene build files?

Dawid

---------------------------------------------------------------------

As we are already talking about properties. How about the following:

We add an <available file="..."/> task to populate a property "ide.used.foobar". This just checks if e.g. Eclipse is enabled (somebody ran ant eclipse before). If this property is true, we don’t run ant clean-jars on clen. Very simply, very efficient.

The issue here is to remove validate from root ant test. This is easy and we can commit asap.

One thing: In my opinion, the root build.xml file should not conatin any logic at all. It should just delegate all targets called down to lucene and solr's build.xml. If I call ant prepare-release it should do this for both projects. The same for ant or validate. No magic on top-level, just plan delegation. Maybe we can do that automatically, so we don’t have to declare all targets and subant them. Maybe some ant default target trick like <target name="*">, I have no idea, that would be cool.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Thursday, July 19, 2012 9:10 PM
> To: [EMAIL PROTECTED]
> Subject: Re: top level "ant test" shouldn't validate?
>
> > We are using that all the time in lots of tasks.
>
> I was thinking about the new "dynamic" properties that are resolved/ computed
> at runtime, not about "present/not present" properties. So something like this:
>
> <target ... if="${onOrOffProperty}">
>
> This has the advantage that you can specify a boolean value. So:
>
> ant -Dclover.enabled=false
>
> wouldn't run this target:
>
>   <target name="clover.setup" if="clover.enabled">
>
> and I think it currently does. As far as I can tell this feature isn't used at all in
> Lucene build files?
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

On Thu, Jul 19, 2012 at 4:42 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> The issue here is to remove validate from root ant test. This is easy and we can commit asap.

its the same exact issue though, its not unrelated at all.

ant test does extra unrelated stuff: [validate]
ant clean does extra unrelated stuff: [clean-jars]

we should fix both!

>
> One thing: In my opinion, the root build.xml file should not conatin any logic at all. It should just delegate all targets called down to lucene and solr's build.xml. If I call ant prepare-release it should do this for both projects. The same for ant or validate. No magic on top-level, just plan delegation. Maybe we can do that automatically, so we don’t have to declare all targets and subant them. Maybe some ant default target trick like <target name="*">, I have no idea, that would be cool.
>

+1

--
lucidimagination.com

---------------------------------------------------------------------

Top level any clean breaks my IDE too! I don't know the fine points of this conversation, but it's super painful and I never call top level ant clean anymore. I kept meaning to look into why it was killing me but never got to it.

Sent from my iPhone

On Jul 19, 2012, at 12:46 PM, Robert Muir <[EMAIL PROTECTED]> wrote:

> +1, we have caged the rat, we should be able to have a simple precommit check.
>
> also top-level 'ant clean' shouldn't call clean-jars.
>
> This *totally messes up* my IDE just because I like to run tests from
> the command-line.
>
> On Thu, Jul 19, 2012 at 12:40 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
>> On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
>>> Any objections to fixing top level "ant test" to simply run tests...?
>>>
>>> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...
>>
>> +1
>>
>> Steve
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

+1 on the original question...

IntelliJ doesn't seem to have the problem, I run "ant clean" from the
top level all the time and my projects that depend on it seem to work
fine.

I vaguely remember in Eclipse having to do something like a project
refresh to get things back in synch, but that may be unrelated....
On Thu, Jul 19, 2012 at 8:56 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
> Top level any clean breaks my IDE too! I don't know the fine points of this conversation, but it's super painful and I never call top level ant clean anymore. I kept meaning to look into why it was killing me but never got to it.
>
> Sent from my iPhone
>
> On Jul 19, 2012, at 12:46 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>> +1, we have caged the rat, we should be able to have a simple precommit check.
>>
>> also top-level 'ant clean' shouldn't call clean-jars.
>>
>> This *totally messes up* my IDE just because I like to run tests from
>> the command-line.
>>
>> On Thu, Jul 19, 2012 at 12:40 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
>>> On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
>>>> Any objections to fixing top level "ant test" to simply run tests...?
>>>>
>>>> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...
>>>
>>> +1
>>>
>>> Steve
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>
>>
>>
>> --
>> lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

That makes two of us. I'm gonna disable this as it just broke my IDE
just now again and I'm over the edge!

On Thu, Jul 19, 2012 at 8:56 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
> Top level any clean breaks my IDE too! I don't know the fine points of this conversation, but it's super painful and I never call top level ant clean anymore. I kept meaning to look into why it was killing me but never got to it.
>
> Sent from my iPhone
>
> On Jul 19, 2012, at 12:46 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
>
>> +1, we have caged the rat, we should be able to have a simple precommit check.
>>
>> also top-level 'ant clean' shouldn't call clean-jars.
>>
>> This *totally messes up* my IDE just because I like to run tests from
>> the command-line.
>>
>> On Thu, Jul 19, 2012 at 12:40 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote:
>>> On 7/19/2012 at 12:35 PM, Michael McCandless wrote:
>>>> Any objections to fixing top level "ant test" to simply run tests...?
>>>>
>>>> Maybe we can add a "precommit" target to run tests, validate, javadocs-lint, ...
>>>
>>> +1
>>>
>>> Steve
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>
>>
>>
>> --
>> lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

--
lucidimagination.com

---------------------------------------------------------------------