Hi, there,

So we are using the Tomcat's JNDI method to set up multiple solr instances
within a tomcat server. Each instance has a solr home directory.

Now we want to set up collection distribution for all these solr home
indexes. My understanding is:

1.  we only need to run rsync-start once use the script under any of the
solr home dirs.
2.  we need to run each of the rsync-enable scripts under the solr home's
bin dirs.
3.  the twiki page at
http://wiki.apache.org/solr/SolrCollectionDistributionScripts  keeps
refering to solr/xxx. Is this "solr" the example solr home dir?  If so,
would it be hard-coded in any of the scripts?  For example, I saw in
snappuller line 226 (solr 1.2):

${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip

Is the above "solr" a hard-coded solr home name? If so, it's not desirable
since we have multiple solr homes with different names.  If not, what is
this "solr"?
thanks,

-Hui

Ok, I should correct myself. For #1, I think we need to

1) config different port for each solr home dir (since they run on the same
host);
2) run rsync-start script under each of the solr home's bin dir.
(btw, just to make clear, we should run rsync-start after rsync-enable that
I understand.)
Can someone confirm my understanding? Does the #3 question suggests a
hard-coded "solr" that shouldn't be?
Thanks,

-Hui

On 9/19/07, Yu-Hui Jin <[EMAIL PROTECTED]> wrote:
>
> Hi, there,
>
> So we are using the Tomcat's JNDI method to set up multiple solr instances
> within a tomcat server. Each instance has a solr home directory.
>
> Now we want to set up collection distribution for all these solr home
> indexes. My understanding is:
>
> 1.  we only need to run rsync-start once use the script under any of the
> solr home dirs.
> 2.  we need to run each of the rsync-enable scripts under the solr home's
> bin dirs.
> 3.  the twiki page at
> http://wiki.apache.org/solr/SolrCollectionDistributionScripts  keeps
> refering to solr/xxx. Is this "solr" the example solr home dir?  If so,
> would it be hard-coded in any of the scripts?  For example, I saw in
> snappuller line 226 (solr 1.2):
>
> ${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
> ${data_dir}/${name}-wip
>
> Is the above "solr" a hard-coded solr home name? If so, it's not desirable
> since we have multiple solr homes with different names.  If not, what is
> this "solr"?
>
>
> thanks,
>
> -Hui
--
Regards,

-Hui

: 1) config different port for each solr home dir (since they run on the same
: host);

you mean a differnet rsync port right? ... yes the scripts as distributed
assume that each rsync daemon will be dedicated to a single solr
"instance" .. the idea beaing that even if you have 12 Solr intances
running on one servlet container port, you have 12 seperate rsync ports so
you can start/stop enable/disable them independently when doing index
rebuilds, etc...

: 2) run rsync-start script under each of the solr home's bin dir.
: (btw, just to make clear, we should run rsync-start after rsync-enable that
: I understand.)

correct, rsyncd-enable just sets the flag file so that rsyncd-start will
function ... the idea being that you can install rsyncd-start in such a
way that it will run whenever your port is startup, or whenever you box is
booted, but disable it from happening without removing the script from
those places.

: Can someone confirm my understanding? Does the #3 question suggests a
: hard-coded "solr" that shouldn't be?

solr/conf, solr/bin, solr/data, solr/logs ... all assume your solr home
directory is named "solr/", but that's not a requirement.  It's a pretty
pervasive documentation shortcut that could be changed if osmeone wanted
to be systematic about it, but I don't think it's all that bad since
that's a decent "common case"

-Hoss

Thanks, Hoss.

For the last question, yes I understand now it's referring to whatever solr
home we have named.  However, there's still the last part of my question
that feels suspicious why the "solr" string is directly coded in the script
(unlike other cases they usually use ${solr_root} to get to specific dirs.
)   I pasted this line again below:

I saw in snappuller line 226 (solr 1.2):

${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip

Is the above "solr" a hard-coded solr home name? If so, it's not desirable
since we have multiple solr homes with different names.  If not, what is
this "solr"?

Thanks,
-Hui
On 9/20/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
>
>
> : 1) config different port for each solr home dir (since they run on the
> same
> : host);
>
> you mean a differnet rsync port right? ... yes the scripts as distributed
> assume that each rsync daemon will be dedicated to a single solr
> "instance" .. the idea beaing that even if you have 12 Solr intances
> running on one servlet container port, you have 12 seperate rsync ports so
> you can start/stop enable/disable them independently when doing index
> rebuilds, etc...
>
> : 2) run rsync-start script under each of the solr home's bin dir.
> : (btw, just to make clear, we should run rsync-start after rsync-enable
> that
> : I understand.)
>
> correct, rsyncd-enable just sets the flag file so that rsyncd-start will
> function ... the idea being that you can install rsyncd-start in such a
> way that it will run whenever your port is startup, or whenever you box is
> booted, but disable it from happening without removing the script from
> those places.
>
> : Can someone confirm my understanding? Does the #3 question suggests a
> : hard-coded "solr" that shouldn't be?
>
> solr/conf, solr/bin, solr/data, solr/logs ... all assume your solr home
> directory is named "solr/", but that's not a requirement.  It's a pretty
> pervasive documentation shortcut that could be changed if osmeone wanted
> to be systematic about it, but I don't think it's all that bad since
> that's a decent "common case"
>
>
>
> -Hoss
>
>
--
Regards,

-Hui

: home we have named.  However, there's still the last part of my question
: that feels suspicious why the "solr" string is directly coded in the script
: (unlike other cases they usually use ${solr_root} to get to specific dirs.
: )   I pasted this line again below:

sorry ... i didn't realize you were talking about the script, i thought
you were talking aboutthe docs.

: I saw in snappuller line 226 (solr 1.2):
:
: ${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
: ${data_dir}/${name}-wip
:
: Is the above "solr" a hard-coded solr home name? If so, it's not desirable

I'm not 100% positive, but I believe that is just an arbitrary "virtual"
path relative the "root" of the rsyncd server ... it could be anything, as
long as snappuller and the rsyncd agree on what it is, so it's hardcoded
to be "solr".

If we used ${solr_root} then the slaves and the master would have to use
teh exact same solr home directory.
-Hoss

ok. Hoss. I think I'll believe you since nobody raised any issue running the
script.  And I'm about to try it out shortly with different solr home names.

So just to help my knowledge, where does this virtual setting of this "solr"
string happen? Should it be in some config file or sth?
thanks,

-Hui

On 9/20/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
>
> : home we have named.  However, there's still the last part of my question
> : that feels suspicious why the "solr" string is directly coded in the
> script
> : (unlike other cases they usually use ${solr_root} to get to specific
> dirs.
> : )   I pasted this line again below:
>
> sorry ... i didn't realize you were talking about the script, i thought
> you were talking aboutthe docs.
>
> : I saw in snappuller line 226 (solr 1.2):
> :
> : ${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
> : ${data_dir}/${name}-wip
> :
> : Is the above "solr" a hard-coded solr home name? If so, it's not
> desirable
>
> I'm not 100% positive, but I believe that is just an arbitrary "virtual"
> path relative the "root" of the rsyncd server ... it could be anything, as
> long as snappuller and the rsyncd agree on what it is, so it's hardcoded
> to be "solr".
>
> If we used ${solr_root} then the slaves and the master would have to use
> teh exact same solr home directory.
>
>
> -Hoss
>
>
--
Regards,

-Hui

: So just to help my knowledge, where does this virtual setting of this "solr"
: string happen? Should it be in some config file or sth?

rsyncd-start creates an rsync config file on the fly ... much of it is
constants, but it fills in the rsync port using a variable from your
config.
-Hoss

The "solr" that you are referring to in your third question in the
name of the rsync area which is map to the solr data directory.  This
is defined in the rsyncd configuration file which is generated on the
fly as Chris has pointed out.  Take a look at rsyncd-start.

snappuller rsync the index from this 'solr' area (the command you have
quoted) on the master.  The name of the rsync area had nothing to do
with the name of the index.  We set up this area for rsyncd so that
one is restricted within this area when trying to access files on the
master going through rsyncd.

The name of the rsyncd area does not have to be 'solr'.  It can be
anything as long as the value in rsyncd-start matches the value in
snappuller.

Bill

On 9/20/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
>
> : So just to help my knowledge, where does this virtual setting of this "solr"
> : string happen? Should it be in some config file or sth?
>
> rsyncd-start creates an rsync config file on the fly ... much of it is
> constants, but it fills in the rsync port using a variable from your
> config.
>
>
>
>
> -Hoss
>
>

Bill,

Thanks for the explanation. That helps my understanding on rsync and the
replication in general.
regards,

-Hui

On 9/20/07, Bill Au <[EMAIL PROTECTED]> wrote:
>
> The "solr" that you are referring to in your third question in the
> name of the rsync area which is map to the solr data directory.  This
> is defined in the rsyncd configuration file which is generated on the
> fly as Chris has pointed out.  Take a look at rsyncd-start.
>
> snappuller rsync the index from this 'solr' area (the command you have
> quoted) on the master.  The name of the rsync area had nothing to do
> with the name of the index.  We set up this area for rsyncd so that
> one is restricted within this area when trying to access files on the
> master going through rsyncd.
>
> The name of the rsyncd area does not have to be 'solr'.  It can be
> anything as long as the value in rsyncd-start matches the value in
> snappuller.
>
> Bill
>
> On 9/20/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
> >
> > : So just to help my knowledge, where does this virtual setting of this
> "solr"
> > : string happen? Should it be in some config file or sth?
> >
> > rsyncd-start creates an rsync config file on the fly ... much of it is
> > constants, but it fills in the rsync port using a variable from your
> > config.
> >
> >
> >
> >
> > -Hoss
> >
> >
>

--
Regards,

-Hui

You are welcome.

Bill

On 9/21/07, Yu-Hui Jin <[EMAIL PROTECTED]> wrote:
>
> Bill,
>
> Thanks for the explanation. That helps my understanding on rsync and the
> replication in general.
>
>
> regards,
>
> -Hui
>
> On 9/20/07, Bill Au <[EMAIL PROTECTED]> wrote:
> >
> > The "solr" that you are referring to in your third question in the
> > name of the rsync area which is map to the solr data directory.  This
> > is defined in the rsyncd configuration file which is generated on the
> > fly as Chris has pointed out.  Take a look at rsyncd-start.
> >
> > snappuller rsync the index from this 'solr' area (the command you have
> > quoted) on the master.  The name of the rsync area had nothing to do
> > with the name of the index.  We set up this area for rsyncd so that
> > one is restricted within this area when trying to access files on the
> > master going through rsyncd.
> >
> > The name of the rsyncd area does not have to be 'solr'.  It can be
> > anything as long as the value in rsyncd-start matches the value in
> > snappuller.
> >
> > Bill
> >
> > On 9/20/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
> > >
> > > : So just to help my knowledge, where does this virtual setting of
> this
> > "solr"
> > > : string happen? Should it be in some config file or sth?
> > >
> > > rsyncd-start creates an rsync config file on the fly ... much of it is
> > > constants, but it fills in the rsync port using a variable from your
> > > config.
> > >
> > >
> > >
> > >
> > > -Hoss
> > >
> > >
> >
>
>
>
> --
> Regards,
>
> -Hui
>