Home | About | Sematext search-lucene.com search-hadoop.com
 Search Lucene and all its subprojects:

Switch to Threaded View
Solr, mail # user - Determining master/slave from ZK in SolrCloud


Copy link to this message
-
Re: Determining master/slave from ZK in SolrCloud
Jamie Johnson 2011-10-04, 16:45
So I see this JIRA which references roles

https://issues.apache.org/jira/browse/SOLR-2765

I'm looking at implementing what Yonik suggested, namely in
solrconfig.xml I have something like

    <core name="${coreName}" instanceDir="." shard="${shard}"
collection="${collection}" roles="searcher,indexer"/>

these will be pulled out and added to the cloudDescriptor so they can
be saved in ZK.  Seem reasonable?

On Tue, Oct 4, 2011 at 12:17 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
> also as an FYI I created this JIRA
>
> https://issues.apache.org/jira/browse/SOLR-2811
>
> which perhaps should be removed if the roles option comes to life.  Is
> there a JIRA on that now?
>
> On Tue, Oct 4, 2011 at 12:12 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
>> Thanks for the reply Mark.  So a couple of questions.
>>
>> When is distributed indexing going to be available on Trunk?  Are
>> there docs on it now?
>>
>> I think having Roles on the shard would scratch the itch here, because
>> as you said I could then include a role which indicated what to do
>> with this server.
>>
>> My use case is actually for something outside of Solr.  As of right
>> now we are not on the latest trunk (actually a few months back I
>> think) but could push to upgrade to it if the distributed indexing
>> code was available today, but management may still shoot that down
>> because of a short timeline.  Suffice it to say that I'll be reading
>> this information by another application to handle distributed indexing
>> externally.  The version that I'm working on requires that the
>> application be responsible for performing the distribution.
>>
>>
>> On Tue, Oct 4, 2011 at 12:05 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>> Because the distributed indexing phase of SolrCloud will not use replication, we have not really gone down this path at all.
>>>
>>> One thing we are considering is adding the ability to add various roles to each shard as hints - eg a shard might be designated a searcher and another an indexer.
>>>
>>> You might be able to piggy back on this to label things master/slave. A ZooKeeper aware replication handler could then use this information.
>>>
>>> There is nothing to stop you from adding this information to zookeeper yourself, using standard zookeeper tools - but just putting the information is only half the problem - something then needs to read it.
>>>
>>> - Mark Miller
>>> lucidimagination.com
>>> 2011.lucene-eurocon.org | Oct 17-20 | Barcelona
>>>
>>> On Oct 4, 2011, at 10:26 AM, Jamie Johnson wrote:
>>>
>>>> Ok, so I am pretty sure this information is not available.  What is
>>>> the most appropriate way to add information like this to ZK?  I can
>>>> obviously look for the system properties enable.master and
>>>> enable.slave, but that won't be fool proof since someone could put
>>>> this in the config file instead and not as a system property.  Is
>>>> there a way to determine this quickly programatically without having
>>>> to go through all of the request handlers in the solrCore?
>>>>
>>>> On Mon, Oct 3, 2011 at 5:52 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
>>>>> Is it possible to determine if a solr instance is a master or a slave
>>>>> in replication terms based on the information that is placed in ZK in
>>>>> SolrCloud?
>>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>