Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Lucene and all its subprojects:

Switch to Threaded View
Solr >> mail # user >> Determining master/slave from ZK in SolrCloud


Copy link to this message
-
Re: Determining master/slave from ZK in SolrCloud
So I see this JIRA which references roles

https://issues.apache.org/jira/browse/SOLR-2765

I'm looking at implementing what Yonik suggested, namely in
solrconfig.xml I have something like

    <core name="${coreName}" instanceDir="." shard="${shard}"
collection="${collection}" roles="searcher,indexer"/>

these will be pulled out and added to the cloudDescriptor so they can
be saved in ZK.  Seem reasonable?

On Tue, Oct 4, 2011 at 12:17 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
> also as an FYI I created this JIRA
>
> https://issues.apache.org/jira/browse/SOLR-2811
>
> which perhaps should be removed if the roles option comes to life.  Is
> there a JIRA on that now?
>
> On Tue, Oct 4, 2011 at 12:12 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
>> Thanks for the reply Mark.  So a couple of questions.
>>
>> When is distributed indexing going to be available on Trunk?  Are
>> there docs on it now?
>>
>> I think having Roles on the shard would scratch the itch here, because
>> as you said I could then include a role which indicated what to do
>> with this server.
>>
>> My use case is actually for something outside of Solr.  As of right
>> now we are not on the latest trunk (actually a few months back I
>> think) but could push to upgrade to it if the distributed indexing
>> code was available today, but management may still shoot that down
>> because of a short timeline.  Suffice it to say that I'll be reading
>> this information by another application to handle distributed indexing
>> externally.  The version that I'm working on requires that the
>> application be responsible for performing the distribution.
>>
>>
>> On Tue, Oct 4, 2011 at 12:05 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
>>> Because the distributed indexing phase of SolrCloud will not use replication, we have not really gone down this path at all.
>>>
>>> One thing we are considering is adding the ability to add various roles to each shard as hints - eg a shard might be designated a searcher and another an indexer.
>>>
>>> You might be able to piggy back on this to label things master/slave. A ZooKeeper aware replication handler could then use this information.
>>>
>>> There is nothing to stop you from adding this information to zookeeper yourself, using standard zookeeper tools - but just putting the information is only half the problem - something then needs to read it.
>>>
>>> - Mark Miller
>>> lucidimagination.com
>>> 2011.lucene-eurocon.org | Oct 17-20 | Barcelona
>>>
>>> On Oct 4, 2011, at 10:26 AM, Jamie Johnson wrote:
>>>
>>>> Ok, so I am pretty sure this information is not available.  What is
>>>> the most appropriate way to add information like this to ZK?  I can
>>>> obviously look for the system properties enable.master and
>>>> enable.slave, but that won't be fool proof since someone could put
>>>> this in the config file instead and not as a system property.  Is
>>>> there a way to determine this quickly programatically without having
>>>> to go through all of the request handlers in the solrCore?
>>>>
>>>> On Mon, Oct 3, 2011 at 5:52 PM, Jamie Johnson <[EMAIL PROTECTED]> wrote:
>>>>> Is it possible to determine if a solr instance is a master or a slave
>>>>> in replication terms based on the information that is placed in ZK in
>>>>> SolrCloud?
>>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB