Neil Hooey
2012-03-02, 20:37
Neil Hooey
2012-03-02, 20:40
Lance Norskog
2012-03-03, 03:00
Mikhail Khludnev
2012-03-03, 10:40
Neil Hooey
2012-03-03, 19:58
Mikhail Khludnev
2012-03-05, 19:54
Michael Ryan
2012-03-05, 21:23
-
How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Neil Hooey 2012-03-02, 20:37
I'm trying to get Solr to run warming queries in parallel with
listener events, but it always does them in sequence, pegging one CPU
while calculating facet counts.
Someone at Lucid Imagination suggested using multiple <listenever
event="firstSearcher"> tags, each with a single facet query in them,
but those are still done in parallel.
Is it possible to run warming queries in parallel, and if so, how?
I'm aware that you could run an external script that forks, but I'd
like to use Solr's native support for this if it exists.
Examples that don't work:
<!-- runs in sequence: multiple facet queries in a single <listener> -->
<query>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
</arr>
</listener>
</query>
<!-- runs in sequence: queries distributed across separate <listener> tags -->
<query>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
</arr>
</listener>
</query>
listener events, but it always does them in sequence, pegging one CPU
while calculating facet counts.
Someone at Lucid Imagination suggested using multiple <listenever
event="firstSearcher"> tags, each with a single facet query in them,
but those are still done in parallel.
Is it possible to run warming queries in parallel, and if so, how?
I'm aware that you could run an external script that forks, but I'd
like to use Solr's native support for this if it exists.
Examples that don't work:
<!-- runs in sequence: multiple facet queries in a single <listener> -->
<query>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
<lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
</arr>
</listener>
</query>
<!-- runs in sequence: queries distributed across separate <listener> tags -->
<query>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
</arr>
</listener>
</query>
+
Neil Hooey 2012-03-02, 20:37
-
Re: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Neil Hooey 2012-03-02, 20:40
> Someone at Lucid Imagination suggested using multiple <listenever
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
I meant to say: "but those are still done in sequence".
On Fri, Mar 2, 2012 at 3:37 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I'm trying to get Solr to run warming queries in parallel with
> listener events, but it always does them in sequence, pegging one CPU
> while calculating facet counts.
>
> Someone at Lucid Imagination suggested using multiple <listenever
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
>
> Is it possible to run warming queries in parallel, and if so, how?
>
> I'm aware that you could run an external script that forks, but I'd
> like to use Solr's native support for this if it exists.
>
> Examples that don't work:
>
> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
> <!-- runs in sequence: queries distributed across separate <listener> tags -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
I meant to say: "but those are still done in sequence".
On Fri, Mar 2, 2012 at 3:37 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I'm trying to get Solr to run warming queries in parallel with
> listener events, but it always does them in sequence, pegging one CPU
> while calculating facet counts.
>
> Someone at Lucid Imagination suggested using multiple <listenever
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
>
> Is it possible to run warming queries in parallel, and if so, how?
>
> I'm aware that you could run an external script that forks, but I'd
> like to use Solr's native support for this if it exists.
>
> Examples that don't work:
>
> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
> <!-- runs in sequence: queries distributed across separate <listener> tags -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
+
Neil Hooey 2012-03-02, 20:40
-
Re: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Lance Norskog 2012-03-03, 03:00
The code does everything in single-threaded mode, but is coded to use
a multi-threaded Java ExecutorService. So, I've filed a request:
https://issues.apache.org/jira/browse/SOLR-3197
On Fri, Mar 2, 2012 at 12:40 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>
> I meant to say: "but those are still done in sequence".
>
>
> On Fri, Mar 2, 2012 at 3:37 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>> I'm trying to get Solr to run warming queries in parallel with
>> listener events, but it always does them in sequence, pegging one CPU
>> while calculating facet counts.
>>
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>>
>> Is it possible to run warming queries in parallel, and if so, how?
>>
>> I'm aware that you could run an external script that forks, but I'd
>> like to use Solr's native support for this if it exists.
>>
>> Examples that don't work:
>>
>> <!-- runs in sequence: multiple facet queries in a single <listener> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>> <!-- runs in sequence: queries distributed across separate <listener> tags -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
--
Lance Norskog
[EMAIL PROTECTED]
a multi-threaded Java ExecutorService. So, I've filed a request:
https://issues.apache.org/jira/browse/SOLR-3197
On Fri, Mar 2, 2012 at 12:40 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>
> I meant to say: "but those are still done in sequence".
>
>
> On Fri, Mar 2, 2012 at 3:37 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>> I'm trying to get Solr to run warming queries in parallel with
>> listener events, but it always does them in sequence, pegging one CPU
>> while calculating facet counts.
>>
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>>
>> Is it possible to run warming queries in parallel, and if so, how?
>>
>> I'm aware that you could run an external script that forks, but I'd
>> like to use Solr's native support for this if it exists.
>>
>> Examples that don't work:
>>
>> <!-- runs in sequence: multiple facet queries in a single <listener> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
>> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>> <!-- runs in sequence: queries distributed across separate <listener> tags -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
--
Lance Norskog
[EMAIL PROTECTED]
+
Lance Norskog 2012-03-03, 03:00
-
Re: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Mikhail Khludnev 2012-03-03, 10:40
Neil,
Would you mind if I ask what particularly do you want to warm by these
queries?
Regards
On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I'm trying to get Solr to run warming queries in parallel with
> listener events, but it always does them in sequence, pegging one CPU
> while calculating facet counts.
>
> Someone at Lucid Imagination suggested using multiple <listenever
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
>
> Is it possible to run warming queries in parallel, and if so, how?
>
> I'm aware that you could run an external script that forks, but I'd
> like to use Solr's native support for this if it exists.
>
> Examples that don't work:
>
> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
> <!-- runs in sequence: queries distributed across separate <listener> tags
> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
--
Sincerely yours
Mikhail Khludnev
Lucid Certified
Apache Lucene/Solr Developer
Grid Dynamics
<http://www.griddynamics.com>
<[EMAIL PROTECTED]>
Would you mind if I ask what particularly do you want to warm by these
queries?
Regards
On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I'm trying to get Solr to run warming queries in parallel with
> listener events, but it always does them in sequence, pegging one CPU
> while calculating facet counts.
>
> Someone at Lucid Imagination suggested using multiple <listenever
> event="firstSearcher"> tags, each with a single facet query in them,
> but those are still done in parallel.
>
> Is it possible to run warming queries in parallel, and if so, how?
>
> I'm aware that you could run an external script that forks, but I'd
> like to use Solr's native support for this if it exists.
>
> Examples that don't work:
>
> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
> <!-- runs in sequence: queries distributed across separate <listener> tags
> -->
> <query>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field1</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field2</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field3</str></lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst><str name="q">*:*</str><str name="facet.field">field4</str></lst>
> </arr>
> </listener>
> </query>
>
--
Sincerely yours
Mikhail Khludnev
Lucid Certified
Apache Lucene/Solr Developer
Grid Dynamics
<http://www.griddynamics.com>
<[EMAIL PROTECTED]>
+
Mikhail Khludnev 2012-03-03, 10:40
-
Re: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Neil Hooey 2012-03-03, 19:58
I need to have those queries trigger the generation of facet counts, which
can take up to 5 minutes for all of them combined.
If the facet counts aren't warmed, then the first query to ask for facet
counts on a particular field will take several minutes to return results.
On Sat, Mar 3, 2012 at 5:40 AM, Mikhail Khludnev <[EMAIL PROTECTED]>
wrote:
> Neil,
>
> Would you mind if I ask what particularly do you want to warm by these
> queries?
>
> Regards
>
> On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>
>> I'm trying to get Solr to run warming queries in parallel with
>> listener events, but it always does them in sequence, pegging one CPU
>> while calculating facet counts.
>>
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>>
>> Is it possible to run warming queries in parallel, and if so, how?
>>
>> I'm aware that you could run an external script that forks, but I'd
>> like to use Solr's native support for this if it exists.
>>
>> Examples that don't work:
>>
>> <!-- runs in sequence: multiple facet queries in a single <listener> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field1</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field2</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field3</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>> <!-- runs in sequence: queries distributed across separate <listener>
tags
>> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field1</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field2</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field3</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Lucid Certified
> Apache Lucene/Solr Developer
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <[EMAIL PROTECTED]>
can take up to 5 minutes for all of them combined.
If the facet counts aren't warmed, then the first query to ask for facet
counts on a particular field will take several minutes to return results.
On Sat, Mar 3, 2012 at 5:40 AM, Mikhail Khludnev <[EMAIL PROTECTED]>
wrote:
> Neil,
>
> Would you mind if I ask what particularly do you want to warm by these
> queries?
>
> Regards
>
> On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
>
>> I'm trying to get Solr to run warming queries in parallel with
>> listener events, but it always does them in sequence, pegging one CPU
>> while calculating facet counts.
>>
>> Someone at Lucid Imagination suggested using multiple <listenever
>> event="firstSearcher"> tags, each with a single facet query in them,
>> but those are still done in parallel.
>>
>> Is it possible to run warming queries in parallel, and if so, how?
>>
>> I'm aware that you could run an external script that forks, but I'd
>> like to use Solr's native support for this if it exists.
>>
>> Examples that don't work:
>>
>> <!-- runs in sequence: multiple facet queries in a single <listener> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field1</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field2</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field3</str></lst>
>> <lst><str name="q">*:*</str><str
name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>> <!-- runs in sequence: queries distributed across separate <listener>
tags
>> -->
>> <query>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field1</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field2</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field3</str></lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst><str name="q">*:*</str><str
name="facet.field">field4</str></lst>
>> </arr>
>> </listener>
>> </query>
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Lucid Certified
> Apache Lucene/Solr Developer
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <[EMAIL PROTECTED]>
+
Neil Hooey 2012-03-03, 19:58
-
Re: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Mikhail Khludnev 2012-03-05, 19:54
Neil,
Still is not clear whether it multi or singe valued fields that
defines usage or FieldCache or UnInvertedField, and per-segment reader vs
top-level reader.
The only concern I have about your approach is the waste of cpu for
calculate facets for huge *:* docsets. I guess you can try to find narrow
way to trigger field cache initialization. Perhaps
http://wiki.apache.org/solr/TermsComponent can be useful for it.
Regards
On Sat, Mar 3, 2012 at 11:58 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I need to have those queries trigger the generation of facet counts, which
> can take up to 5 minutes for all of them combined.
>
> If the facet counts aren't warmed, then the first query to ask for facet
> counts on a particular field will take several minutes to return results.
>
> On Sat, Mar 3, 2012 at 5:40 AM, Mikhail Khludnev <
> [EMAIL PROTECTED]>
> wrote:
> > Neil,
> >
> > Would you mind if I ask what particularly do you want to warm by these
> > queries?
> >
> > Regards
> >
> > On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> >
> >> I'm trying to get Solr to run warming queries in parallel with
> >> listener events, but it always does them in sequence, pegging one CPU
> >> while calculating facet counts.
> >>
> >> Someone at Lucid Imagination suggested using multiple <listenever
> >> event="firstSearcher"> tags, each with a single facet query in them,
> >> but those are still done in parallel.
> >>
> >> Is it possible to run warming queries in parallel, and if so, how?
> >>
> >> I'm aware that you could run an external script that forks, but I'd
> >> like to use Solr's native support for this if it exists.
> >>
> >> Examples that don't work:
> >>
> >> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> >> <query>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field1</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field2</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field3</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field4</str></lst>
> >> </arr>
> >> </listener>
> >> </query>
> >>
> >> <!-- runs in sequence: queries distributed across separate <listener>
> tags
> >> -->
> >> <query>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field1</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field2</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field3</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field4</str></lst>
> >> </arr>
> >> </listener>
> >> </query>
> >>
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> > Lucid Certified
> > Apache Lucene/Solr Developer
> > Grid Dynamics
> >
> > <http://www.griddynamics.com>
> > <[EMAIL PROTECTED]>
>
--
Sincerely yours
Mikhail Khludnev
Lucid Certified
Apache Lucene/Solr Developer
Grid Dynamics
<http://www.griddynamics.com>
<[EMAIL PROTECTED]>
Still is not clear whether it multi or singe valued fields that
defines usage or FieldCache or UnInvertedField, and per-segment reader vs
top-level reader.
The only concern I have about your approach is the waste of cpu for
calculate facets for huge *:* docsets. I guess you can try to find narrow
way to trigger field cache initialization. Perhaps
http://wiki.apache.org/solr/TermsComponent can be useful for it.
Regards
On Sat, Mar 3, 2012 at 11:58 PM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> I need to have those queries trigger the generation of facet counts, which
> can take up to 5 minutes for all of them combined.
>
> If the facet counts aren't warmed, then the first query to ask for facet
> counts on a particular field will take several minutes to return results.
>
> On Sat, Mar 3, 2012 at 5:40 AM, Mikhail Khludnev <
> [EMAIL PROTECTED]>
> wrote:
> > Neil,
> >
> > Would you mind if I ask what particularly do you want to warm by these
> > queries?
> >
> > Regards
> >
> > On Sat, Mar 3, 2012 at 12:37 AM, Neil Hooey <[EMAIL PROTECTED]> wrote:
> >
> >> I'm trying to get Solr to run warming queries in parallel with
> >> listener events, but it always does them in sequence, pegging one CPU
> >> while calculating facet counts.
> >>
> >> Someone at Lucid Imagination suggested using multiple <listenever
> >> event="firstSearcher"> tags, each with a single facet query in them,
> >> but those are still done in parallel.
> >>
> >> Is it possible to run warming queries in parallel, and if so, how?
> >>
> >> I'm aware that you could run an external script that forks, but I'd
> >> like to use Solr's native support for this if it exists.
> >>
> >> Examples that don't work:
> >>
> >> <!-- runs in sequence: multiple facet queries in a single <listener> -->
> >> <query>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field1</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field2</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field3</str></lst>
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field4</str></lst>
> >> </arr>
> >> </listener>
> >> </query>
> >>
> >> <!-- runs in sequence: queries distributed across separate <listener>
> tags
> >> -->
> >> <query>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field1</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field2</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field3</str></lst>
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst><str name="q">*:*</str><str
> name="facet.field">field4</str></lst>
> >> </arr>
> >> </listener>
> >> </query>
> >>
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> > Lucid Certified
> > Apache Lucene/Solr Developer
> > Grid Dynamics
> >
> > <http://www.griddynamics.com>
> > <[EMAIL PROTECTED]>
>
--
Sincerely yours
Mikhail Khludnev
Lucid Certified
Apache Lucene/Solr Developer
Grid Dynamics
<http://www.griddynamics.com>
<[EMAIL PROTECTED]>
+
Mikhail Khludnev 2012-03-05, 19:54
-
RE: How can Solr do parallel query warming with <firstSearcher> and <newSearcher>?Michael Ryan 2012-03-05, 21:23
+
Michael Ryan 2012-03-05, 21:23
