| clear query|facets|time |
Search criteria: throttle merge.
Results from 1 to 10 from
102 (0.386s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Throttling merges - Lucene - [mail # dev]
|
|
...It may be useful to allow users to throttle merges. A callback that IW passes into SegmentMerger would suffice where individual SM methods make use of the callback. I suppose this could...
|
|
... slow down overall merging by adding a potentially useless method call. However if merging typically consumes IO resources for an extended period of time, this offers a way for the user to tune...
|
|
|
Author: Jason Rutherglen,
2009-07-18, 14:37
|
|
|
Re: Throttling merges - Lucene - [mail # dev]
|
|
... that when openInput/createOutput is called we can optionally specify the "context" (reader, merging, writer, etc.), and then somehow add throttling in there. Mike On Sat, Jul 18, 2009 at 10:37 AM...
|
|
...The goal is to be like "ionice" right? Meaning, lower the priority of IO caused by merging? I agree that makes sense. I wonder if we could implement it in the Directory level, so...
|
|
|
Author: Michael McCandless,
2009-07-18, 16:12
|
|
|
Re: Throttling merges - Lucene - [mail # dev]
|
|
...Yep, pretty much like "ionice" for Linux. At the Directory level makes the most sense with the context passed in. Yeah I'm not sure how exactly that will work yet. On Sat,...
|
|
|
Author: Jason Rutherglen,
2009-07-20, 23:03
|
|
|
Exposing merge throttling in Solr - Lucene - [mail # dev]
|
|
...Hi, Lucene has segment merge throttling, but I think that's not exposed in Solr anywhere, is it? Should I open a JIRA? Otis ...
|
|
|
Author: Otis Gospodnetic,
2013-06-09, 03:08
|
|
|
Re: Exposing merge throttling in Solr - Lucene - [mail # dev]
|
|
...Otis, Solr has had IO throttling since 4.1: Steve On Jun 8, 2013, at 11:08 PM, Otis Gospodnetic wrote: ...
|
|
|
Author: Steve Rowe,
2013-06-09, 03:49
|
|
|
Re: Exposing merge throttling in Solr - Lucene - [mail # dev]
|
|
...Ah, beautiful, missed/forgot about that one despite it being Radim Kolar's request originally. Otis On Sat, Jun 8, 2013 at 11:49 PM, Steve Rowe wrote: ...
|
|
|
Author: Otis Gospodnetic,
2013-06-09, 04:12
|
|
|
[LUCENE-2898] CMS merge throttling is not aggressive enough - Lucene - [issue]
|
|
...I hit this crab while working on the NRT benchmarker (in luceneutil).
CMS today forcefully idles any incoming threads, when there are too many merges pending.
This is the last line...
|
|
... of defense that it has, since it also juggles thread priorities (and forcefully idles the biggest merges) to try to reduce the outstanding merge count.
But when it cannot keep up it has no choice...
|
[+ show more]
[- hide]
| ... but to stall those threads responsible for making new segments.
However, the logic is in the wrong place now – the stalling happens after pulling the next merge from IW. This is poor because... |
| ... it means if you have N indexing threads, you allow max + N outstanding merges.
I have a simple fix, which is to just move the stall logic to before we pull the next merge from IW.... |
|
|
http://issues.apache.org/jira/browse/LUCENE-2898
Author: Michael McCandless,
2011-03-30, 16:50
|
|
|
Re: [jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling - Lucene - [mail # dev]
|
|
...Jack, yes re ownership. Mind opening an issue and reattaching? I saw this was done to 2 issues. Thanks! Otis SOLR Performance Monitoring - http://sematext.com/spm On Dec 8,...
|
|
|
Author: Otis Gospodnetic,
2012-12-08, 18:23
|
|
|
Re: IO bandwidth throttling - Lucene - [mail # user]
|
|
... see maxMergeDocs, but not maxBufferedDocs with the IndexWriter. Otis, is there really some parameter with that name? I think maxMergeDocs may not work well here because the size...
|
|
|
Author: Gopikrishnan Subramani,
2005-08-02, 20:27
|
|
|
Re: setSegmentsPerTier >= setMaxMergeAtOnce ? - Lucene - [mail # user]
|
|
... to bytes read-ahead, whether you have an SSD, whether you throttle merge rate (RateLimitedDirWrapper), etc. Mike McCandless http://blog.mikemccandless.com On Sun, Jun 9, 2013 at 7:08 AM, Boaz...
|
|
...Hi Boaz, That's correct! But what is "too big" of a merge is an app-level decision / requires testing in the "real" context / depends on things like how much free RAM the OS can dedicate...
|
|
|
Author: Michael McCandless,
2013-06-09, 12:31
|
|
|
|
