Trevor Watson
2011-11-29, 17:11
Christopher Currens
2011-11-29, 17:16
Trevor Watson
2011-11-29, 19:59
Trevor Watson
2011-11-30, 16:14
Prescott Nasser
2011-11-30, 16:25
Christopher Currens
2011-11-30, 17:26
Digy
2011-11-30, 18:27
Granroth, Neal V.
2011-11-30, 19:25
Digy
2011-11-30, 20:04
Prescott Nasser
2011-11-30, 20:35
Granroth, Neal V.
2011-11-30, 20:41
Christopher Currens
2011-11-30, 21:33
Digy
2011-11-30, 22:09
Digy
2011-11-30, 22:26
-
[Lucene.Net] Re: Memory Leak in 2.9.2.2Trevor Watson 2011-11-29, 17:11
Sorry for the duplicate post. I was on the road and posted both via my web
mail and office mail by mistake
The increase is a very gradual, the program starts at about 160,000k
according to task manager (I know that's not entirely accurate, but it was
the best I had at the time) and would, after adding 25,000-40,000 result in
an out of memory exception (800,000k according to taskmanager). I tried
building a copy of 2.9.4 to test, but could not find one that worked in
visual studio 2005
I did notice using Ants memory profiler that there were a number of
byte[32789] arrays that I didn't know where they came from in memory.
On Monday, November 28, 2011, Christopher Currens <[EMAIL PROTECTED]>
wrote:
> Hi Trevor,
>
> What kind of memory increase are we talking about? Also, how big are the
> documents that you are indexing, the ones returned from getFileInfoDoc()?
> Is it putting an entire file into the index? Pre 2.9.3 versions had
> issues with holding onto allocated byte arrays far beyond when they were
> used. The memory could only be freed via closing the IndexWriter.
>
> I'm a little unclear on exactly what's happening. Are you noticing memory
> spike and stay constant at that level or is it a gradual increase? Is it
> causing your application to error, (ie OutOfMemory exception, etc)?
>
>
> Thanks,
> Christopher
>
> On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
>> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005 (.NET
>> 2.0) environment. We had a piece of software that WAS working. I'm not
>> sure what has changed however, the following code results in a memory
leak
>> in the Lucene.Net component (or a failure to clean up used memory).
>>
>> The code in issue is here:
>>
>> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
>> delayCommit, string sDataPath)
>> {
>> Document doc = getFileInfoDoc(sDataPath);
>> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
>> if (this.FileID == 0)
>> {
>> string s = "";
>> }
>> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
>> this.fileID.ToString("000000000")), doc, analyzer);
>>
>> analyzer = null;
>> doc = null;
>> if (!delayCommit)
>> iw.Commit();
>> }
>>
>> Commenting out the line iw.UpdateDocument resulted in no memory increase.
>> I also tried replacing it with a deleteDocument and AddDocument and the
>> memory increased the same as using the UpdateDocument function
>>
>> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but it's
the
>> UpdateDocument line specifically that gives me the issue.
>>
>> Any assistance would be greatly appreciated.
>>
>> Trevor Watson
>>
>
mail and office mail by mistake
The increase is a very gradual, the program starts at about 160,000k
according to task manager (I know that's not entirely accurate, but it was
the best I had at the time) and would, after adding 25,000-40,000 result in
an out of memory exception (800,000k according to taskmanager). I tried
building a copy of 2.9.4 to test, but could not find one that worked in
visual studio 2005
I did notice using Ants memory profiler that there were a number of
byte[32789] arrays that I didn't know where they came from in memory.
On Monday, November 28, 2011, Christopher Currens <[EMAIL PROTECTED]>
wrote:
> Hi Trevor,
>
> What kind of memory increase are we talking about? Also, how big are the
> documents that you are indexing, the ones returned from getFileInfoDoc()?
> Is it putting an entire file into the index? Pre 2.9.3 versions had
> issues with holding onto allocated byte arrays far beyond when they were
> used. The memory could only be freed via closing the IndexWriter.
>
> I'm a little unclear on exactly what's happening. Are you noticing memory
> spike and stay constant at that level or is it a gradual increase? Is it
> causing your application to error, (ie OutOfMemory exception, etc)?
>
>
> Thanks,
> Christopher
>
> On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
>> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005 (.NET
>> 2.0) environment. We had a piece of software that WAS working. I'm not
>> sure what has changed however, the following code results in a memory
leak
>> in the Lucene.Net component (or a failure to clean up used memory).
>>
>> The code in issue is here:
>>
>> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
>> delayCommit, string sDataPath)
>> {
>> Document doc = getFileInfoDoc(sDataPath);
>> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
>> if (this.FileID == 0)
>> {
>> string s = "";
>> }
>> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
>> this.fileID.ToString("000000000")), doc, analyzer);
>>
>> analyzer = null;
>> doc = null;
>> if (!delayCommit)
>> iw.Commit();
>> }
>>
>> Commenting out the line iw.UpdateDocument resulted in no memory increase.
>> I also tried replacing it with a deleteDocument and AddDocument and the
>> memory increased the same as using the UpdateDocument function
>>
>> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but it's
the
>> UpdateDocument line specifically that gives me the issue.
>>
>> Any assistance would be greatly appreciated.
>>
>> Trevor Watson
>>
>
+
Trevor Watson 2011-11-29, 17:11
-
Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2Christopher Currens 2011-11-29, 17:16
Do you know how big the documents are that you are trying to delete/update?
I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
Thanks,
Christopher
On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> Sorry for the duplicate post. I was on the road and posted both via my web
> mail and office mail by mistake
>
> The increase is a very gradual, the program starts at about 160,000k
> according to task manager (I know that's not entirely accurate, but it was
> the best I had at the time) and would, after adding 25,000-40,000 result in
> an out of memory exception (800,000k according to taskmanager). I tried
> building a copy of 2.9.4 to test, but could not find one that worked in
> visual studio 2005
>
> I did notice using Ants memory profiler that there were a number of
> byte[32789] arrays that I didn't know where they came from in memory.
>
> On Monday, November 28, 2011, Christopher Currens <[EMAIL PROTECTED]
> >
> wrote:
> > Hi Trevor,
> >
> > What kind of memory increase are we talking about? Also, how big are the
> > documents that you are indexing, the ones returned from getFileInfoDoc()?
> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> > issues with holding onto allocated byte arrays far beyond when they were
> > used. The memory could only be freed via closing the IndexWriter.
> >
> > I'm a little unclear on exactly what's happening. Are you noticing
> memory
> > spike and stay constant at that level or is it a gradual increase? Is it
> > causing your application to error, (ie OutOfMemory exception, etc)?
> >
> >
> > Thanks,
> > Christopher
> >
> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005 (.NET
> >> 2.0) environment. We had a piece of software that WAS working. I'm not
> >> sure what has changed however, the following code results in a memory
> leak
> >> in the Lucene.Net component (or a failure to clean up used memory).
> >>
> >> The code in issue is here:
> >>
> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
> >> delayCommit, string sDataPath)
> >> {
> >> Document doc = getFileInfoDoc(sDataPath);
> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> if (this.FileID == 0)
> >> {
> >> string s = "";
> >> }
> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> this.fileID.ToString("000000000")), doc, analyzer);
> >>
> >> analyzer = null;
> >> doc = null;
> >> if (!delayCommit)
> >> iw.Commit();
> >> }
> >>
> >> Commenting out the line iw.UpdateDocument resulted in no memory
> increase.
> >> I also tried replacing it with a deleteDocument and AddDocument and the
> >> memory increased the same as using the UpdateDocument function
> >>
> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but it's
> the
> >> UpdateDocument line specifically that gives me the issue.
> >>
> >> Any assistance would be greatly appreciated.
> >>
> >> Trevor Watson
> >>
> >
>
I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
Thanks,
Christopher
On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> Sorry for the duplicate post. I was on the road and posted both via my web
> mail and office mail by mistake
>
> The increase is a very gradual, the program starts at about 160,000k
> according to task manager (I know that's not entirely accurate, but it was
> the best I had at the time) and would, after adding 25,000-40,000 result in
> an out of memory exception (800,000k according to taskmanager). I tried
> building a copy of 2.9.4 to test, but could not find one that worked in
> visual studio 2005
>
> I did notice using Ants memory profiler that there were a number of
> byte[32789] arrays that I didn't know where they came from in memory.
>
> On Monday, November 28, 2011, Christopher Currens <[EMAIL PROTECTED]
> >
> wrote:
> > Hi Trevor,
> >
> > What kind of memory increase are we talking about? Also, how big are the
> > documents that you are indexing, the ones returned from getFileInfoDoc()?
> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> > issues with holding onto allocated byte arrays far beyond when they were
> > used. The memory could only be freed via closing the IndexWriter.
> >
> > I'm a little unclear on exactly what's happening. Are you noticing
> memory
> > spike and stay constant at that level or is it a gradual increase? Is it
> > causing your application to error, (ie OutOfMemory exception, etc)?
> >
> >
> > Thanks,
> > Christopher
> >
> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005 (.NET
> >> 2.0) environment. We had a piece of software that WAS working. I'm not
> >> sure what has changed however, the following code results in a memory
> leak
> >> in the Lucene.Net component (or a failure to clean up used memory).
> >>
> >> The code in issue is here:
> >>
> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
> >> delayCommit, string sDataPath)
> >> {
> >> Document doc = getFileInfoDoc(sDataPath);
> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> if (this.FileID == 0)
> >> {
> >> string s = "";
> >> }
> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> this.fileID.ToString("000000000")), doc, analyzer);
> >>
> >> analyzer = null;
> >> doc = null;
> >> if (!delayCommit)
> >> iw.Commit();
> >> }
> >>
> >> Commenting out the line iw.UpdateDocument resulted in no memory
> increase.
> >> I also tried replacing it with a deleteDocument and AddDocument and the
> >> memory increased the same as using the UpdateDocument function
> >>
> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but it's
> the
> >> UpdateDocument line specifically that gives me the issue.
> >>
> >> Any assistance would be greatly appreciated.
> >>
> >> Trevor Watson
> >>
> >
>
+
Christopher Currens 2011-11-29, 17:16
-
[Lucene.Net] Re: Memory Leak in 2.9.2.2Trevor Watson 2011-11-29, 19:59
I can send you the dll that I am using if you would like. The documents
are _mostly_ small documents. Emails and office docs size of plain text
On Tuesday, November 29, 2011, Christopher Currens <[EMAIL PROTECTED]>
wrote:
> Do you know how big the documents are that you are trying to
delete/update?
> I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
>
>
> Thanks,
> Christopher
>
> On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
>> Sorry for the duplicate post. I was on the road and posted both via my
web
>> mail and office mail by mistake
>>
>> The increase is a very gradual, the program starts at about 160,000k
>> according to task manager (I know that's not entirely accurate, but it
was
>> the best I had at the time) and would, after adding 25,000-40,000 result
in
>> an out of memory exception (800,000k according to taskmanager). I tried
>> building a copy of 2.9.4 to test, but could not find one that worked in
>> visual studio 2005
>>
>> I did notice using Ants memory profiler that there were a number of
>> byte[32789] arrays that I didn't know where they came from in memory.
>>
>> On Monday, November 28, 2011, Christopher Currens <
[EMAIL PROTECTED]
>> >
>> wrote:
>> > Hi Trevor,
>> >
>> > What kind of memory increase are we talking about? Also, how big are
the
>> > documents that you are indexing, the ones returned from
getFileInfoDoc()?
>> > Is it putting an entire file into the index? Pre 2.9.3 versions had
>> > issues with holding onto allocated byte arrays far beyond when they
were
>> > used. The memory could only be freed via closing the IndexWriter.
>> >
>> > I'm a little unclear on exactly what's happening. Are you noticing
>> memory
>> > spike and stay constant at that level or is it a gradual increase? Is
it
>> > causing your application to error, (ie OutOfMemory exception, etc)?
>> >
>> >
>> > Thanks,
>> > Christopher
>> >
>> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
(.NET
>> >> 2.0) environment. We had a piece of software that WAS working. I'm
not
>> >> sure what has changed however, the following code results in a memory
>> leak
>> >> in the Lucene.Net component (or a failure to clean up used memory).
>> >>
>> >> The code in issue is here:
>> >>
>> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
>> >> delayCommit, string sDataPath)
>> >> {
>> >> Document doc = getFileInfoDoc(sDataPath);
>> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
>> >> if (this.FileID == 0)
>> >> {
>> >> string s = "";
>> >> }
>> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
>> >> this.fileID.ToString("000000000")), doc, analyzer);
>> >>
>> >> analyzer = null;
>> >> doc = null;
>> >> if (!delayCommit)
>> >> iw.Commit();
>> >> }
>> >>
>> >> Commenting out the line iw.UpdateDocument resulted in no memory
>> increase.
>> >> I also tried replacing it with a deleteDocument and AddDocument and
the
>> >> memory increased the same as using the UpdateDocument function
>> >>
>> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but
it's
>> the
>> >> UpdateDocument line specifically that gives me the issue.
>> >>
>> >> Any assistance would be greatly appreciated.
>> >>
>> >> Trevor Watson
>> >>
>> >
>>
>
are _mostly_ small documents. Emails and office docs size of plain text
On Tuesday, November 29, 2011, Christopher Currens <[EMAIL PROTECTED]>
wrote:
> Do you know how big the documents are that you are trying to
delete/update?
> I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
>
>
> Thanks,
> Christopher
>
> On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
>> Sorry for the duplicate post. I was on the road and posted both via my
web
>> mail and office mail by mistake
>>
>> The increase is a very gradual, the program starts at about 160,000k
>> according to task manager (I know that's not entirely accurate, but it
was
>> the best I had at the time) and would, after adding 25,000-40,000 result
in
>> an out of memory exception (800,000k according to taskmanager). I tried
>> building a copy of 2.9.4 to test, but could not find one that worked in
>> visual studio 2005
>>
>> I did notice using Ants memory profiler that there were a number of
>> byte[32789] arrays that I didn't know where they came from in memory.
>>
>> On Monday, November 28, 2011, Christopher Currens <
[EMAIL PROTECTED]
>> >
>> wrote:
>> > Hi Trevor,
>> >
>> > What kind of memory increase are we talking about? Also, how big are
the
>> > documents that you are indexing, the ones returned from
getFileInfoDoc()?
>> > Is it putting an entire file into the index? Pre 2.9.3 versions had
>> > issues with holding onto allocated byte arrays far beyond when they
were
>> > used. The memory could only be freed via closing the IndexWriter.
>> >
>> > I'm a little unclear on exactly what's happening. Are you noticing
>> memory
>> > spike and stay constant at that level or is it a gradual increase? Is
it
>> > causing your application to error, (ie OutOfMemory exception, etc)?
>> >
>> >
>> > Thanks,
>> > Christopher
>> >
>> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
(.NET
>> >> 2.0) environment. We had a piece of software that WAS working. I'm
not
>> >> sure what has changed however, the following code results in a memory
>> leak
>> >> in the Lucene.Net component (or a failure to clean up used memory).
>> >>
>> >> The code in issue is here:
>> >>
>> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, bool
>> >> delayCommit, string sDataPath)
>> >> {
>> >> Document doc = getFileInfoDoc(sDataPath);
>> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
>> >> if (this.FileID == 0)
>> >> {
>> >> string s = "";
>> >> }
>> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
>> >> this.fileID.ToString("000000000")), doc, analyzer);
>> >>
>> >> analyzer = null;
>> >> doc = null;
>> >> if (!delayCommit)
>> >> iw.Commit();
>> >> }
>> >>
>> >> Commenting out the line iw.UpdateDocument resulted in no memory
>> increase.
>> >> I also tried replacing it with a deleteDocument and AddDocument and
the
>> >> memory increased the same as using the UpdateDocument function
>> >>
>> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but
it's
>> the
>> >> UpdateDocument line specifically that gives me the issue.
>> >>
>> >> Any assistance would be greatly appreciated.
>> >>
>> >> Trevor Watson
>> >>
>> >
>>
>
+
Trevor Watson 2011-11-29, 19:59
-
[Lucene.Net] Re: Memory Leak in 2.9.2.2Trevor Watson 2011-11-30, 16:14
You said "pre 2.9.3" I checked the apache lucene.net page to try to see if
I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
2.9.4(g). Was this an un-released version? Or am I looking in the wrong
spot for updates to lucene.net?
Thanks for all your help
On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> I can send you the dll that I am using if you would like. The documents
> are _mostly_ small documents. Emails and office docs size of plain text
>
>
> On Tuesday, November 29, 2011, Christopher Currens <
> [EMAIL PROTECTED]> wrote:
> > Do you know how big the documents are that you are trying to
> delete/update?
> > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> >
> >
> > Thanks,
> > Christopher
> >
> > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Sorry for the duplicate post. I was on the road and posted both via my
> web
> >> mail and office mail by mistake
> >>
> >> The increase is a very gradual, the program starts at about 160,000k
> >> according to task manager (I know that's not entirely accurate, but it
> was
> >> the best I had at the time) and would, after adding 25,000-40,000
> result in
> >> an out of memory exception (800,000k according to taskmanager). I tried
> >> building a copy of 2.9.4 to test, but could not find one that worked in
> >> visual studio 2005
> >>
> >> I did notice using Ants memory profiler that there were a number of
> >> byte[32789] arrays that I didn't know where they came from in memory.
> >>
> >> On Monday, November 28, 2011, Christopher Currens <
> [EMAIL PROTECTED]
> >> >
> >> wrote:
> >> > Hi Trevor,
> >> >
> >> > What kind of memory increase are we talking about? Also, how big are
> the
> >> > documents that you are indexing, the ones returned from
> getFileInfoDoc()?
> >> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> >> > issues with holding onto allocated byte arrays far beyond when they
> were
> >> > used. The memory could only be freed via closing the IndexWriter.
> >> >
> >> > I'm a little unclear on exactly what's happening. Are you noticing
> >> memory
> >> > spike and stay constant at that level or is it a gradual increase?
> Is it
> >> > causing your application to error, (ie OutOfMemory exception, etc)?
> >> >
> >> >
> >> > Thanks,
> >> > Christopher
> >> >
> >> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> >> > [EMAIL PROTECTED]> wrote:
> >> >
> >> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
> (.NET
> >> >> 2.0) environment. We had a piece of software that WAS working. I'm
> not
> >> >> sure what has changed however, the following code results in a memory
> >> leak
> >> >> in the Lucene.Net component (or a failure to clean up used memory).
> >> >>
> >> >> The code in issue is here:
> >> >>
> >> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw,
> bool
> >> >> delayCommit, string sDataPath)
> >> >> {
> >> >> Document doc = getFileInfoDoc(sDataPath);
> >> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> >> if (this.FileID == 0)
> >> >> {
> >> >> string s = "";
> >> >> }
> >> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> >> this.fileID.ToString("000000000")), doc, analyzer);
> >> >>
> >> >> analyzer = null;
> >> >> doc = null;
> >> >> if (!delayCommit)
> >> >> iw.Commit();
> >> >> }
> >> >>
> >> >> Commenting out the line iw.UpdateDocument resulted in no memory
> >> increase.
> >> >> I also tried replacing it with a deleteDocument and AddDocument and
> the
> >> >> memory increased the same as using the UpdateDocument function
> >> >>
> >> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but
> it's
> >> the
> >> >> UpdateDocument line specifically that gives me the issue.
> >> >>
> >> >> Any assistance would be greatly appreciated.
> >> >>
> >> >> Trevor Watson
I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
2.9.4(g). Was this an un-released version? Or am I looking in the wrong
spot for updates to lucene.net?
Thanks for all your help
On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> I can send you the dll that I am using if you would like. The documents
> are _mostly_ small documents. Emails and office docs size of plain text
>
>
> On Tuesday, November 29, 2011, Christopher Currens <
> [EMAIL PROTECTED]> wrote:
> > Do you know how big the documents are that you are trying to
> delete/update?
> > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> >
> >
> > Thanks,
> > Christopher
> >
> > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Sorry for the duplicate post. I was on the road and posted both via my
> web
> >> mail and office mail by mistake
> >>
> >> The increase is a very gradual, the program starts at about 160,000k
> >> according to task manager (I know that's not entirely accurate, but it
> was
> >> the best I had at the time) and would, after adding 25,000-40,000
> result in
> >> an out of memory exception (800,000k according to taskmanager). I tried
> >> building a copy of 2.9.4 to test, but could not find one that worked in
> >> visual studio 2005
> >>
> >> I did notice using Ants memory profiler that there were a number of
> >> byte[32789] arrays that I didn't know where they came from in memory.
> >>
> >> On Monday, November 28, 2011, Christopher Currens <
> [EMAIL PROTECTED]
> >> >
> >> wrote:
> >> > Hi Trevor,
> >> >
> >> > What kind of memory increase are we talking about? Also, how big are
> the
> >> > documents that you are indexing, the ones returned from
> getFileInfoDoc()?
> >> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> >> > issues with holding onto allocated byte arrays far beyond when they
> were
> >> > used. The memory could only be freed via closing the IndexWriter.
> >> >
> >> > I'm a little unclear on exactly what's happening. Are you noticing
> >> memory
> >> > spike and stay constant at that level or is it a gradual increase?
> Is it
> >> > causing your application to error, (ie OutOfMemory exception, etc)?
> >> >
> >> >
> >> > Thanks,
> >> > Christopher
> >> >
> >> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> >> > [EMAIL PROTECTED]> wrote:
> >> >
> >> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
> (.NET
> >> >> 2.0) environment. We had a piece of software that WAS working. I'm
> not
> >> >> sure what has changed however, the following code results in a memory
> >> leak
> >> >> in the Lucene.Net component (or a failure to clean up used memory).
> >> >>
> >> >> The code in issue is here:
> >> >>
> >> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw,
> bool
> >> >> delayCommit, string sDataPath)
> >> >> {
> >> >> Document doc = getFileInfoDoc(sDataPath);
> >> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> >> if (this.FileID == 0)
> >> >> {
> >> >> string s = "";
> >> >> }
> >> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> >> this.fileID.ToString("000000000")), doc, analyzer);
> >> >>
> >> >> analyzer = null;
> >> >> doc = null;
> >> >> if (!delayCommit)
> >> >> iw.Commit();
> >> >> }
> >> >>
> >> >> Commenting out the line iw.UpdateDocument resulted in no memory
> >> increase.
> >> >> I also tried replacing it with a deleteDocument and AddDocument and
> the
> >> >> memory increased the same as using the UpdateDocument function
> >> >>
> >> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but
> it's
> >> the
> >> >> UpdateDocument line specifically that gives me the issue.
> >> >>
> >> >> Any assistance would be greatly appreciated.
> >> >>
> >> >> Trevor Watson
+
Trevor Watson 2011-11-30, 16:14
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Prescott Nasser 2011-11-30, 16:25
We just released 2.9.4 - the website didn't update last night, so ill have to try and update it later today. But if you follow the link to download 2.9.2 dist you'll see folders for 2.9.4.
I'll send an email to the user and dev lists once i get the website to update
________________________________
From: Trevor Watson
Sent: 11/30/2011 8:14 AM
To: [EMAIL PROTECTED]
Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
You said "pre 2.9.3" I checked the apache lucene.net page to try to see if
I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
2.9.4(g). Was this an un-released version? Or am I looking in the wrong
spot for updates to lucene.net?
Thanks for all your help
On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> I can send you the dll that I am using if you would like. The documents
> are _mostly_ small documents. Emails and office docs size of plain text
>
>
> On Tuesday, November 29, 2011, Christopher Currens <
> [EMAIL PROTECTED]> wrote:
> > Do you know how big the documents are that you are trying to
> delete/update?
> > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> >
> >
> > Thanks,
> > Christopher
> >
> > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Sorry for the duplicate post. I was on the road and posted both via my
> web
> >> mail and office mail by mistake
> >>
> >> The increase is a very gradual, the program starts at about 160,000k
> >> according to task manager (I know that's not entirely accurate, but it
> was
> >> the best I had at the time) and would, after adding 25,000-40,000
> result in
> >> an out of memory exception (800,000k according to taskmanager). I tried
> >> building a copy of 2.9.4 to test, but could not find one that worked in
> >> visual studio 2005
> >>
> >> I did notice using Ants memory profiler that there were a number of
> >> byte[32789] arrays that I didn't know where they came from in memory.
> >>
> >> On Monday, November 28, 2011, Christopher Currens <
> [EMAIL PROTECTED]
> >> >
> >> wrote:
> >> > Hi Trevor,
> >> >
> >> > What kind of memory increase are we talking about? Also, how big are
> the
> >> > documents that you are indexing, the ones returned from
> getFileInfoDoc()?
> >> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> >> > issues with holding onto allocated byte arrays far beyond when they
> were
> >> > used. The memory could only be freed via closing the IndexWriter.
> >> >
> >> > I'm a little unclear on exactly what's happening. Are you noticing
> >> memory
> >> > spike and stay constant at that level or is it a gradual increase?
> Is it
> >> > causing your application to error, (ie OutOfMemory exception, etc)?
> >> >
> >> >
> >> > Thanks,
> >> > Christopher
> >> >
> >> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> >> > [EMAIL PROTECTED]> wrote:
> >> >
> >> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
> (.NET
> >> >> 2.0) environment. We had a piece of software that WAS working. I'm
> not
> >> >> sure what has changed however, the following code results in a memory
> >> leak
> >> >> in the Lucene.Net component (or a failure to clean up used memory).
> >> >>
> >> >> The code in issue is here:
> >> >>
> >> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw,
> bool
> >> >> delayCommit, string sDataPath)
> >> >> {
> >> >> Document doc = getFileInfoDoc(sDataPath);
> >> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> >> if (this.FileID == 0)
> >> >> {
> >> >> string s = "";
> >> >> }
> >> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> >> this.fileID.ToString("000000000")), doc, analyzer);
> >> >>
> >> >> analyzer = null;
> >> >> doc = null;
> >> >> if (!delayCommit)
> >> >> iw.Commit();
> >> >> }
> >> >>
> >> >> Commenting out the line iw.UpdateDocument resulted in no memory
I'll send an email to the user and dev lists once i get the website to update
________________________________
From: Trevor Watson
Sent: 11/30/2011 8:14 AM
To: [EMAIL PROTECTED]
Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
You said "pre 2.9.3" I checked the apache lucene.net page to try to see if
I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
2.9.4(g). Was this an un-released version? Or am I looking in the wrong
spot for updates to lucene.net?
Thanks for all your help
On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
[EMAIL PROTECTED]> wrote:
> I can send you the dll that I am using if you would like. The documents
> are _mostly_ small documents. Emails and office docs size of plain text
>
>
> On Tuesday, November 29, 2011, Christopher Currens <
> [EMAIL PROTECTED]> wrote:
> > Do you know how big the documents are that you are trying to
> delete/update?
> > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> >
> >
> > Thanks,
> > Christopher
> >
> > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Sorry for the duplicate post. I was on the road and posted both via my
> web
> >> mail and office mail by mistake
> >>
> >> The increase is a very gradual, the program starts at about 160,000k
> >> according to task manager (I know that's not entirely accurate, but it
> was
> >> the best I had at the time) and would, after adding 25,000-40,000
> result in
> >> an out of memory exception (800,000k according to taskmanager). I tried
> >> building a copy of 2.9.4 to test, but could not find one that worked in
> >> visual studio 2005
> >>
> >> I did notice using Ants memory profiler that there were a number of
> >> byte[32789] arrays that I didn't know where they came from in memory.
> >>
> >> On Monday, November 28, 2011, Christopher Currens <
> [EMAIL PROTECTED]
> >> >
> >> wrote:
> >> > Hi Trevor,
> >> >
> >> > What kind of memory increase are we talking about? Also, how big are
> the
> >> > documents that you are indexing, the ones returned from
> getFileInfoDoc()?
> >> > Is it putting an entire file into the index? Pre 2.9.3 versions had
> >> > issues with holding onto allocated byte arrays far beyond when they
> were
> >> > used. The memory could only be freed via closing the IndexWriter.
> >> >
> >> > I'm a little unclear on exactly what's happening. Are you noticing
> >> memory
> >> > spike and stay constant at that level or is it a gradual increase?
> Is it
> >> > causing your application to error, (ie OutOfMemory exception, etc)?
> >> >
> >> >
> >> > Thanks,
> >> > Christopher
> >> >
> >> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson <
> >> > [EMAIL PROTECTED]> wrote:
> >> >
> >> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005
> (.NET
> >> >> 2.0) environment. We had a piece of software that WAS working. I'm
> not
> >> >> sure what has changed however, the following code results in a memory
> >> leak
> >> >> in the Lucene.Net component (or a failure to clean up used memory).
> >> >>
> >> >> The code in issue is here:
> >> >>
> >> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw,
> bool
> >> >> delayCommit, string sDataPath)
> >> >> {
> >> >> Document doc = getFileInfoDoc(sDataPath);
> >> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer();
> >> >> if (this.FileID == 0)
> >> >> {
> >> >> string s = "";
> >> >> }
> >> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId",
> >> >> this.fileID.ToString("000000000")), doc, analyzer);
> >> >>
> >> >> analyzer = null;
> >> >> doc = null;
> >> >> if (!delayCommit)
> >> >> iw.Commit();
> >> >> }
> >> >>
> >> >> Commenting out the line iw.UpdateDocument resulted in no memory
+
Prescott Nasser 2011-11-30, 16:25
-
Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2Christopher Currens 2011-11-30, 17:26
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser <[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via my
> > web
> > >> mail and office mail by mistake
> > >>
> > >> The increase is a very gradual, the program starts at about 160,000k
> > >> according to task manager (I know that's not entirely accurate, but it
> > was
> > >> the best I had at the time) and would, after adding 25,000-40,000
> > result in
> > >> an out of memory exception (800,000k according to taskmanager). I
> tried
> > >> building a copy of 2.9.4 to test, but could not find one that worked
> in
> > >> visual studio 2005
> > >>
> > >> I did notice using Ants memory profiler that there were a number of
> > >> byte[32789] arrays that I didn't know where they came from in memory.
> > >>
> > >> On Monday, November 28, 2011, Christopher Currens <
> > [EMAIL PROTECTED]
> > >> >
> > >> wrote:
> > >> > Hi Trevor,
> > >> >
> > >> > What kind of memory increase are we talking about? Also, how big
> are
> > the
> > >> > documents that you are indexing, the ones returned from
> > getFileInfoDoc()?
> > >> > Is it putting an entire file into the index? Pre 2.9.3 versions
> had
> > >> > issues with holding onto allocated byte arrays far beyond when they
> > were
> > >> > used. The memory could only be freed via closing the IndexWriter.
> > >> >
> > >> > I'm a little unclear on exactly what's happening. Are you noticing
> > >> memory
> > >> > spike and stay constant at that level or is it a gradual increase?
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser <[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via my
> > web
> > >> mail and office mail by mistake
> > >>
> > >> The increase is a very gradual, the program starts at about 160,000k
> > >> according to task manager (I know that's not entirely accurate, but it
> > was
> > >> the best I had at the time) and would, after adding 25,000-40,000
> > result in
> > >> an out of memory exception (800,000k according to taskmanager). I
> tried
> > >> building a copy of 2.9.4 to test, but could not find one that worked
> in
> > >> visual studio 2005
> > >>
> > >> I did notice using Ants memory profiler that there were a number of
> > >> byte[32789] arrays that I didn't know where they came from in memory.
> > >>
> > >> On Monday, November 28, 2011, Christopher Currens <
> > [EMAIL PROTECTED]
> > >> >
> > >> wrote:
> > >> > Hi Trevor,
> > >> >
> > >> > What kind of memory increase are we talking about? Also, how big
> are
> > the
> > >> > documents that you are indexing, the ones returned from
> > getFileInfoDoc()?
> > >> > Is it putting an entire file into the index? Pre 2.9.3 versions
> had
> > >> > issues with holding onto allocated byte arrays far beyond when they
> > were
> > >> > used. The memory could only be freed via closing the IndexWriter.
> > >> >
> > >> > I'm a little unclear on exactly what's happening. Are you noticing
> > >> memory
> > >> > spike and stay constant at that level or is it a gradual increase?
+
Christopher Currens 2011-11-30, 17:26
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Digy 2011-11-30, 18:27
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via
my
> > web
> > >> mail and office mail by mistake
> > >>
> > >> The increase is a very gradual, the program starts at about 160,000k
> > >> according to task manager (I know that's not entirely accurate, but
it
> > was
> > >> the best I had at the time) and would, after adding 25,000-40,000
> > result in
> > >> an out of memory exception (800,000k according to taskmanager). I
> tried
> > >> building a copy of 2.9.4 to test, but could not find one that worked
> in
> > >> visual studio 2005
> > >>
> > >> I did notice using Ants memory profiler that there were a number of
> > >> byte[32789] arrays that I didn't know where they came from in memory.
> > >>
> > >> On Monday, November 28, 2011, Christopher Currens <
> > [EMAIL PROTECTED]
> > >> >
> > >> wrote:
> > >> > Hi Trevor,
> > >> >
> > >> > What kind of memory increase are we talking about? Also, how big
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via
my
> > web
> > >> mail and office mail by mistake
> > >>
> > >> The increase is a very gradual, the program starts at about 160,000k
> > >> according to task manager (I know that's not entirely accurate, but
it
> > was
> > >> the best I had at the time) and would, after adding 25,000-40,000
> > result in
> > >> an out of memory exception (800,000k according to taskmanager). I
> tried
> > >> building a copy of 2.9.4 to test, but could not find one that worked
> in
> > >> visual studio 2005
> > >>
> > >> I did notice using Ants memory profiler that there were a number of
> > >> byte[32789] arrays that I didn't know where they came from in memory.
> > >>
> > >> On Monday, November 28, 2011, Christopher Currens <
> > [EMAIL PROTECTED]
> > >> >
> > >> wrote:
> > >> > Hi Trevor,
> > >> >
> > >> > What kind of memory increase are we talking about? Also, how big
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Digy 2011-11-30, 18:27
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Granroth, Neal V. 2011-11-30, 19:25
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via
my
> > web
> > >> mail and office mail by mistake
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
> > >
> > >
> > > Thanks,
> > > Christopher
> > >
> > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> Sorry for the duplicate post. I was on the road and posted both via
my
> > web
> > >> mail and office mail by mistake
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Granroth, Neal V. 2011-11-30, 19:25
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Digy 2011-11-30, 20:04
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
> >
> > On Tuesday, November 29, 2011, Christopher Currens <
> > [EMAIL PROTECTED]> wrote:
> > > Do you know how big the documents are that you are trying to
> > delete/update?
> > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it.
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Digy 2011-11-30, 20:04
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Prescott Nasser 2011-11-30, 20:35
Probably makes for a good wiki entry
Sent from my Windows Phone
________________________________
From: Digy
Sent: 11/30/2011 12:04 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Sent from my Windows Phone
________________________________
From: Digy
Sent: 11/30/2011 12:04 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
>
> Thanks for all your help
>
> On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson <
> [EMAIL PROTECTED]> wrote:
>
> > I can send you the dll that I am using if you would like. The documents
> > are _mostly_ small documents. Emails and office docs size of plain text
> >
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Prescott Nasser 2011-11-30, 20:35
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Granroth, Neal V. 2011-11-30, 20:41
Or maybe put the changes within a conditional compile code block?
Thanks DIGY, works great.
- Neal
-----Original Message-----
From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 2:35 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Probably makes for a good wiki entry
Sent from my Windows Phone
________________________________
From: Digy
Sent: 11/30/2011 12:04 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Thanks DIGY, works great.
- Neal
-----Original Message-----
From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 2:35 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Probably makes for a good wiki entry
Sent from my Windows Phone
________________________________
From: Digy
Sent: 11/30/2011 12:04 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd
DIGY
-----Original Message-----
From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 9:26 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
DIGY,
Thanks for the tip, but could you be a little more specific?
Where and how are "extension-methods calls" replaced?
For example, how would I change the CloseableThreadLocalExtensions method
public static void Set<T>(this ThreadLocal<T> t, T val)
{
t.Value = val;
}
to eliminate the compile error
Error 2 Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?
due to the Set<T> parameter "this ThreadLocal<t> t" ?
- Neal
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 12:27 PM
To: [EMAIL PROTECTED]
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal
(like uncommenting ThreadLocal<T> class and replacing extension-methods
calls
with static calls to CloseableThreadLocalExtensions)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
I'm not sure if you can use 2.9.4, though, it looks like you're using
VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
classes only available in 4.0 (or 3.5?). However, if you can, I would
suggest updating, as 2.9.4 should be a fairly stable release.
The leak I'm talking about is addressed here:
https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
or may not be what your issue is. You say that it was at one time working
fine, I assume you mean no memory leak. I would take some time to see what
else in your code has changed. Make sure you're calling Close on whatever
needs to be closed (IndexWriter/IndexReader/Analyzers, etc).
Unfortunately for us, memory leaks are hard to debug over email, and it's
difficult for us to tell if it's any change to your code or an issue with
Lucene .NET. As far as I can tell, this is the only memory leak I can find
that affects 2.9.2.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
<[EMAIL PROTECTED]>wrote:
> We just released 2.9.4 - the website didn't update last night, so ill have
> to try and update it later today. But if you follow the link to download
> 2.9.2 dist you'll see folders for 2.9.4.
>
> I'll send an email to the user and dev lists once i get the website to
> update
> ________________________________
> From: Trevor Watson
> Sent: 11/30/2011 8:14 AM
> To: [EMAIL PROTECTED]
> Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> You said "pre 2.9.3" I checked the apache lucene.net page to try to see
> if
> I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and
> 2.9.4(g). Was this an un-released version? Or am I looking in the wrong
> spot for updates to lucene.net?
my
it
memory).
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Granroth, Neal V. 2011-11-30, 20:41
-
Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2Christopher Currens 2011-11-30, 21:33
Trevor,
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
>
>
> Trevor,
>
>
>
> I'm not sure if you can use 2.9.4, though, it looks like you're using
>
> VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
>
> classes only available in 4.0 (or 3.5?). However, if you can, I would
>
> suggest updating, as 2.9.4 should be a fairly stable release.
>
>
>
> The leak I'm talking about is addressed here:
>
> https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
>
> isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
>
> or may not be what your issue is. You say that it was at one time working
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
>
>
> Trevor,
>
>
>
> I'm not sure if you can use 2.9.4, though, it looks like you're using
>
> VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use
>
> classes only available in 4.0 (or 3.5?). However, if you can, I would
>
> suggest updating, as 2.9.4 should be a fairly stable release.
>
>
>
> The leak I'm talking about is addressed here:
>
> https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code
>
> isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may
>
> or may not be what your issue is. You say that it was at one time working
+
Christopher Currens 2011-11-30, 21:33
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Digy 2011-11-30, 22:09
If I recall it correctly, last memory leak problem for 2.9.2 was reported in
~August from RavenDB, and it was fixed in 2.9.4(g)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 11:33 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
>
>
> Trevor,
>
>
>
> I'm not sure if you can use 2.9.4, though, it looks like you're using
>
> VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we
use
>
> classes only available in 4.0 (or 3.5?). However, if you can, I would
may
what
find
wrong
worked
IndexWriter.
increase?
2005
iw,
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
~August from RavenDB, and it was fixed in 2.9.4(g)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 11:33 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
>
>
> Trevor,
>
>
>
> I'm not sure if you can use 2.9.4, though, it looks like you're using
>
> VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we
use
>
> classes only available in 4.0 (or 3.5?). However, if you can, I would
may
what
find
wrong
worked
IndexWriter.
increase?
2005
iw,
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Digy 2011-11-30, 22:09
-
RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2Digy 2011-11-30, 22:26
... and it was related with CloseableThreadLocal (fixed in 2.9.4(g)) which
now creates
compilation problem against .Net20 :)
DIGY
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Thursday, December 01, 2011 12:09 AM
To: '[EMAIL PROTECTED]'
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
If I recall it correctly, last memory leak problem for 2.9.2 was reported in
~August from RavenDB, and it was fixed in 2.9.4(g)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 11:33 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
use
may
what
find
wrong
worked
IndexWriter.
increase?
2005
iw,
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
now creates
compilation problem against .Net20 :)
DIGY
-----Original Message-----
From: Digy [mailto:[EMAIL PROTECTED]]
Sent: Thursday, December 01, 2011 12:09 AM
To: '[EMAIL PROTECTED]'
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
If I recall it correctly, last memory leak problem for 2.9.2 was reported in
~August from RavenDB, and it was fixed in 2.9.4(g)
DIGY
-----Original Message-----
From: Christopher Currens [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 30, 2011 11:33 PM
To: [EMAIL PROTECTED]
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2
Trevor,
Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2. Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace). I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up. That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us. If it's
possible you can write a small program that has the same behavior, that
could help us track it down.
As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays). The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.
Thanks,
Christopher
On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. <
[EMAIL PROTECTED]> wrote:
> Or maybe put the changes within a conditional compile code block?
>
> Thanks DIGY, works great.
>
> - Neal
>
> -----Original Message-----
> From: Prescott Nasser [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 2:35 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> Probably makes for a good wiki entry
>
> Sent from my Windows Phone
> ________________________________
> From: Digy
> Sent: 11/30/2011 12:04 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> OK, here is the code that can be compiled against .NET 2.0
> http://pastebin.com/k2f7JfPd
>
> DIGY
>
>
> -----Original Message-----
> From: Granroth, Neal V. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 9:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> DIGY,
>
> Thanks for the tip, but could you be a little more specific?
> Where and how are "extension-methods calls" replaced?
>
> For example, how would I change the CloseableThreadLocalExtensions method
>
> public static void Set<T>(this ThreadLocal<T> t, T val)
> {
> t.Value = val;
> }
>
>
> to eliminate the compile error
>
> Error 2 Cannot define a new extension method because the compiler
> required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
> be
> found. Are you missing a reference to System.Core.dll?
>
> due to the Set<T> parameter "this ThreadLocal<t> t" ?
>
>
> - Neal
>
>
> -----Original Message-----
> From: Digy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 12:27 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2
>
> FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
> CloseableThreadLocal
>
> (like uncommenting ThreadLocal<T> class and replacing extension-methods
> calls
>
> with static calls to CloseableThreadLocalExtensions)
>
>
>
>
>
> DIGY
>
>
>
>
>
> -----Original Message-----
> From: Christopher Currens [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 30, 2011 7:26 PM
use
may
what
find
wrong
worked
IndexWriter.
increase?
2005
iw,
Checked by AVG - www.avg.com
Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
+
Digy 2011-11-30, 22:26
