Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #30615

Re: Out of memory error in SAX parsing with validation

Show all headers | View raw

Am 26.12.2014 00:45, schrieb Arne Vajhøj:
> On 12/25/2014 6:32 PM, Sebastian wrote:
>> does anyone here know something about the memory requirements for
>> validating XML with SAX? I've encountered what I think is a memory leak
>> with the Xerces version included in JDK 7 and 8.
>>
>> I'm using a SAX parser (XMLReader) to parse a large XML file.
>
>> Using a non-validating parser, I can process a 7 GB file containing 25
>> million small elements (each having ca. 3 - 5 subelements) with just 64
>> MB of heap space. With XML validation against a DTD turned on, 1024 MB
>> do not suffice. I have taken a cursory glance at the heap with
>> JVisualVM, and see millions of QName instances being created and never
>> being GC'ed. I suspect this to be at least a part of the problem.
>>
>> Can anyone enlighten me as to why SAX would require so much memory for
>> validation? Isn't it enough to know that each element is well-formed?
>
> If you ask it to validate against a DTD then it is obviously not
> enough to check for well-formed-ness.
>
> Arne
>
>
sorry, I did mean "valid". At the end of an element, shouldn't the 
parser be able to release all resources associated with validating the 
current "level", i. e. everything except information about the ancestors 
of the next element? After all, a DTD cannot contain constraints like: 
"if you have seen element X, no element Y must occur",
which would ncessitate retaining information about siblings.

-- Sebastian

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Out of memory error in SAX parsing with validation Sebastian <news@seyweiler.dyndns.org> - 2014-12-26 00:32 +0100
  Re: Out of memory error in SAX parsing with validation Arne Vajhøj <arne@vajhoej.dk> - 2014-12-25 18:45 -0500
    Re: Out of memory error in SAX parsing with validation Sebastian <news@seyweiler.dyndns.org> - 2014-12-26 11:45 +0100
      Re: Out of memory error in SAX parsing with validation Arne Vajhøj <arne@vajhoej.dk> - 2014-12-26 08:53 -0500
        Re: Out of memory error in SAX parsing with validation Sebastian <news@seyweiler.dyndns.org> - 2014-12-27 15:48 +0100

csiph-web