Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Stefan Behnel <stefan_ml@behnel.de>
Subject: Re: parsing multiple root element XML into text
Date: Fri, 09 May 2014 21:51:56 +0200
References: <0e5e9a24-3663-4293-a530-239486cf28fc@googlegroups.com> <87oaz7uvo4.fsf@dpt-info.u-strasbg.fr> <87a9arfdha.fsf@elektro.pacujo.net> <87k39vupnc.fsf@dpt-info.u-strasbg.fr> <8738gjf813.fsf@elektro.pacujo.net> <87y4ybdt46.fsf@elektro.pacujo.net> <lkimp0$428$1@ger.gmane.org> <536D07DF.1080800@arskom.com.tr>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0
In-Reply-To: <536D07DF.1080800@arskom.com.tr>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.9827.1399665133.18130.python-list@python.org>
Lines: 53
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:71186

Burak Arslan, 09.05.2014 18:52:
> On 05/09/14 16:55, Stefan Behnel wrote:
>> ElementTree has gained a nice API in
>> Py3.4 that supports this in a much saner way than SAX, using iterators.
>> Basically, you just dump in some data that you received and get back an
>> iterator over the elements (and their subtrees) that it generated from it.
>> Intercept on the right top elements and you get your next subtree as soon
>> as it's ready.
> 
> Here's a small script:

A bit hard to read, though.


>     events = etree.iterparse(istr, events=("start", "end"))
>     stack = deque()
>     for event, element in events:
>     if event == "start":
>     stack.append(element)
>     elif event == "end":
>     stack.pop()
>      
>     if len(stack) == 0:
>     break
>      
>     print(istr.tell(), "%5s, %4s, %s" % (event, element.tag, element.text))
> 
> where istr is an input-stream. (Fully working example:
> https://gist.github.com/plq/025005a71e8135c46800)
> 
> I was expecting to have istr.tell() return the position where the first
> root element ends, which would make it possible to continue parsing with
> another call to etree.iterparse(). But istr.tell() returns the position
> of EOF after the first call to next() on the iterator it returns.

Correct, because it finished parsing. It controls the reading process and
reads ahead, that's how iterparse() works.


> Without the stack check, the loop eventually throws an exception and the
> offset value in that exception is None.
> 
> So I'm lost here, how it'd possible to parse OP's document with lxml?

See my earlier post. Instead of XMLParser, just use the XMLPullParser for
incremental (non-blocking) parsing and processing.

To make this clear, though: to use an XML parser, you need well formed XML,
and that means exactly one root element.

Stefan