Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!newsreader4.netcologne.de!news.netcologne.de!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!194.109.133.86.MISMATCH!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.003 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'method.': 0.05; 'modify': 0.05; 'problem:': 0.07; 'python': 0.09; 'happens.': 0.09; 'parsed': 0.09; 'parsers': 0.09; 'portions': 0.09; 'subject:howto': 0.09; 'subset': 0.09; 'underlying': 0.09; 'xml.': 0.09; '"this': 0.13; 'library': 0.15; 'file,': 0.15; 'disk.': 0.16; 'dropping': 0.16; 'iteratively': 0.16; 'optionally': 0.16; 'python;': 0.16; 'subject:XML': 0.16; 'text?': 0.16; 'alternate': 0.17; 'issue,': 0.17; 'memory': 0.18; 'file.': 0.20; 'trying': 0.21; 'supposed': 0.21; 'converted': 0.22; 'dropped': 0.22; 'file:': 0.22; 'parse': 0.22; 'example': 0.23; "i've": 0.23; 'header:User-Agent:1': 0.26; 'thanks!': 0.26; 'scanned': 0.27; 'library.': 0.27; 'start,': 0.27; 'tree': 0.27; 'subject:/': 0.28; 'bold': 0.29; 'restricted': 0.29; 'convert': 0.29; 'handled': 0.29; 'source': 0.29; "i'm": 0.29; 'error': 0.30; 'figure': 0.30; 'gets': 0.32; 'file': 0.32; 'impression': 0.33; 'like:': 0.33; 'to:addr:python-list': 0.33; 'another': 0.33; 'text': 0.34; '(1)': 0.34; 'or,': 0.34; 'needed': 0.35; 'text.': 0.35; 'so,': 0.35; 'open': 0.35; 'continue': 0.35; 'there': 0.35; 'but': 0.36; 'problems': 0.36; 'xml': 0.37; 'files': 0.38; 'some': 0.38; 'to:addr:python.org': 0.39; 'step': 0.39; 'where': 0.40; 'end': 0.40; 'think': 0.40; 'subject:, ': 0.61; 'telling': 0.61; 'first': 0.61; 'free': 0.61; 'received:phx3.secureserver.net': 0.62; 'received:prod.phx3.secureserver.net': 0.62; 'close': 0.63; 'header:Reply-To:1': 0.68; 'reply-to:no real name:2**0': 0.72; 'day': 0.73; 'grow': 0.74; 'tags,': 0.81; '(2),': 0.84; 'me;': 0.84; 'subject:XHTML': 0.84; 'tree,': 0.84; 'do:': 0.91; 'drops': 0.91; 'received:173.201': 0.91 Date: Wed, 23 Jan 2013 15:22:23 +0000 From: Andrew Robinson User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120909 Thunderbird/15.0.1 MIME-Version: 1.0 To: python-list@python.org Subject: XML/XHTML/HTML differences, bugs... and howto Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: andrew3@r3dsolutions.com List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 69 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1358983473 news.xs4all.nl 6985 [2001:888:2000:d::a6]:42983 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:37517 Good day :), I've been exploring XML parsers in python; particularly: xml.etree.cElementTree; and I'm trying to figure out how to do it incrementally, for very large XML files -- although I don't think the problems are restricted to incremental parsing. First problem: I've come across an issue where etree silently drops text without telling me; and separate. I am under the impression that XHTML is a subset of XML (eg:defined tags), and that once an HTML file is converted to XHTML, the body of the document can be handled entirely as XML. If I convert a (partial/contrived) html file like:

This is example bold text.

to XHTML, I might do --right or wrong-- (1):

This is example bold text.

or, alternate difference: (2): "

This is example bold text.

" But, when I parse with etree, in example (1) both "This is an example" and "text." are dropped; The missing text is part of the start, or end event tags, in the incrementally parsed method. Likewise: In example (2), only "text" gets dropped. So, etree is silently dropping all text following a close tag, but before another open tag happens. Q: Isn't XML supposed to error out when invalid xml is parsed? Is there a way in etree to recover/access the dropped text? If not -- is the a python library issue, or the underlying expat.so, etc. library. Secondly; I have an XML file which will grow larger than memory on a target machine, so here's what I want to do: Given a source XML file, and a destination file: 1) iteratively scan part of the source tree. 2) Optionally Modify some of scanned tree. 3) Write partial scan/tree out to the destination file. 4) Free memory of no-longer needed (partial) source XML. 5) continue scanning a new section of the source file... eg: goto step 1 until source file is exhausted. But, I don't see a way to write portions of an XML tree, or iteratively write a tree to disk. How can this be done? :) Thanks!