Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #37542

Re: XML/XHTML/HTML differences, bugs... and howto

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'method.': 0.05; 'modify': 0.05; 'problem:': 0.07; 'python': 0.09; 'parsed': 0.09; 'parsers': 0.09; 'portions': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:howto': 0.09; 'subset': 0.09; 'url:github': 0.09; 'xml.': 0.09; '"this': 0.13; 'library': 0.15; 'file,': 0.15; 'compatible,': 0.16; 'disk.': 0.16; 'examples:': 0.16; 'from:addr:behnel.de': 0.16; 'from:addr:stefan_ml': 0.16; 'from:name:stefan behnel': 0.16; 'gained': 0.16; 'iteratively': 0.16; 'optionally': 0.16; 'python;': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'recently.': 0.16; 'subject:XML': 0.16; 'url:py': 0.16; 'alternate': 0.17; 'documented': 0.17; 'stefan': 0.17; 'memory': 0.18; 'file.': 0.20; 'trying': 0.21; 'supposed': 0.21; 'converted': 0.22; 'file:': 0.22; 'libraries': 0.22; 'parse': 0.22; 'example': 0.23; "i've": 0.23; 'external': 0.24; 'feature': 0.24; 'specifically': 0.24; 'header:In-Reply-To:1': 0.25; 'header :User-Agent:1': 0.26; 'scanned': 0.27; 'andrew': 0.27; 'start,': 0.27; 'tree': 0.27; 'header:X-Complaints-To:1': 0.28; 'subject:/': 0.28; '(unless': 0.29; 'bold': 0.29; 'markup': 0.29; 'restricted': 0.29; 'convert': 0.29; 'handled': 0.29; 'manual': 0.29; 'source': 0.29; 'this.': 0.29; "i'm": 0.29; 'error': 0.30; 'figure': 0.30; 'gets': 0.32; 'file': 0.32; 'impression': 0.33; 'like:': 0.33; 'to:addr:python-list': 0.33; 'tutorial': 0.33; 'text': 0.34; '(1)': 0.34; 'or,': 0.34; 'needed': 0.35; 'text.': 0.35; 'continue': 0.35; 'there': 0.35; 'received:org': 0.36; 'but': 0.36; 'should': 0.36; 'problems': 0.36; 'xml': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'some': 0.38; 'several': 0.39; 'to:addr:python.org': 0.39; 'step': 0.39; 'where': 0.40; 'header:Received:5': 0.40; 'end': 0.40; 'think': 0.40; 'most': 0.61; 'subject:, ': 0.61; 'telling': 0.61; 'first': 0.61; 'free': 0.61; 'here': 0.65; 'stated': 0.69; 'day': 0.73; 'grow': 0.74; 'tags,': 0.81; '(2),': 0.84; 'me;': 0.84; 'received:87.139': 0.84; 'subject:XHTML': 0.84; 'tree,': 0.84; 'url:master': 0.84; 'do:': 0.91; 'drops': 0.91; 'url:tutorial': 0.93
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Stefan Behnel <stefan_ml@behnel.de>
Subject Re: XML/XHTML/HTML differences, bugs... and howto
Date Thu, 24 Jan 2013 07:42:22 +0100
References <5100002F.7020809@r3dsolutions.com>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding 7bit
X-Gmane-NNTP-Posting-Host p578ba676.dip0.t-ipconnect.de
User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
In-Reply-To <5100002F.7020809@r3dsolutions.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.945.1359009757.2939.python-list@python.org> (permalink)
Lines 81
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1359009757 news.xs4all.nl 6852 [2001:888:2000:d::a6]:60671
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:37542

Show key headers only | View raw

Andrew Robinson, 23.01.2013 16:22:
> Good day :),
> 
> I've been exploring XML parsers in python; particularly:
> xml.etree.cElementTree; and I'm trying to figure out how to do it
> incrementally, for very large XML files -- although I don't think the
> problems are restricted to incremental parsing.
> 
> First problem:
> I've come across an issue where etree silently drops text without telling
> me; and separate.
> 
> I am under the impression that XHTML is a subset of XML (eg:defined tags),
> and that once an HTML file is converted to XHTML, the body of the document
> can be handled entirely as XML.
> 
> If I convert a (partial/contrived) html file like:
> 
> <html>
>     <div>
>         <p> This is example <b>bold</b> text.
>     </div>
> </html>
> 
> to XHTML, I might do --right or wrong-- (1):
> 
> <html>
>     <div>
>         <p /> This is example <b>bold</b> text.
>     </div>
> </html>
> 
> or, alternate difference: (2): "<p> This is example <b>bold</b> text. </p>"
> 
> But, when I parse with etree,  in example (1) both "This is an example" and
> "text." are dropped;
> The missing text is part of the start, or end event tags, in the
> incrementally parsed method.
> 
> Likewise: In example (2), only "text" gets dropped.

Nope, you should read the manual on this. Here's a tutorial:

http://lxml.de/tutorial.html#elements-contain-text

This is using lxml.etree, which is the Python XML library most people use
these days. It's ElementTree compatible, so the tutorial also works for ET
(unless stated otherwise).


> Isn't XML supposed to error out when invalid xml is parsed?

It does.


> I have an XML file which will grow larger than memory on a target machine,
> so here's what I want to do:
> 
> Given a source XML file, and a destination file:
> 1) iteratively scan part of the source tree.
> 2) Optionally Modify some of scanned tree.
> 3) Write partial scan/tree out to the destination file.
> 4) Free memory of no-longer needed (partial) source XML.
> 5) continue scanning a new section of the source file... eg: goto step 1
> until source file is exhausted.
> 
> But, I don't see a way to write portions of an XML tree, or iteratively
> write a tree to disk.
> How can this be done?

There are several ways to do it. Python has a couple of external libraries
available that are made specifically for generating markup incrementally.

lxml also gained that feature recently. It's not documented yet, but here
are usage examples:

https://github.com/lxml/lxml/blob/master/src/lxml/tests/test_incremental_xmlfile.py

Stefan

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: XML/XHTML/HTML differences, bugs... and howto Stefan Behnel <stefan_ml@behnel.de> - 2013-01-24 07:42 +0100

csiph-web