Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'encoding': 0.05; 'column': 0.07; 'indicated': 0.07; 'urllib2': 0.07; 'moreover,': 0.09; 'parsing': 0.09; 'subject:parsing': 0.09; 'true)': 0.09; 'python': 0.11; '"w")': 0.16; 'chrome,': 0.16; 'filesystem': 0.16; 'from:addr:mrabarnett.plus.com': 0.16; 'from:addr:python': 0.16; 'from:name:mrab': 0.16; 'json:': 0.16; 'message- id:@mrabarnett.plus.com': 0.16; 'received:192.168.1.4': 0.16; 'received:84.93': 0.16; 'received:84.93.230': 0.16; 'server:': 0.16; 'skip:g 50': 0.16; 'stringio': 0.16; 'subject: \n ': 0.16; 'subject:XML': 0.16; 'subject:skip:x 10': 0.16; 'url:json': 0.16; 'weird': 0.16; 'wrote:': 0.18; 'file,': 0.19; 'thanks.': 0.20; 'input': 0.22; 'import': 0.22; 'header:User-Agent:1': 0.23; 'error': 0.23; 'skip:e 30': 0.24; 'specify': 0.24; 'looks': 0.24; "i've": 0.25; 'source': 0.25; 'script': 0.25; 'skip:" 40': 0.26; 'header:In-Reply-To:1': 0.27; 'wondering': 0.29; 'xml': 0.29; 'returned': 0.30; "i'm": 0.30; 'included': 0.31; '>>>>': 0.31; 'directly,': 0.31; 'dom': 0.31; 'skip:= 40': 0.31; 'file': 0.32; 'worked': 0.33; 'running': 0.33; '(most': 0.33; 'beginning': 0.33; 'skip:# 10': 0.33; 'problem': 0.35; 'subject:with': 0.35; 'skip:u 20': 0.35; 'good.': 0.35; 'received:84': 0.35; 'but': 0.35; 'there': 0.35; 'data,': 0.36; 'format.': 0.36; 'ubuntu': 0.36; 'hi,': 0.36; 'url:org': 0.36; 'example,': 0.37; 'too': 0.37; 'stopped': 0.38; 'to:addr:python-list': 0.38; 'recent': 0.39; 'to:addr:python.org': 0.39; 'skip:p 20': 0.39; 'according': 0.40; 'skip:x 10': 0.40; 'skip:u 10': 0.60; 'then,': 0.60; 'ago.': 0.61; 'below.': 0.71; 'fat': 0.84; 'otten': 0.84; 'validator': 0.84; 'lazy': 0.91 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.1 cv=JLW1sq6b c=1 sm=1 tr=0 a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17 a=0Bzu9jTXAAAA:8 a=sASEtNAQL0YA:10 a=2bsVvZDbDyIA:10 a=ihvODaAuJD4A:10 a=IkcTkHD0fZMA:10 a=EBOSESyhAAAA:8 a=L5uGUicSzeIA:10 a=XgWIqx5dAAAA:8 a=q4vJPVb_oNI3wty4RZcA:9 a=QEXdDO2ut3YA:10 X-AUTH: mrabarnett:2500 Date: Thu, 13 Feb 2014 20:19:26 +0000 From: MRAB User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: python-list@python.org Subject: Re: XML parsing ExpatError with xml.dom.minidom at line 1, column 0 References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 86 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1392322769 news.xs4all.nl 2954 [2001:888:2000:d::a6]:33218 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:66231 On 2014-02-13 20:10, Peter Otten wrote: > ming wrote: > >> Hi, >> i've a Python script which stopped working about a month ago. But until >> then, it worked flawlessly for months (if not years). A tiny >> self-contained 7-line script is provided below. >> >> i ran into an XML parsing problem with xml.dom.minidom and the error >> message is included below. The weird thing is if i used an XML validator >> on the web to validate against this particular URL directly, it is all >> good. Moreover, i saved the page source in Firefox or Chrome then >> validated against the saved XML file, it's also all good. >> >> Since the error happened at the very beginning of the input (line 1, >> column 0) as indicated below, i was wondering if this is an encoding >> mismatch. However, according to the saved page source in FireFox or >> Chrome, there is the following at the beginning: >> >> >> >> >> ================================================= >> #!/usr/bin/env python >> >> import urllib2 >> from xml.dom.minidom import parseString >> >> fd = urllib2.urlopen('http://api.worldbank.org/countries') >> data = fd.read() >> fd.close() >> dom = parseString(data) >> ================================================= >> >> >> >> ================================================= >> Traceback (most recent call last): >> File "./bugReport.py", line 9, in >> dom = parseString(data) >> File "/usr/lib/python2.7/xml/dom/minidom.py", line 1931, in parseString >> return expatbuilder.parseString(string) >> File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 940, in >> parseString >> return builder.parseString(string) >> File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 223, in >> parseString >> parser.Parse(string, True) >> xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, >> column 0 ================================================= >> >> >> i'm running Python 2.7.5+ on Ubuntu 13.10. >> >> Thanks. > > Looking into the data returned from the server: > >>>> data = urllib2.urlopen("http://api.worldbank.org/countries").read() >>>> with open("tmp.dat", "w") as f: f.write(data) > ... >>>> > [1]+ Angehalten python > $ file tmp.dat > tmp.dat: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT) > > OK, let's expand: > > $ fg > python > > >>>> import gzip, StringIO >>>> expanded_data = gzip.GzipFile(fileobj=StringIO.StringIO(data)).read() >>>> import xml.dom.minidom >>>> xml.dom.minidom.parseString(expanded_data) > > > There may be a way to uncompress the gzipped data transparently, but I'm too > lazy to look it up... > From a brief look at the docs, it looks like you can specify the format. For example, for JSON: fd = urlopen('http://api.worldbank.org/countries?format=json')