Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'automate': 0.05; 'complaining': 0.07; 'subject:code': 0.07; 'python': 0.08; "(it's": 0.09; 'encoding:': 0.09; 'subject:characters': 0.09; 'suggestions,': 0.09; 'url:peps': 0.09; 'utf-8': 0.09; 'am,': 0.12; 'error:': 0.12; 'def': 0.15; '-*-': 0.16; 'code?': 0.16; 'coding:': 0.16; 'declared;': 0.16; 'from:addr:alec.taylor6': 0.16; 'from:name:alec taylor': 0.16; 'odt': 0.16; 'subject:non': 0.16; 'syntaxerror:': 0.16; 'url:pep-0263': 0.16; '\xa0for': 0.16; '\xa0what': 0.16; 'mon,': 0.16; 'wrote:': 0.16; 'converted': 0.18; 'convert': 0.19; 'stuff': 0.22; 'header:In-Reply-To:1': 0.22; 'received:209.85.213.46': 0.23; 'received:mail- yw0-f46.google.com': 0.23; 'sep': 0.23; '\xa0if': 0.23; 'pm,': 0.24; 'there.': 0.24; 'specify': 0.24; 'guess': 0.26; 'skip:b 20': 0.26; "i'm": 0.27; 'url:mailman': 0.28; 'import': 0.28; 'message- id:@mail.gmail.com': 0.29; 'writer': 0.30; 'thanks': 0.30; 'specified': 0.31; 'subject:?': 0.31; 'error': 0.32; 'yet': 0.32; 'does': 0.32; 'comment': 0.32; "can't": 0.33; 'to:addr:python- list': 0.33; 'url:listinfo': 0.33; '(as': 0.34; 'but,': 0.34; 'character': 0.34; 'quite': 0.34; 'file.': 0.34; 'subject:How': 0.35; 'post': 0.36; 'file': 0.36; 'url:python': 0.36; 'question': 0.36; 'using': 0.37; 'but': 0.37; 'something': 0.37; 'some': 0.38; 'received:google.com': 0.38; 'url:org': 0.38; 'received:209.85': 0.38; 'should': 0.38; 'subject:: ': 0.39; 'characters': 0.39; 'to:addr:python.org': 0.39; 'subject:from': 0.40; 'might': 0.40; 'more': 0.60; 'removal': 0.64; 'tag': 0.64; 'taylor': 0.67; 'paper': 0.72; 'evening,': 0.84; 'subject:removal': 0.84; 'writer,': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=XLOH6RtoPmzrAIe0GNQ4kSaHZ+VEDUHeASW4JnH/p+U=; b=w8MBNFtUQk/DazqDDw5rymZoGGk2LcTbpOsg534HFqz6aagM2L8mpiXpQ18Asry63x 7JxUgQokpC8yQWqjorq7GDrecafaud34qjvFgmy9/zYsj+9vrEos1D5IQZa5H9Dpcph2 UjqCWt9QzPfcxnI6kz/mVLVJsNIrKnjeby9R4= MIME-Version: 1.0 In-Reply-To: References: <4E6DC028.1020101@islandtraining.com> Date: Mon, 12 Sep 2011 18:33:14 +1000 Subject: Re: How do I automate the removal of all non-ascii characters from my code? From: Alec Taylor To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 69 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1315816397 news.xs4all.nl 2512 [2001:888:2000:d::a6]:53218 X-Complaints-To: abuse@xs4all.nl Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:13161 from creole import html2creole from BeautifulSoup import BeautifulSoup VALID_TAGS =3D ['strong', 'em', 'p', 'ul', 'li', 'br', 'b', 'i', 'a', 'h1',= 'h2'] def sanitize_html(value): =A0 =A0soup =3D BeautifulSoup(value) =A0 =A0for tag in soup.findAll(True): =A0 =A0 =A0 =A0if tag.name not in VALID_TAGS: =A0 =A0 =A0 =A0 =A0 =A0tag.hidden =3D True =A0 =A0return soup.renderContents() html2creole(u(sanitize_html('''

Abstract

=A0 =A0

[more stuff here] """)) On Mon, Sep 12, 2011 at 6:17 PM, Gary Herron w= rote: > On 09/12/2011 12:49 AM, Alec Taylor wrote: >> >> Good evening, >> >> I have converted ODT to HTML using LibreOffice Writer, because I want >> to convert from HTML to Creole using python-creole. Unfortunately I >> get this error: "File "Convert to Creole.py", line 17 >> SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py >> on line 18, but no encoding declared; see >> http://www.python.org/peps/pep-0263.html for details". >> >> Unfortunately I can't post my document yet (it's a research paper I'm >> working on), but I'm sure you'll get the same result if you write up a >> document in LibreOffice Writer and add some End Notes. >> >> How do I automate the removal of all non-ascii characters from my code? >> >> Thanks for all suggestions, >> >> Alec Taylor > > > > This question does not quite make sense. =A0The error message is complain= ing > about a python file. =A0What does that file have to do with ODT to HTML > conversion and LibreOffice? > > The error message means the python file (wherever it came from) has a > non-ascii character (as you noted), and so it needs something to tell it > what such a character means. =A0(That what the encoding is.) > > A comment like this in line 1 or 2 will specify an encoding: > =A0# -*- coding: -*- > but, we'll have to know more about the file "Convert to Creole.py" to gue= ss > what encoding name should be specified there. > > You might try utf-8 or latin-1. > > > -- > http://mail.python.org/mailman/listinfo/python-list >