Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <SRS0=QWVk=35=ieee.org=davea@srs.perfora.net>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.033
X-Spam-Evidence: '*H*': 0.93; '*S*': 0.00; 'subject:code': 0.07; 'subject:characters': 0.09; 'exception': 0.12; 'literal,': 0.16; 'posted,': 0.16; 'received:192.168.1.104': 0.16; 'subject: \n ': 0.16; 'subject:non': 0.16; 'cc:addr:python-list': 0.16; 'mon,': 0.16; 'wrote:': 0.16; 'part,': 0.18; 'convert': 0.19; 'cc:no real name:2**0': 0.20; "haven't": 0.20; 'appears': 0.20; 'trying': 0.21; "doesn't": 0.22; 'stuff': 0.22; 'cc:2**0': 0.22; 'header:In- Reply-To:1': 0.22; '(usually': 0.23; 'sep': 0.23; 'pm,': 0.24; 'expect': 0.25; 'code': 0.25; 'guess': 0.26; 'string': 0.26; 'up.': 0.26; "i'm": 0.27; 'code,': 0.28; 'problem': 0.28; 'stefan': 0.29; 'cc:addr:python.org': 0.30; 'character.': 0.30; 'subject:?': 0.31; "didn't": 0.31; 'source': 0.33; 'probably': 0.33; 'instead': 0.33; 'copying': 0.33; 'header:User-Agent:1': 0.34; 'things': 0.34; 'uses': 0.35; 'subject:How': 0.35; 'unless': 0.36; 'file': 0.36; 'problems': 0.36; 'but': 0.37; 'steven': 0.38; 'subject:: ': 0.39; 'received:192': 0.39; 'delete': 0.40; 'subject:from': 0.40; "it's": 0.40; 'more': 0.60; 'you.': 0.62; 'fall': 0.64; 'header:Reply-To:1': 0.71; 'reply-to:no real name:2**0': 0.71; 'spaces': 0.73; '"smart': 0.84; '02:59': 0.84; 'encoding,': 0.84; 'pasting': 0.84; 'subject:removal': 0.84; 'investigated': 0.91; 'subject:\t': 0.93
Date: Mon, 12 Sep 2011 08:09:59 -0400
From: Dave Angel <davea@ieee.org>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.21) Gecko/20110831 Thunderbird/3.1.13
MIME-Version: 1.0
To: Steven D'Aprano <steve+comp.lang.python@pearwood.info>
Subject: Re: How do I automate the removal of all non-ascii characters from my	code?
References: <CAO+9iGc0f7vFLrjjm24vzZqoMV04eL4fBQrYOTfDt-eYCNEacQ@mail.gmail.com>	<4E6DC028.1020101@islandtraining.com>	<CAO+9iGfHABoWnz-Podk9J5D3EJFgQ-9th=ky2Z-uW8MbQJA12A@mail.gmail.com>	<CAO+9iGe2JcUHY8u+KEZfv3r8VmBdJ9h9NLwL3nO2Wpm80zRT2Q@mail.gmail.com>	<mailman.1021.1315817058.27778.python-list@python.org> <4e6dc7b4$0$29986$c3e8da3$5496439d@news.astraweb.com>
In-Reply-To: <4e6dc7b4$0$29986$c3e8da3$5496439d@news.astraweb.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:gHLZNxU0F9s3lpelRlxlRnxfomtr+DQPij/Q2tuRJXZ +r2flaKHD5vv1ZrD2vTKLGJR2n41AbY2kmn+fu8PD8cJPojBi3 kpYKTpBMmgfD5HNcEh4CUwDUWf3393wgSP5sWmftNL06sZ30ZU V0kLa3T5PIDAWkYgFdbkryCoAM57oSErnuNVLgcZGGxBef2zR+ FdDF7f4VS9GwlevUQ49hxQgKjz+vjq3sw1F7G9a7d12aGCJ3zW nc0Rs/UHyfsCl0+QlQC4GrPGYi6JLumeznrivpqf3pNSvbeoLL dihMhF58EuzunSgs8IZGNhSXVARDMJhikUr8FhVDf62YJ4Mkg= =
Cc: python-list@python.org
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: davea@ieee.org
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.1028.1315829434.27778.python-list@python.org>
Lines: 24
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1315829434 news.xs4all.nl 2510 [2001:888:2000:d::a6]:56678
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:13170

On 01/-10/-28163 02:59 PM, Steven D'Aprano wrote:
> On Mon, 12 Sep 2011 06:43 pm Stefan Behnel wrote:
>
>> I'm not sure what you are trying to say with the above code, but if it's
>> the code that fails for you with the exception you posted, I would guess
>> that the problem is in the "[more stuff here]" part, which likely contains
>> a non-ASCII character. Note that you didn't declare the source file
>> encoding above. Do as Gary told you.
> Even with a source code encoding, you will probably have problems with
> source files including \xe2 and other "bad" chars. Unless they happen to
> fall inside a quoted string literal, I would expect to get a SyntaxError.
>
> I have come across this myself. While I haven't really investigated in great
> detail, it appears to happen when copying and pasting code from a document
> (usually HTML) which uses non-breaking spaces instead of \x20 space
> characters. All it takes is just one to screw things up.
>
>

For me, more common than non-breaking space is the "smart quotes" 
characters.  In that case, one probably doesn't want to delete them, but 
instead convert them into standard quotes.

DaveA