Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!ecngs!feeder2.ecngs.de!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'url:pypi': 0.03; 'binary': 0.05; 'subject:skip:s 10': 0.05; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; "'replace')": 0.16; 'from:addr:cheimes.de': 0.16; 'from:addr:lists': 0.16; 'from:name:christian heimes': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'us-ascii': 0.16; 'skip:u 30': 0.17; 'unicode': 0.17; 'tim': 0.18; '>>>': 0.18; 'import': 0.21; "i'd": 0.22; "i've": 0.23; 'header :In-Reply-To:1': 0.25; 'header:User-Agent:1': 0.26; '[1]': 0.27; '[2]': 0.27; 'possible,': 0.27; "doesn't": 0.28; 'header:X -Complaints-To:1': 0.28; 'becomes': 0.30; 'stuff': 0.30; 'url:python': 0.32; 'print': 0.32; 'to:addr:python-list': 0.33; 'text': 0.34; 'christian': 0.34; 'mapping': 0.35; 'subject:?': 0.35; 'there': 0.35; 'received:org': 0.36; 'but': 0.36; 'url:org': 0.36; 'stock': 0.36; 'anything': 0.36; 'subject:: ': 0.38; 'supports': 0.38; 'to:addr:python.org': 0.39; 'subject:-': 0.40; 'header:Received:5': 0.40; 'skip:u 10': 0.60; 'information': 0.63; 'more': 0.63; 'url:0': 0.67; 'url:4': 0.72; 'chinese': 0.78; 'extensions.': 0.84; 'hundred': 0.95
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: Christian Heimes <lists@cheimes.de>
Subject: Re: Least-lossy string.encode to us-ascii?
Date: Fri, 14 Sep 2012 00:00:45 +0200
References: <50524F6F.6070604@tim.thechases.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Gmane-NNTP-Posting-Host: f048038022.adsl.alicedsl.de
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0
In-Reply-To: <50524F6F.6070604@tim.thechases.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.643.1347573660.27098.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1347573660 news.xs4all.nl 6871 [2001:888:2000:d::a6]:36876
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:29080

Am 13.09.2012 23:26, schrieb Tim Chase:
> I've got a bunch of text in Portuguese and to transmit them, need to
> have them in us-ascii (7-bit).  I'd like to keep as much information
> as possible, just stripping accents, cedillas, tildes, etc.  So
> "serviço móvil" becomes "servico movil".  Is there anything stock
> that I've missed?  I can do mystring.encode('us-ascii', 'replace')
> but that doesn't keep as much information as I'd hope.

The unidecode [1] package contains a large mapping of unicode chars to
ASCII. It even supports cool stuff like Chinese to ASCII:

>>> import unidecode
>>> print u"\u5317\u4EB0"
北亰
>>> print unidecode.unidecode(u"\u5317\u4EB0")
Bei Jing

icu4c and pyicu [2] may contain more methods for conversion but they
require binary extensions. By the way ICU can do a lot of cool, too:

>>> import icu
>>> rbf = icu.RuleBasedNumberFormat(icu.URBNFRuleSetTag.SPELLOUT,
icu.Locale.getUS())
>>> rbf.format(23)
u'twenty-three'
>>> rbf.format(100000)
u'one hundred thousand'

Regards,
Christian

[1] http://pypi.python.org/pypi/Unidecode/0.04.9
[2] http://pypi.python.org/pypi/PyICU/1.4