Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.dougwise.org!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.002 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:" ': 0.03; 'ascii': 0.07; 'character,': 0.07; 'refers': 0.07; 'url:pypi': 0.09; 'from:addr:python': 0.09; 'properties,': 0.09; 'this:': 0.11; 'wrote:': 0.14; 'from:addr:mrabarnett.plus.com': 0.16; 'from:name:mrab': 0.16; 'identifier': 0.16; 'message- id:@mrabarnett.plus.com': 0.16; 'name).': 0.16; 'received:84.92': 0.16; 'received:84.92.122': 0.16; 'received:84.92.122.60': 0.16; 'reply-to:addr:python-list': 0.16; 'subject:Extracting': 0.16; 'underscore': 0.16; 'exists': 0.19; 'expressions': 0.19; 'skip:r 30': 0.19; 'wondering': 0.19; '(or': 0.22; 'header:In-Reply-To:1': 0.22; 'module,': 0.23; 'restrict': 0.23; '(not': 0.24; 'extract': 0.25; 'received:84': 0.25; "doesn't": 0.28; 'class': 0.29; 'unicode': 0.29; 'digits': 0.31; 'pattern': 0.31; 'to:addr:python- list': 0.32; 'character': 0.33; 'using': 0.34; 'regular': 0.34; 'there': 0.35; 'characters': 0.35; 'header:User-Agent:1': 0.35; 'digit': 0.35; 'reply-to:addr:python.org': 0.35; 'feature': 0.36; 'url:python': 0.37; 'but': 0.38; 'languages': 0.38; 'url:org': 0.38; 'adds': 0.39; 'to:addr:python.org': 0.39; 'could': 0.39; 'would': 0.40; 'permit': 0.60; 'here:': 0.61; 'back': 0.61; 'special': 0.66; 'due': 0.67; 'reply-to:no real name:2**0': 0.72; 'header:Reply-To:1': 0.72; 'candide': 0.84; 'letters,': 0.91; 'universe': 0.91 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnkIAFRclk3Unw4S/2dsb2JhbACYZIx+d8EShWsEkHQ Date: Sat, 02 Apr 2011 00:15:22 +0100 From: MRAB User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: python-list@python.org Subject: Re: Extracting "true" words References: <4d963c1d$0$1584$426a34cc@news.free.fr> In-Reply-To: <4d963c1d$0$1584$426a34cc@news.free.fr> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: python-list@python.org List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 28 NNTP-Posting-Host: 82.94.164.166 X-Trace: 1301699738 news.xs4all.nl 81478 [::ffff:82.94.164.166]:39547 X-Complaints-To: abuse@xs4all.nl Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:2415 On 01/04/2011 21:55, candide wrote: > Back again with my study of regular expressions ;) There exists a > special character allowing alphanumeric extraction, the special > character \w (BTW, what the letter 'w' refers to?). But this feature > doesn't permit to extract true words; by "true" I mean word composed > only of _alphabetic_ letters (not digit nor underscore). > The 'w' refers to a 'word' character, although in regex it refers to letters, digits and the underscore character '_' due to its use in computer languages (basically, the characters of an identifier or name). > > So I was wondering what is the pattern to extract (or to match) _true_ > words ? Of course, I don't restrict myself to the ascii universe so that > the pattern [a-zA-Z]+ doesn't meet my needs. > Using the re module, you would have to create a character class out of all the possible letters, something like this: letter_class = u"[" + u"".join(unichr(c) for c in range(0x10000) if unichr(c).isalpha()) + u"]" Alternatively, you could try the new regex implementation here: http://pypi.python.org/pypi/regex which adds support for Unicode properties, and do something like this: words = regex.findall(ur"\p{Letter}+", unicode_text)