Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <50e262a1$0$30003$c3e8da3$5496439d@news.astraweb.com>
References: <50e02485$0$3109$ba620e4c@news.skynet.be> <roy-4F8381.10201930122012@news.panix.com> <50e262a1$0$30003$c3e8da3$5496439d@news.astraweb.com>
Date: Wed, 2 Jan 2013 00:09:45 +0100
Subject: Re: ignore case only for a part of the regex?
From: Vlastimil Brom <vlastimil.brom@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1536.1357081794.29569.python-list@python.org>
Lines: 111
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:35928

2013/1/1 Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
> On Sun, 30 Dec 2012 10:20:19 -0500, Roy Smith wrote:
>
>> The way I would typically do something like this is build my regexes in
>> all lower case and .lower() the text I was matching against them.  I'm
>> curious what you're doing where you want to enforce case sensitivity in
>> one part of a header, but not in another.
>
> Well, sometimes you have things that are case sensitive, and other things
> which are not, and sometimes you need to match them at the same time. I
> don't think this is any more unusual than (say) wanting to match an
> otherwise lowercase word whether or not it comes at the start of a
> sentence:
>
> "[Pp]rogramming"
>
> is conceptually equivalent to "match case-insensitive `p`, and case-
> sensitive `rogramming`".
>
>
> By the way, although there is probably nothing you can (easily) do about
> this prior to Python 3.3, converting to lowercase is not the right way to
> do case-insensitive matching. It happens to work correctly for ASCII, but
> it is not correct for all alphabetic characters.
>
>
> py> 'Stra=C3=9Fe'.lower()
> 'stra=C3=9Fe'
> py> 'Stra=C3=9Fe'.upper()
> 'STRASSE'
>
>
> The right way is to casefold first, then match:
>
> py> 'Stra=C3=9Fe'.casefold()
> 'strasse'
>
>
> Curiously, there is an uppercase =C3=9F in old German. In recent years so=
me
> typographers have started using it instead of SS, but it's still rare,
> and the official German rules have =C3=9F transform into SS and vice vers=
a.
> It's in Unicode, but few fonts show it:
>
> py> unicodedata.lookup('LATIN CAPITAL LETTER SHARP S')
> '=E1=BA=9E'
>
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list

Hi,
just for completeness, the mentioned regex library can take care of
casfolding in case insensitive matching (in all supported versions:
Python 2.5-2.7 and 3.1-3.3); i.e.:
# case sensitive match:
>>> for m in regex.findall(ur"Stra=C3=9Fe", u" STRA=C3=9FE STRASSE STRA=E1=
=BA=9EE Strasse Stra=C3=9Fe "): print m
...
Stra=C3=9Fe

# case insensitive match:
>>> for m in regex.findall(ur"(?i)Stra=C3=9Fe", u" STRA=C3=9FE STRASSE STRA=
=E1=BA=9EE Strasse Stra=C3=9Fe "): print m
...
STRA=C3=9FE
STRA=E1=BA=9EE
Stra=C3=9Fe

# case insensitive match with casefolding:
>>> for m in regex.findall(ur"(?if)Stra=C3=9Fe", u" STRA=C3=9FE STRASSE STR=
A=E1=BA=9EE Strasse Stra=C3=9Fe "): print m
...
STRA=C3=9FE
STRASSE
STRA=E1=BA=9EE
Strasse
Stra=C3=9Fe
>>>
>>>

# after enabling the backwards incompatible modern matching behaviour,
casefolding is by default turned on for case insensitive matches
>>> for m in regex.findall(ur"(?V1i)Stra=C3=9Fe", u" STRA=C3=9FE STRASSE ST=
RA=E1=BA=9EE Strasse Stra=C3=9Fe "): print m
...
STRA=C3=9FE
STRASSE
STRA=E1=BA=9EE
Strasse
Stra=C3=9Fe
>>>


As a small addition, the originally posted pattern r'^Msg-(?:(?i)id):'
would actually work as expected in this modern matching mode in regex
- enabled with the V1 flag. In this case the flag-setting (?i) only
affects the following parts of the pattern, not the whole pattern like
in the current "re" and V0-compatibility-mode "regex"

>>> regex.findall(r"(?V1)Msg-(?:(?i)id):", "the regex should match  Msg-id:=
, Msg-Id:, ...  but not  msg-id:, MSG-ID:  and so on")
['Msg-id:', 'Msg-Id:']
>>>

regards,
      vbr