Path: csiph.com!usenet.pasdenom.info!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=tY/D/wcqCAz7rgSXy/P0AJSI4QA4ERF0M+DCMv8/ZL2fyIL4mTNcxEfsgBF0dAN/VioYytxdq4h522tj+KwrmwRO05WJAOr+7CorK9RGKYYx1d+Hnsqkv1NX4WEVhdvnG6F8LJPzwhT3YoZgvBl0boPSk5XAU3WbVJ4h03aVA4o=;
Date: Thu, 16 Jan 2014 11:37:29 -0800 (PST)
From: Albert-Jan Roskam <fomcl@yahoo.com>
Subject: Re: Guessing the encoding from a BOM
To: Chris Angelico <rosuav@gmail.com>
In-Reply-To: <CAPTjJmqyO0UHrq31510iNeoQ2TcrJnosV0A6oHQOt5i-gz3njA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: "python-list@python.org" <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.5601.1389901239.18130.python-list@python.org>
Lines: 49
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:64102

--------------------------------------------
On Thu, 1/16/14, Chris Angelico <rosuav@gmail.com> wrote:

 Subject: Re: Guessing the encoding from a BOM
 To:=20
 Cc: "python-list@python.org" <python-list@python.org>
 Date: Thursday, January 16, 2014, 7:06 PM
=20
 On Fri, Jan 17, 2014 at 5:01 AM,
 Bj=F6rn Lindqvist <bjourne@gmail.com>
 wrote:
 > 2014/1/16 Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
 >> def guess_encoding_from_bom(filename, default):
 >>=A0 =A0=A0=A0with open(filename, 'rb')
 as f:
 >>=A0 =A0 =A0 =A0=A0=A0sig =3D
 f.read(4)
 >>=A0 =A0=A0=A0if
 sig.startswith((b'\xFE\xFF', b'\xFF\xFE')):
 >>=A0 =A0 =A0 =A0=A0=A0return
 'utf_16'
 >>=A0 =A0=A0=A0elif
 sig.startswith((b'\x00\x00\xFE\xFF', b'\xFF\xFE\x00\x00')):
 >>=A0 =A0 =A0 =A0=A0=A0return
 'utf_32'
 >>=A0 =A0=A0=A0else:
 >>=A0 =A0 =A0 =A0=A0=A0return
 default
 >
 > You might want to add the utf8 bom too:
 '\xEF\xBB\xBF'.
=20
 I'd actually rather not. It would tempt people to pollute
 UTF-8 files
 with a BOM, which is not necessary unless you are MS
 Notepad.
=20

 =3D=3D=3D> Can you elaborate on that? Unless your utf-8 files will only co=
ntain ascii characters I do not understand why you would not want a bom utf=
-8.

Btw, isn't "read_encoding_from_bom" a better function name than "guess_enco=
ding_from_bom"? I thought the point of BOMs was that there would be no more=
 need to guess?

Thanks!

Albert-Jan