Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64102

Re: Guessing the encoding from a BOM

Show all headers | View raw

--------------------------------------------
On Thu, 1/16/14, Chris Angelico <rosuav@gmail.com> wrote:

 Subject: Re: Guessing the encoding from a BOM
 To: 
 Cc: "python-list@python.org" <python-list@python.org>
 Date: Thursday, January 16, 2014, 7:06 PM
 
 On Fri, Jan 17, 2014 at 5:01 AM,
 Björn Lindqvist <bjourne@gmail.com>
 wrote:
 > 2014/1/16 Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
 >> def guess_encoding_from_bom(filename, default):
 >>     with open(filename, 'rb')
 as f:
 >>         sig =
 f.read(4)
 >>     if
 sig.startswith((b'\xFE\xFF', b'\xFF\xFE')):
 >>         return
 'utf_16'
 >>     elif
 sig.startswith((b'\x00\x00\xFE\xFF', b'\xFF\xFE\x00\x00')):
 >>         return
 'utf_32'
 >>     else:
 >>         return
 default
 >
 > You might want to add the utf8 bom too:
 '\xEF\xBB\xBF'.
 
 I'd actually rather not. It would tempt people to pollute
 UTF-8 files
 with a BOM, which is not necessary unless you are MS
 Notepad.
 

 ===> Can you elaborate on that? Unless your utf-8 files will only contain ascii characters I do not understand why you would not want a bom utf-8.

Btw, isn't "read_encoding_from_bom" a better function name than "guess_encoding_from_bom"? I thought the point of BOMs was that there would be no more need to guess?

Thanks!

Albert-Jan

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Guessing the encoding from a BOM Albert-Jan Roskam <fomcl@yahoo.com> - 2014-01-16 11:37 -0800
  Re: Guessing the encoding from a BOM Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-17 01:18 +0000

csiph-web