Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64095

Re: Guessing the encoding from a BOM

Show all headers | View raw

On Fri, Jan 17, 2014 at 5:01 AM, Björn Lindqvist <bjourne@gmail.com> wrote:
> 2014/1/16 Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
>> def guess_encoding_from_bom(filename, default):
>>     with open(filename, 'rb') as f:
>>         sig = f.read(4)
>>     if sig.startswith((b'\xFE\xFF', b'\xFF\xFE')):
>>         return 'utf_16'
>>     elif sig.startswith((b'\x00\x00\xFE\xFF', b'\xFF\xFE\x00\x00')):
>>         return 'utf_32'
>>     else:
>>         return default
>
> You might want to add the utf8 bom too: '\xEF\xBB\xBF'.

I'd actually rather not. It would tempt people to pollute UTF-8 files
with a BOM, which is not necessary unless you are MS Notepad.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Guessing the encoding from a BOM Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-16 02:13 +0000
  Re: Guessing the encoding from a BOM Ben Finney <ben+python@benfinney.id.au> - 2014-01-16 14:47 +1100
    Re: Guessing the encoding from a BOM Steven D'Aprano <steve@pearwood.info> - 2014-01-16 06:55 +0000
      Re: Guessing the encoding from a BOM Ethan Furman <ethan@stoneleaf.us> - 2014-01-15 23:29 -0800
  Re: Guessing the encoding from a BOM Chris Angelico <rosuav@gmail.com> - 2014-01-16 16:01 +1100
    Re: Guessing the encoding from a BOM Steven D'Aprano <steve@pearwood.info> - 2014-01-16 06:45 +0000
  Re: Guessing the encoding from a BOM Ethan Furman <ethan@stoneleaf.us> - 2014-01-15 21:40 -0800
  Re: Guessing the encoding from a BOM Björn Lindqvist <bjourne@gmail.com> - 2014-01-16 19:01 +0100
  Re: Guessing the encoding from a BOM Chris Angelico <rosuav@gmail.com> - 2014-01-17 05:06 +1100
  Re: Guessing the encoding from a BOM Tim Chase <python.list@tim.thechases.com> - 2014-01-16 12:50 -0600

csiph-web