Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64147 > unrolled thread

Re: Guessing the encoding from a BOM

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Guessing the encoding from a BOM Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-17 09:10 +0000

#64147 — Re: Guessing the encoding from a BOM

On 17/01/2014 01:40, Tim Chase wrote:
> On 2014-01-17 11:14, Chris Angelico wrote:
>> UTF-8 specifies the byte order
>> as part of the protocol, so you don't need to mark it.
>
> You don't need to mark it when writing, but some idiots use it
> anyway.  If you're sniffing a file for purposes of reading, you need
> to look for it and remove it from the actual data that gets returned
> from the file--otherwise, your data can see it as corruption.  I end
> up with lots of CSV files from customers who have polluted it with
> Notepad or had Excel insert some UTF-8 BOM when exporting.  This
> means my first column-name gets the BOM prefixed onto it when the
> file is passed to csv.DictReader, grr.
>
> -tkc
>

My code that used to handle CSV files from M$ Money had to allow for a 
single NUL byte right at the end of the file.  Thankfully I've now moved 
on to gnucash.

Slight aside, any chance of changing the subject of this thread, or even 
ending the thread completely?  Why?  Every time I see it I picture 
Inspector Clouseau, "A BOM!!!" :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web