Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #97042

Re: Readlines returns non ASCII character

Show all headers | View raw

On Wed, Sep 23, 2015 at 6:47 AM, SANKAR . <shankarphy@gmail.com> wrote:
> Hi all,
>
> I am not a expert programmer but I have to extract information from a large
> file.
>  I used  codecs.open(..) with UTF16 encoding to read this file. It could
> read all the lines in the file but returns with the non Ascii characters.
> Below are 5 sample lines. How do I avoid having this non Ascii items. Is
> there a better way to read this?

I suspect that what you want is not "non-ASCII" but just to read the
file without all the mojibake, which is likely an indication that
you're using the wrong encoding.

Do you know that UTF-16 is actually the encoding of the file?

Based on the spaces that appear between adjacent characters, I would
guess that this is probably in a 32-bit encoding, perhaps UTF-32. On
the other hand, the repeated 0x00ff 0x00fe 0x00ff are very curious; I
don't see how that could be valid UTF-32. Are you sure that this is a
text file and not some propietary binary data format?

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Readlines returns non ASCII character Ian Kelly <ian.g.kelly@gmail.com> - 2015-09-23 10:07 -0600

csiph-web