Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61657

Re: Movie (MPAA) ratings and Python?

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Wed, Dec 11, 2013 at 5:01 PM, Ned Batchelder <ned@nedbatchelder.com>wrote:

> On 12/11/13 6:39 PM, Dan Stromberg wrote:
>
>>
>> On Wed, Dec 11, 2013 at 3:24 PM, Steven D'Aprano
>> <steve+comp.lang.python@pearwood.info
>> <mailto:steve+comp.lang.python@pearwood.info>> wrote:
>>
>>     On Wed, 11 Dec 2013 15:07:35 -0800, Dan Stromberg wrote:
>>
>>      >  $ chardet mpaa-ratings-reasons.list
>>      > mpaa-ratings-reasons.list: windows-1255 (confidence: 0.97)
>>      >
>>      > I'm aware that chardet is playing guessing games, though one
>>     would hope
>>      > it would guess well most of the time, and give a reasonable
>>     confidence
>>      > rating.
>>
>>     What reason do you have for thinking that Windows-1255 isn't a
>>     reasonable
>>     guess? If the bulk of the text is Latin-1 except perhaps for one or
>> two
>>     Hebrew characters (or what chardet thinks are Hebrew characters), it
>> may
>>     actually be a reasonable guess.
>>
>>
>> I get a traceback if I try to read the file as Windows-1255.  I don't
>> get a traceback if I read it as ISO-8859-1.
>>
>>     If it is a poor guess, perhaps you ought to report it to the chardet
>>     maintainers as a good example of a poor guess.
>>
>> I was considering that, and may do so.
>>
>> I've also been wondering if ISO-8859-1 is just an octet-oriented codec,
>> so it'll read about anything.  There are clearly non-7-bit-ASCII
>> characters in the file that look like line noise in an mrxvt.
>>
>
> Both ISO-8859-1 and Windows-1255 are octet-oriented, I don't see why one
> would raise an exception when the other didn't.  Unless the exception isn't
> on the decode, but instead on your attempt to output the result. Can you
> show the full traceback you're seeing?
>

$ ./movie-ratings
Traceback (most recent call last):
  File "./movie-ratings", line 85, in <module>
    main()
  File "./movie-ratings", line 68, in main
    ratings =
get_ratings('/home/dstromberg/src/home-svn/movie-ratings/trunk/mpaa-ratings-reasons.list')
  File "./movie-ratings", line 52, in get_ratings
    for line in ratings_file:
  File "/usr/local/cpython-3.3/lib/python3.3/encodings/cp1255.py", line 23,
in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0xfc in position
1225: character maps to <undefined>

BTW, other than satisfying our respective curiosities, I consider this
project finished.  It's probably not getting ratings for my entire movie
collection, but it is getting them for a significant fraction, which is all
I was really looking for.  Now I know which ones are rated PG, so I can
decide whether to let my 8 year old watch them.

This is with cpython-3.3.

Thanks.  ^_^

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Movie (MPAA) ratings and Python? Dan Stromberg <drsalists@gmail.com> - 2013-12-11 15:07 -0800
  Re: Movie (MPAA) ratings and Python? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-11 23:24 +0000
    Re: Movie (MPAA) ratings and Python? Dan Stromberg <drsalists@gmail.com> - 2013-12-11 15:39 -0800
    Re: Movie (MPAA) ratings and Python? Ned Batchelder <ned@nedbatchelder.com> - 2013-12-11 20:01 -0500
    Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) Ben Finney <ben+python@benfinney.id.au> - 2013-12-12 12:12 +1100
      Re: Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) rusi <rustompmody@gmail.com> - 2013-12-11 19:23 -0800
        Re: Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) Chris Angelico <rosuav@gmail.com> - 2013-12-12 15:27 +1100
        Re: Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-12 11:05 +0000
          Re: Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) Steve Hayes <hayesstw@telkomsa.net> - 2013-12-12 15:36 +0200
    Re: Disable HTML in forum messages (was: Movie (MPAA) ratings and Python?) Ian Kelly <ian.g.kelly@gmail.com> - 2013-12-11 18:31 -0700
    Re: Movie (MPAA) ratings and Python? Ian Kelly <ian.g.kelly@gmail.com> - 2013-12-11 18:37 -0700
    Re: Movie (MPAA) ratings and Python? Dan Stromberg <drsalists@gmail.com> - 2013-12-11 19:52 -0800
    Re: Movie (MPAA) ratings and Python? Michael Torrie <torriem@gmail.com> - 2013-12-11 23:22 -0700
    Re: Movie (MPAA) ratings and Python? Dave Angel <davea@davea.name> - 2013-12-12 08:56 -0500

csiph-web