Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110037 > unrolled thread

Contradictory error messages in Python 3.4 - standard library issue!

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Contradictory error messages in Python 3.4 - standard library issue! Harrison Chudleigh <harrison.chudleigh1@education.nsw.gov.au> - 2016-06-17 08:25 +1000
    Re: Contradictory error messages in Python 3.4 - standard library issue! Steven D'Aprano <steve@pearwood.info> - 2016-06-17 11:58 +1000

#110037 — Contradictory error messages in Python 3.4 - standard library issue!

While working on a program, I ran into an error with the usage of the
module tokenize. The following message was displayed.
 File
"/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py",
line 467, in tokenize
    encoding, consumed = detect_encoding(readline)
  File
"/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py",
line 409, in detect_encoding
    if first.startswith(BOM_UTF8):
TypeError: startswith first arg must be str or a tuple of str, not bytes
Undaunted, I changed the error on line 409. The line then read:

if first.startswith(BOM_UTF8):
*******************************************************************************
This message is intended for the addressee named and may contain privileged information or confidential information or both. If you are not the intended recipient please delete it and notify the sender.

[toc] | [next] | [standalone]

#110042

Hi Harrison, and welcome!


On Fri, 17 Jun 2016 08:25 am, Harrison Chudleigh wrote:

> While working on a program, I ran into an error with the usage of the
> module tokenize.

So you have an error with the use of tokenize. Fair enough.

But why do you imagine that the errors lies in the module itself, rather
that your use of it? There are probably tens of thousands, maybe hundreds
of thousands, of people using that module (directly or indirectly). The
chances that you have identified a bug in the module that nobody else has
seen before is pretty slim.

And that certainly shouldn't be your first assumption.


> The following message was displayed. 
>  File
> "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py",
> line 467, in tokenize
>     encoding, consumed = detect_encoding(readline)
>   File
> "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py",
> line 409, in detect_encoding
>     if first.startswith(BOM_UTF8):
> TypeError: startswith first arg must be str or a tuple of str, not bytes
> Undaunted, I changed the error on line 409. The line then read:
> 
> if first.startswith(BOM_UTF8):

I'm not seeing any actual difference between the before and after:

    if first.startswith(BOM_UTF8):
    if first.startswith(BOM_UTF8):

but in another email, you state that you changed it to:

    if first.startswith('BOM_UTF8'):


Changing the source code of the standard library is almost never what you
want to be do. I've been programming in Python for 20+ years, and the
number of times I've edited the source code of the standard library to fix
a program is exactly zero.

But making random changes to the source code of the standard library without
understanding what it is doing or why is NEVER what you want to do.

All you have accomplished by editing the code is changing the situation
from "one in a 100,000 chance of a bug in the standard library"
to "certainly a bug in a non-standard module shared by absolutely nobody
else in the world".

Before we can help you, you MUST revert the module back to the way it was.
You have introduced a bug to the module, but worse, you've now made it so
that it is different from everyone else's tokenize module. Any errors you
have received after making that change, or changes, are irrelevant.

Then you need to show us how you are calling tokenize. Show us the ENTIRE
traceback, starting with the line beginning "Traceback", not just the last
line with the error message. Once we've seen that, we may be able to help
you, or we may have more questions, but that's the bare minimum we need.



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web