Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #87387 > unrolled thread

Odd ValueError using float

Back to article view | Back to comp.lang.python

Contents

  Odd ValueError using float emile <emile@fenx.com> - 2015-03-13 15:10 -0700
    Re: Odd ValueError using float Paul Rubin <no.email@nospam.invalid> - 2015-03-13 19:33 -0700
      Re: Odd ValueError using float Chris Angelico <rosuav@gmail.com> - 2015-03-14 14:09 +1100
      Re: Odd ValueError using float emile <emile@fenx.com> - 2015-03-14 08:28 -0700
      Re: Odd ValueError using float Chris Angelico <rosuav@gmail.com> - 2015-03-15 02:52 +1100
      Re: Odd ValueError using float Peter Otten <__peter__@web.de> - 2015-03-14 17:08 +0100
      Re: Odd ValueError using float emile <emile@fenx.com> - 2015-03-14 10:08 -0700
      Re: Odd ValueError using float emile <emile@fenx.com> - 2015-03-14 10:17 -0700
      Re: Odd ValueError using float Peter Otten <__peter__@web.de> - 2015-03-14 19:24 +0100
      Re: Odd ValueError using float emile <emile@fenx.com> - 2015-03-14 13:01 -0700
      Re: Odd ValueError using float Peter Otten <__peter__@web.de> - 2015-03-15 15:01 +0100
      Re: Odd ValueError using float emile <emile@fenx.com> - 2015-03-17 13:48 -0700

#87387 — Odd ValueError using float

On an older WinXP SP2 box with python2.6 that's been running this 
process fine for years, this week we're seeing:

 > c:\python26\lib\site-packages\fenx\sql_interface.py(144)normalized()
-> return val
(Pdb) val
*** NameError: name 'val' is not defined
(Pdb) l
139         try:
140             val = round(float(decval),1)
141         except:
142             import pdb; pdb.set_trace()
143         #val = round(float(int(decval)/10**((decval%100)*100)),1)
144  ->     return val
145
146
147     def foodList(reference):
148         # return a list of (code,val) tuples for the ingredients of 
the foodlist referenced
149         # local internal use only to create sqlFoodListsByID
(Pdb) decval
'4'
(Pdb) len(decval)
1
(Pdb) int(decval)
*** ValueError: invalid literal for int() with base 10: '41.700000000000003'
(Pdb)


Any ideas?

Thanks,

Emile

[toc] | [next] | [standalone]

#87403

emile <emile@fenx.com> writes:
> *** NameError: name 'val' is not defined
> (Pdb) l
> 139         try:
> 140             val = round(float(decval),1)
> 141         except:
> 142             import pdb; pdb.set_trace()

If 'float' or 'round' throw an exception, the assignment to 'val' never
happens, so 'val' is undefined.  Try examining the value of 'decval' in
the debugger to see what is making the conversion fail.

[toc] | [prev] | [next] | [standalone]

#87404

On Sat, Mar 14, 2015 at 1:33 PM, Paul Rubin <no.email@nospam.invalid> wrote:
> emile <emile@fenx.com> writes:
>> *** NameError: name 'val' is not defined
>> (Pdb) l
>> 139         try:
>> 140             val = round(float(decval),1)
>> 141         except:
>> 142             import pdb; pdb.set_trace()
>
> If 'float' or 'round' throw an exception, the assignment to 'val' never
> happens, so 'val' is undefined.  Try examining the value of 'decval' in
> the debugger to see what is making the conversion fail.

That's exactly what the OP did :) Trouble is, it didn't help, because
it sure looked as if decval was the string '4'. My best guess was -
and is - that it's not just a string. We're looking at an SQL
interface routine here, so it may be that there's a string subclass
that length-limits itself, on the assumption that it's going into a
fixed-length database field. Let's see if I can recreate the OP's
situation...

>>> def Char(maxlen):
    class CharN(str):
        def __repr__(self):
            return repr(self[:maxlen])
    CharN.__qualname__ = CharN.__name__ = "Char(%d)"%maxlen
    return CharN

>>> four = Char(1)('41.700000000000003')
>>> four
'4'
>>> int(four)
Traceback (most recent call last):
  File "<pyshell#28>", line 1, in <module>
    int(four)
ValueError: invalid literal for int() with base 10: '4'

... well, close. Anyway, a string subclass could probably do this by
accident somehow.

ChrisA

[toc] | [prev] | [next] | [standalone]

#87421

On 03/13/2015 08:09 PM, Chris Angelico wrote:
> On Sat, Mar 14, 2015 at 1:33 PM, Paul Rubin <no.email@nospam.invalid> wrote:
>> emile <emile@fenx.com> writes:
>>> *** NameError: name 'val' is not defined
>>> (Pdb) l
>>> 139         try:
>>> 140             val = round(float(decval),1)
>>> 141         except:
>>> 142             import pdb; pdb.set_trace()
>>
>> If 'float' or 'round' throw an exception, the assignment to 'val' never
>> happens, so 'val' is undefined.  Try examining the value of 'decval' in
>> the debugger to see what is making the conversion fail.
>
> That's exactly what the OP did :) Trouble is, it didn't help, because
> it sure looked as if decval was the string '4'. My best guess was -
> and is - that it's not just a string. We're looking at an SQL
> interface routine here, so it may be that there's a string subclass
> that length-limits itself, on the assumption that it's going into a
> fixed-length database field.

It ran almost to completion before generating the error again --

(Pdb) decval
'4'
(Pdb) type(decval)
<type 'str'>
(Pdb) len(decval)
1
(Pdb) int(decval)
*** ValueError: invalid literal for int() with base 10: '41.700000000000003'

So there's still something amiss.

Emile

[toc] | [prev] | [next] | [standalone]

#87423

On Sun, Mar 15, 2015 at 2:28 AM, emile <emile@fenx.com> wrote:
> It ran almost to completion before generating the error again --
>
> (Pdb) decval
> '4'
> (Pdb) type(decval)
> <type 'str'>
> (Pdb) len(decval)
> 1
> (Pdb) int(decval)
> *** ValueError: invalid literal for int() with base 10: '41.700000000000003'
>
> So there's still something amiss.

Compare these two lines' outputs:

print("str %d, int %d" % (id(str), id(int)))
print("str %d, int %d" % (id(type("")), id(type(0)))

Any difference in id would indicate that the names have been shadowed
- seems unlikely, but worth checking. And then, just to be absolutely
sure:

type(decval) is type("")

There is another, and very nasty, possibility. If you're working with
a C extension that has a refcount bug in it, all sorts of bizarre
things can happen - crucial type objects getting garbage collected,
objects getting disposed of and others taking their places, all kinds
of weird and wonderful things. Normally that'll eventually cause a
crash, but who knows...

ChrisA

[toc] | [prev] | [next] | [standalone]

#87426

emile wrote:

> On 03/13/2015 08:09 PM, Chris Angelico wrote:
>> On Sat, Mar 14, 2015 at 1:33 PM, Paul Rubin <no.email@nospam.invalid>
>> wrote:
>>> emile <emile@fenx.com> writes:
>>>> *** NameError: name 'val' is not defined
>>>> (Pdb) l
>>>> 139         try:
>>>> 140             val = round(float(decval),1)
>>>> 141         except:
>>>> 142             import pdb; pdb.set_trace()
>>>
>>> If 'float' or 'round' throw an exception, the assignment to 'val' never
>>> happens, so 'val' is undefined.  Try examining the value of 'decval' in
>>> the debugger to see what is making the conversion fail.
>>
>> That's exactly what the OP did :) Trouble is, it didn't help, because
>> it sure looked as if decval was the string '4'. My best guess was -
>> and is - that it's not just a string. We're looking at an SQL
>> interface routine here, so it may be that there's a string subclass
>> that length-limits itself, on the assumption that it's going into a
>> fixed-length database field.
> 
> It ran almost to completion before generating the error again --
> 
> (Pdb) decval
> '4'
> (Pdb) type(decval)
> <type 'str'>
> (Pdb) len(decval)
> 1
> (Pdb) int(decval)
> *** ValueError: invalid literal for int() with base 10:
> '41.700000000000003'
> 
> So there's still something amiss.

Why are you checking

int(decval)

when the actual failing expression is

round(float(decval),1)

? Try to evaluate the latter in the debugger, and if that gives no clue 
change

>>>> 139         try:
>>>> 140             val = round(float(decval),1)
>>>> 141         except:
>>>> 142             import pdb; pdb.set_trace()

to just

val = round(float(decval), 1)

to get a meaningful traceback and post that.

[toc] | [prev] | [next] | [standalone]

#87432

On 03/14/2015 09:08 AM, Peter Otten wrote:
> emile wrote:
>
>> On 03/13/2015 08:09 PM, Chris Angelico wrote:
>>> On Sat, Mar 14, 2015 at 1:33 PM, Paul Rubin <no.email@nospam.invalid>
>>> wrote:
>>>> emile <emile@fenx.com> writes:
>>>>> *** NameError: name 'val' is not defined
>>>>> (Pdb) l
>>>>> 139         try:
>>>>> 140             val = round(float(decval),1)
>>>>> 141         except:
>>>>> 142             import pdb; pdb.set_trace()
>>>>
>>>> If 'float' or 'round' throw an exception, the assignment to 'val' never
>>>> happens, so 'val' is undefined.  Try examining the value of 'decval' in
>>>> the debugger to see what is making the conversion fail.
>>>
>>> That's exactly what the OP did :) Trouble is, it didn't help, because
>>> it sure looked as if decval was the string '4'. My best guess was -
>>> and is - that it's not just a string. We're looking at an SQL
>>> interface routine here, so it may be that there's a string subclass
>>> that length-limits itself, on the assumption that it's going into a
>>> fixed-length database field.
>>
>> It ran almost to completion before generating the error again --
>>
>> (Pdb) decval
>> '4'
>> (Pdb) type(decval)
>> <type 'str'>
>> (Pdb) len(decval)
>> 1
>> (Pdb) int(decval)
>> *** ValueError: invalid literal for int() with base 10:
>> '41.700000000000003'
>>
>> So there's still something amiss.
>
> Why are you checking
>
> int(decval)


because it sure smells like int should work:

(Pdb) "3"<decval<"5"
True


>
> when the actual failing expression is
>
> round(float(decval),1)
>
> ? Try to evaluate the latter in the debugger, and if that gives no clue
> change
>
>>>>> 139         try:
>>>>> 140             val = round(float(decval),1)
>>>>> 141         except:
>>>>> 142             import pdb; pdb.set_trace()
>
> to just
>
> val = round(float(decval), 1)
>
> to get a meaningful traceback and post that.

I don't get a traceback -- it spews:

Fatal Python error: deletion of interned string failed

This application has requested the Runtime to terminate it in an unusual 
way.
Please contact the application's support team for more information.

then crashes and I get a Microsoft pop-up that says python.exe has 
encountered a problem and needs to close.

Emile

[toc] | [prev] | [next] | [standalone]

#87435

On 03/14/2015 08:52 AM, Chris Angelico wrote:
> On Sun, Mar 15, 2015 at 2:28 AM, emile <emile@fenx.com> wrote:
>> It ran almost to completion before generating the error again --
>>
>> (Pdb) decval
>> '4'
>> (Pdb) type(decval)
>> <type 'str'>
>> (Pdb) len(decval)
>> 1
>> (Pdb) int(decval)
>> *** ValueError: invalid literal for int() with base 10: '41.700000000000003'
>>
>> So there's still something amiss.
>
> Compare these two lines' outputs:
>
> print("str %d, int %d" % (id(str), id(int)))
> print("str %d, int %d" % (id(type("")), id(type(0)))
>

(Pdb) print("str %d, int %d" % (id(str), id(int)))
str 505366496, int 505399904
(Pdb) print("str %d, int %d" % (id(type("")), id(type(0))))
str 505366496, int 505399904



> Any difference in id would indicate that the names have been shadowed
> - seems unlikely, but worth checking. And then, just to be absolutely
> sure:
>
> type(decval) is type("")

(Pdb) type(decval) is type("")
True


> There is another, and very nasty, possibility. If you're working with
> a C extension that has a refcount bug in it, all sorts of bizarre
> things can happen - crucial type objects getting garbage collected,
> objects getting disposed of and others taking their places, all kinds
> of weird and wonderful things. Normally that'll eventually cause a
> crash, but who knows...

Here're the imports:

import os, time, re, shutil, subprocess
from fenx.BBxXlate.bbxfile import BBxFile      # a pure python module
from fenx import RealPyOdbc2

and of course, pdb when I break for the error condition.

Emile

[toc] | [prev] | [next] | [standalone]

#87438

emile wrote:

> On 03/14/2015 09:08 AM, Peter Otten wrote:
>> emile wrote:
>>
>>> On 03/13/2015 08:09 PM, Chris Angelico wrote:
>>>> On Sat, Mar 14, 2015 at 1:33 PM, Paul Rubin <no.email@nospam.invalid>
>>>> wrote:
>>>>> emile <emile@fenx.com> writes:
>>>>>> *** NameError: name 'val' is not defined
>>>>>> (Pdb) l
>>>>>> 139         try:
>>>>>> 140             val = round(float(decval),1)
>>>>>> 141         except:
>>>>>> 142             import pdb; pdb.set_trace()
>>>>>
>>>>> If 'float' or 'round' throw an exception, the assignment to 'val'
>>>>> never
>>>>> happens, so 'val' is undefined.  Try examining the value of 'decval'
>>>>> in the debugger to see what is making the conversion fail.
>>>>
>>>> That's exactly what the OP did :) Trouble is, it didn't help, because
>>>> it sure looked as if decval was the string '4'. My best guess was -
>>>> and is - that it's not just a string. We're looking at an SQL
>>>> interface routine here, so it may be that there's a string subclass
>>>> that length-limits itself, on the assumption that it's going into a
>>>> fixed-length database field.
>>>
>>> It ran almost to completion before generating the error again --
>>>
>>> (Pdb) decval
>>> '4'
>>> (Pdb) type(decval)
>>> <type 'str'>
>>> (Pdb) len(decval)
>>> 1
>>> (Pdb) int(decval)
>>> *** ValueError: invalid literal for int() with base 10:
>>> '41.700000000000003'
>>>
>>> So there's still something amiss.
>>
>> Why are you checking
>>
>> int(decval)
> 
> 
> because it sure smells like int should work:
> 
> (Pdb) "3"<decval<"5"
> True

That's a normal string comparison when decval is a string. This and the 
ValueError is expected Python behaviour:

Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> decval = "41.700000000000003"
>>> "3" < decval < "5"
True
>>> int(decval)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '41.700000000000003'


>> when the actual failing expression is
>>
>> round(float(decval),1)
>>
>> ? Try to evaluate the latter in the debugger, and if that gives no clue
>> change
>>
>>>>>> 139         try:
>>>>>> 140             val = round(float(decval),1)
>>>>>> 141         except:
>>>>>> 142             import pdb; pdb.set_trace()
>>
>> to just
>>
>> val = round(float(decval), 1)
>>
>> to get a meaningful traceback and post that.
> 
> I don't get a traceback -- it spews:
> 
> Fatal Python error: deletion of interned string failed
> 
> This application has requested the Runtime to terminate it in an unusual
> way.
> Please contact the application's support team for more information.
> 
> then crashes and I get a Microsoft pop-up that says python.exe has
> encountered a problem and needs to close.

That does look bad. Most likely an extension written in C corrupts the 
interpreter or it's even a bug in the interpreter itself.

[toc] | [prev] | [next] | [standalone]

#87444

On 03/14/2015 11:24 AM, Peter Otten wrote:
> emile wrote:
>
>> On 03/14/2015 09:08 AM, Peter Otten wrote:
<snip>
>>> Why are you checking
>>>
>>> int(decval)
>>
>>
>> because it sure smells like int should work:
>>
>> (Pdb) "3"<decval<"5"
>> True
>
> That's a normal string comparison when decval is a string. This and the
> ValueError is expected Python behaviour:


yes -- but i'd previously shown decval to have a length of 1, and how 
many things then fit that equation?

<snip>

>>> to get a meaningful traceback and post that.
>>
>> I don't get a traceback -- it spews:
>>
>> Fatal Python error: deletion of interned string failed
>>
>> This application has requested the Runtime to terminate it in an unusual
>> way.
>> Please contact the application's support team for more information.
>>
>> then crashes and I get a Microsoft pop-up that says python.exe has
>> encountered a problem and needs to close.
>
> That does look bad. Most likely an extension written in C corrupts the
> interpreter or it's even a bug in the interpreter itself.

I'm tight on time the rest of the day, but I think I'll next zap all the 
pyc versions, install a fresh 2.6.x python, and let it recompile.  About 
the only theory I have at the moment to explain the sudden failure after 
years of non-failure is disk based bit-rot.

Emile

[toc] | [prev] | [next] | [standalone]

#87472

emile wrote:

> On 03/14/2015 11:24 AM, Peter Otten wrote:
>> emile wrote:
>>
>>> On 03/14/2015 09:08 AM, Peter Otten wrote:
> <snip>
>>>> Why are you checking
>>>>
>>>> int(decval)
>>>
>>>
>>> because it sure smells like int should work:
>>>
>>> (Pdb) "3"<decval<"5"
>>> True
>>
>> That's a normal string comparison when decval is a string. This and the
>> ValueError is expected Python behaviour:
> 
> 
> yes -- but i'd previously shown decval to have a length of 1, and how
> many things then fit that equation?
> 
> <snip>
> 
>>>> to get a meaningful traceback and post that.
>>>
>>> I don't get a traceback -- it spews:
>>>
>>> Fatal Python error: deletion of interned string failed
>>>
>>> This application has requested the Runtime to terminate it in an unusual
>>> way.
>>> Please contact the application's support team for more information.
>>>
>>> then crashes and I get a Microsoft pop-up that says python.exe has
>>> encountered a problem and needs to close.
>>
>> That does look bad. Most likely an extension written in C corrupts the
>> interpreter or it's even a bug in the interpreter itself.
> 
> I'm tight on time the rest of the day, but I think I'll next zap all the
> pyc versions, install a fresh 2.6.x python, and let it recompile.  About
> the only theory I have at the moment to explain the sudden failure after
> years of non-failure is disk based bit-rot.

Probably not helpful, but I can provoke the behaviour you see by toggling 
bytes with ctypes, thus simulating a corrupted str object:

Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> s = "41.700000000000003"
>>> ctypes.c_ubyte.from_address(id(s) + 16).value = 1
>>> s
'4'
>>> len(s)
1
>>> float(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for float(): 41.700000000000003
>>> int(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '41.700000000000003'

[toc] | [prev] | [next] | [standalone]

#87648

On 03/15/2015 07:01 AM, Peter Otten wrote:
> Probably not helpful, but I can provoke the behaviour you see by toggling
> bytes with ctypes, thus simulating a corrupted str object:
>
> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
> [GCC 4.8.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import ctypes
>>>> s = "41.700000000000003"
>>>> ctypes.c_ubyte.from_address(id(s) + 16).value = 1
>>>> s
> '4'
>>>> len(s)
> 1
>>>> float(s)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ValueError: invalid literal for float(): 41.700000000000003
>>>> int(s)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ValueError: invalid literal for int() with base 10: '41.700000000000003'


I dug through the code and ctypes was in the mix.  Without discovering 
the actual issue, I did manage to work around the issue by replacing:

  val = round(float(decval),1)

with

  val = round(float("".join([ii if ii=="." else str(int(ii)) for ii in 
decval])),1)


Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web