Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #196972 > unrolled thread

Re: Two aces up Python's sleeve

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Two aces up Python's sleeve Mild Shock <janburse@fastmail.fm> - 2024-11-06 17:27 +0100
    Re: Two aces up Python's sleeve Annada Behera <annada@tilde.green> - 2024-11-07 12:55 +0530
      Re: Two aces up Python's sleeve Mild Shock <janburse@fastmail.fm> - 2024-11-07 15:04 +0100
        Re: Two aces up Python's sleeve Greg Ewing <greg.ewing@canterbury.ac.nz> - 2024-11-08 11:15 +1300
          Re: Two aces up Python's sleeve dn <PythonList@DancesWithMice.info> - 2024-11-08 13:07 +1300
          Re: Two aces up Python's sleeve Mild Shock <janburse@fastmail.fm> - 2024-11-08 01:25 +0100
            Re: Two aces up Python's sleeve Mild Shock <janburse@fastmail.fm> - 2024-11-08 01:29 +0100
              Re: Two aces up Python's sleeve Mild Shock <janburse@fastmail.fm> - 2024-11-08 01:47 +0100
      Re: Two aces up Python's sleeve (Posting On Python-List Prohibited) Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-11-08 01:10 +0000
        Re: Two aces up Python's sleeve (Posting On Python-List Prohibited) Mild Shock <janburse@fastmail.fm> - 2024-11-08 02:40 +0100
          Re: Two aces up Python's sleeve (Posting On Python-List Prohibited) dn <PythonList@DancesWithMice.info> - 2024-11-09 08:09 +1300
            Re: Two aces up Python's sleeve (Posting On Python-List Prohibited) Mild Shock <janburse@fastmail.fm> - 2024-11-08 20:49 +0100
          Re: Two aces up Python's sleeve (Posting On Python-List Prohibited) Thomas Passin <list1@tompassin.net> - 2024-11-08 17:00 -0500

#196972 — Re: Two aces up Python's sleeve

Then please explain why I have to write:

    i += 1

Instead of the shorter:

    i ++

My short-term memory is really stressed.

Stefan Ram schrieb:
> ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>> last_item = my_list[ -1 ]
>> Way cleaner than my_list[ len( my_list )- 1 ], don't you think?
> 
>    In "The Mental Game of Python," Raymond Hettinger spills the
>    beans about our noggins only being able to juggle 7 +/- 2
>    things in our short-term memory.
> 
>    So, "last_item = my_list[ -1 ]" might still make the cut,
>    while "my_list[ len( my_list)- 1 ]" could be biting off
>    more than we can chew.
> 
> |The problem is, the number of brain registers this uses is
> |10. This is no longer a decryption effort. This is a puzzle.
> |At the moment you put it together, you fully understand it.
> |But if this is embedded in bigger code, every time you hit
> |this line, you're going to have to pick apart "what does this
> |thing do?".
> Raymond Hettinger
> 
> 

[toc] | [next] | [standalone]

#196973

>Then please explain why I have to write:
>
>    i += 1
>
>Instead of the shorter:
>
>    i ++
>
>My short-term memory is really stressed.

I heard this behavior is because python's integers are immutable.
For example:

    >>> x,y = 5,5
    >>> id(x) == id(y)
    True

5 is a object that x and y points to. ++x or x++ will redefine 5 to
6, which the interpreter forbids to keep it's state mathematically
consistent. Also, by not supporting x++ and ++x, it avoids the pre-
and post-increment (substitute-increment v. increment-substitute) bugs
that plagues C and it's children.

[toc] | [prev] | [next] | [standalone]

#196976

This only works for small integers. I guess
this is because tagged pointers are used
nowadays ? For large integers, also known

as bigint, it doesn't work:

Python 3.13.0a1 (tags/v3.13.0a1:ad056f0, Oct 13 2023, 09:51:17)

 >>> x, y = 5, 4+1
 >>> id(x) == id(y)
True

 >>> x, y = 10**200, 10**199*10
 >>> x == y
True
 >>> id(x) == id(y)
False

In tagged pointers a small integer is
directly inlined into the pointer. The
pointer has usually some higher bits,

that identify the type and when masking
to see the lower bits, one gets the
integer value.

But I don't know for sure whats going on,
would need to find a CPython documentation.

P.S.: I also tested PyPy it doesn't show
the same behaviour, because it computes
an exaberated id():

Python 3.10.14 (39dc8d3c85a7, Aug 27 2024, 14:33:33)
[PyPy 7.3.17 with MSC v.1929 64 bit (AMD64)]
 >>>> x, y = 5, 4+1
 >>>> id(x) == id(y)
True

 >>>> x, y = 10**200, 10**199*10
 >>>> id(x) == id(y)
True
 >>>> id(x)
1600000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
000000000000000001

Quite funny!

Annada Behera schrieb:
>> Then please explain why I have to write:
>>
>>      i += 1
>>
>> Instead of the shorter:
>>
>>      i ++
>>
>> My short-term memory is really stressed.
> 
> I heard this behavior is because python's integers are immutable.
> For example:
> 
>      >>> x,y = 5,5
>      >>> id(x) == id(y)
>      True
> 
> 5 is a object that x and y points to. ++x or x++ will redefine 5 to
> 6, which the interpreter forbids to keep it's state mathematically
> consistent. Also, by not supporting x++ and ++x, it avoids the pre-
> and post-increment (substitute-increment v. increment-substitute) bugs
> that plagues C and it's children.
> 
> 

[toc] | [prev] | [next] | [standalone]

#196977

On 8/11/24 3:04 am, Mild Shock wrote:
> This only works for small integers. I guess
> this is because tagged pointers are used
> nowadays ?

No, it's because integers in a certain small range are cached. Not sure 
what the actual range is nowadays, it used to be something like -5 to 
256 I think.

BTW you have to be careful testing this, because the compiler sometimes 
does constant folding, so you need to be sure it's actually computing 
the numbers at run time.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#196978

On 8/11/24 11:15, Greg Ewing via Python-list wrote:
> On 8/11/24 3:04 am, Mild Shock wrote:
>> This only works for small integers. I guess
>> this is because tagged pointers are used
>> nowadays ?
> 
> No, it's because integers in a certain small range are cached. Not sure 
> what the actual range is nowadays, it used to be something like -5 to 
> 256 I think.
> 
> BTW you have to be careful testing this, because the compiler sometimes 
> does constant folding, so you need to be sure it's actually computing 
> the numbers at run time.

Haven't seen the OP. Is the Newsgroup link forwarding to the email-list 
correctly?


Integer interning is indeed valid for -5 <= i <= 256
("it works on my machine"! see below)

 >>> a = 0; b = 0; c = 0; d = 0
 >>> while a is b:
...     print( a, b, end=" ", )
...     print( c, d, ) if c is d else print()
...     a += 1; b += 1; c -= 1; d -= 1
...
0 0 0 0
1 1 -1 -1
2 2 -2 -2
3 3 -3 -3
4 4 -4 -4
5 5 -5 -5
6 6
7 7
8 8
9 9
...
254 254
255 255
256 256
 >>>

Be aware that this is implementation-dependent and not guaranteed to 
hold forever.

  dn  ~  python
Python 3.12.7 (main, Oct  1 2024, 00:00:00) [GCC 13.3.1 20240913 (Red 
Hat 13.3.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.


See also https://docs.python.org/3/library/sys.html#sys.intern

Thus could decide what is interned for yourself:

a_string = sys.intern( str( 1000 ) )

-- 
Regards,
=dn

[toc] | [prev] | [next] | [standalone]

#196979

Hi,

In Java its possible to work this way
with the Integer datatype, just call
Integer.valueOf().

I am not sure whether CPython does the
same. Because it shows me the same behaviour
for small integers that are more than

only in the range -128 to 128. You can try yourself:

Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
 >>> x,y = 10**10, 10**9*10
 >>> id(x) == id(y)
True

Maybe the idea that objects have an address
that can be accessed via id() has been abandoned.
This is already seen in PyPy. So maybe we

are falsly assuming that id() gives na object address.

Greg Ewing schrieb:
> On 8/11/24 3:04 am, Mild Shock wrote:
>> This only works for small integers. I guess
>> this is because tagged pointers are used
>> nowadays ?
> 
> No, it's because integers in a certain small range are cached. Not sure 
> what the actual range is nowadays, it used to be something like -5 to 
> 256 I think.
> 
> BTW you have to be careful testing this, because the compiler sometimes 
> does constant folding, so you need to be sure it's actually computing 
> the numbers at run time.
> 

[toc] | [prev] | [next] | [standalone]

#196980

For example this article:

https://www.codementor.io/@arpitbhayani/python-caches-integers-16jih595jk

about the integer singletons claims:

 >>> x, y = 257, 257
 >>> id(x) == id(y)
False

But on Windows my recent CPython doesn't do that:

Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
 >>> x, y = 257, 257
 >>> id(x) == id(y)
True

Mild Shock schrieb:
> Hi,
> 
> In Java its possible to work this way
> with the Integer datatype, just call
> Integer.valueOf().
> 
> I am not sure whether CPython does the
> same. Because it shows me the same behaviour
> for small integers that are more than
> 
> only in the range -128 to 128. You can try yourself:
> 
> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>  >>> x,y = 10**10, 10**9*10
>  >>> id(x) == id(y)
> True
> 
> Maybe the idea that objects have an address
> that can be accessed via id() has been abandoned.
> This is already seen in PyPy. So maybe we
> 
> are falsly assuming that id() gives na object address.
> 
> Greg Ewing schrieb:
>> On 8/11/24 3:04 am, Mild Shock wrote:
>>> This only works for small integers. I guess
>>> this is because tagged pointers are used
>>> nowadays ?
>>
>> No, it's because integers in a certain small range are cached. Not 
>> sure what the actual range is nowadays, it used to be something like 
>> -5 to 256 I think.
>>
>> BTW you have to be careful testing this, because the compiler 
>> sometimes does constant folding, so you need to be sure it's actually 
>> computing the numbers at run time.
>>
> 

[toc] | [prev] | [next] | [standalone]

#196981

The wiked brain of ChatGPT gives me a lead:

PEP 659
Storing data caches before the bytecode.

Maybe its an effect of constant folding
and constant pooling by the compiler?

Mild Shock schrieb:
> 
> For example this article:
> 
> https://www.codementor.io/@arpitbhayani/python-caches-integers-16jih595jk
> 
> about the integer singletons claims:
> 
>  >>> x, y = 257, 257
>  >>> id(x) == id(y)
> False
> 
> But on Windows my recent CPython doesn't do that:
> 
> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>  >>> x, y = 257, 257
>  >>> id(x) == id(y)
> True
> 
> Mild Shock schrieb:
>> Hi,
>>
>> In Java its possible to work this way
>> with the Integer datatype, just call
>> Integer.valueOf().
>>
>> I am not sure whether CPython does the
>> same. Because it shows me the same behaviour
>> for small integers that are more than
>>
>> only in the range -128 to 128. You can try yourself:
>>
>> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>>  >>> x,y = 10**10, 10**9*10
>>  >>> id(x) == id(y)
>> True
>>
>> Maybe the idea that objects have an address
>> that can be accessed via id() has been abandoned.
>> This is already seen in PyPy. So maybe we
>>
>> are falsly assuming that id() gives na object address.
>>
>> Greg Ewing schrieb:
>>> On 8/11/24 3:04 am, Mild Shock wrote:
>>>> This only works for small integers. I guess
>>>> this is because tagged pointers are used
>>>> nowadays ?
>>>
>>> No, it's because integers in a certain small range are cached. Not 
>>> sure what the actual range is nowadays, it used to be something like 
>>> -5 to 256 I think.
>>>
>>> BTW you have to be careful testing this, because the compiler 
>>> sometimes does constant folding, so you need to be sure it's actually 
>>> computing the numbers at run time.
>>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#196982 — Re: Two aces up Python's sleeve (Posting On Python-List Prohibited)

On Thu, 07 Nov 2024 12:55:53 +0530, Annada Behera wrote:

> I heard this behavior is because python's integers are immutable.

Nothing to do with that.

> ++x or x++ will redefine 5 to 6, which the interpreter forbids ...

One of those is actually syntactically valid.

It just won’t do what you expect it to do.

[toc] | [prev] | [next] | [standalone]

#196983 — Re: Two aces up Python's sleeve (Posting On Python-List Prohibited)

Well you can use your Browser, since
JavaScript understand post and pre increment:

 > x = 5
5
 > x ++
5
 > x = 5
5
 > ++ x
6

So we have x ++ equals in Python:

     x + = 1
     x - 1

And ++ x equals in Python:

     x += 1
     x

But I don't know how to combine an
assignment and an expression into one
expession. In JavaScript one can use

the comma:

 > x = 5
5
 > y = (x += 1, x - 1)
5
 > x = 5
5
 > y = (x += 1, x)
6

But in Python the comma would create a tuple.

Lawrence D'Oliveiro schrieb:
> On Thu, 07 Nov 2024 12:55:53 +0530, Annada Behera wrote:
> 
>> I heard this behavior is because python's integers are immutable.
> 
> Nothing to do with that.
> 
>> ++x or x++ will redefine 5 to 6, which the interpreter forbids ...
> 
> One of those is actually syntactically valid.
> 
> It just won’t do what you expect it to do.
> 

[toc] | [prev] | [next] | [standalone]

#196984 — Re: Two aces up Python's sleeve (Posting On Python-List Prohibited)

On 8/11/24 14:40, Mild Shock via Python-list wrote:
> Well you can use your Browser, since
> JavaScript understand post and pre increment:

Question: are we talking Python or JavaScript?


> So we have x ++ equals in Python:

Trying to find a word-for-word translation serves as badly in 
computer-programming languages as it does in human spoken-languages. 
Learn how to adapt and embrace the differences...


>      x + = 1
>      x - 1

The above probably only 'works' (the way you expect) in the REPL.


> But I don't know how to combine an
> assignment and an expression into one
> expession. In JavaScript one can use

Again!

"Everything should be made as simple as possible, but no simpler."

Check out "The Zen of Python" and PEP-0008 for Python idioms.


> the comma:
> 
>  > x = 5
> 5
>  > y = (x += 1, x - 1)
> 5
>  > x = 5
> 5
>  > y = (x += 1, x)
> 6
> 
> But in Python the comma would create a tuple.

Exactly, just as driving on the left side of the road will be fine in 
some countries but cause a crash in others. Learn the local rules FIRST!


The 'walrus operator' could be applied:

 >>> x = 5
 >>> y = (x := x + 1); x
6
 >>> x, y
(6, 6)

However, if such were submitted for Code Review, unhappiness would result.


Was the question re-phrased to: how to ... in Python, we'd end-up with 
something more like this:

 >>> x = 5  # define
 >>> x += 1  # increment
 >>> y = x  # alias
 >>> x, y
(6, 6)

-- 
Regards,
=dn

[toc] | [prev] | [next] | [standalone]

#196985 — Re: Two aces up Python's sleeve (Posting On Python-List Prohibited)

Ok here you go, the "walrus operator" is
actually a good lead, we have that this
here from JavaScript:

    x++

    respectively

    ++x

Can be replaced by the Python expression:

    (x := x + 1) - 1

    respectively

    (x := x + 1)

Here is a test only testing x++:

Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
 >>> x = 5
 >>> (x := x + 1) - 1
5
 >>> x
6

dn schrieb:
... irrational drivel removed ..> The 'walrus operator' could be applied:
> 
>  >>> x = 5
>  >>> y = (x := x + 1); x
> 6
>  >>> x, y
> (6, 6)... irrational drivel removed ..

[toc] | [prev] | [next] | [standalone]

#196986 — Re: Two aces up Python's sleeve (Posting On Python-List Prohibited)

On 11/8/2024 2:09 PM, dn via Python-list wrote:
> On 8/11/24 14:40, Mild Shock via Python-list wrote:
>> Well you can use your Browser, since
>> JavaScript understand post and pre increment:
> 
> Question: are we talking Python or JavaScript?
> 
> 
>> So we have x ++ equals in Python:
> 
> Trying to find a word-for-word translation serves as badly in computer- 
> programming languages as it does in human spoken-languages. Learn how to 
> adapt and embrace the differences...
> 
> 
>>      x + = 1
>>      x - 1
> 
> The above probably only 'works' (the way you expect) in the REPL.
> 
> 
>> But I don't know how to combine an
>> assignment and an expression into one
>> expession. In JavaScript one can use
> 
> Again!
> 
> "Everything should be made as simple as possible, but no simpler."
> 
> Check out "The Zen of Python" and PEP-0008 for Python idioms.
> 
> 
>> the comma:
>>
>>  > x = 5
>> 5
>>  > y = (x += 1, x - 1)
>> 5
>>  > x = 5
>> 5
>>  > y = (x += 1, x)
>> 6
>>
>> But in Python the comma would create a tuple.
> 
> Exactly, just as driving on the left side of the road will be fine in 
> some countries but cause a crash in others. Learn the local rules FIRST!
> 
> 
> The 'walrus operator' could be applied:
> 
>  >>> x = 5
>  >>> y = (x := x + 1); x
> 6
>  >>> x, y
> (6, 6)
> 
> However, if such were submitted for Code Review, unhappiness would result.
> 
> 
> Was the question re-phrased to: how to ... in Python, we'd end-up with 
> something more like this:
> 
>  >>> x = 5  # define
>  >>> x += 1  # increment
>  >>> y = x  # alias
>  >>> x, y
> (6, 6)

Or, still Pythonic but simpler:

 >>> x = 5
 >>> y = x = x + 1
 >>> x, y
(6, 6)

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web