Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #77780 > unrolled thread

Re: Thread-ID - how much could be?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Thread-ID - how much could be? Ervin Hegedüs <airween@gmail.com> - 2014-09-11 23:32 +0200
    Re: Thread-ID - how much could be? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-09-12 11:29 +1000
      Re: Thread-ID - how much could be? Chris Angelico <rosuav@gmail.com> - 2014-09-12 11:49 +1000
      Re: Thread-ID - how much could be? Skip Montanaro <skip@pobox.com> - 2014-09-11 20:38 -0500
      Re: Thread-ID - how much could be? Cameron Simpson <cs@zip.com.au> - 2014-09-12 13:41 +1000
      Re: Thread-ID - how much could be? Chris Angelico <rosuav@gmail.com> - 2014-09-12 15:41 +1000
      Re: Thread-ID - how much could be? Ervin Hegedüs <airween@gmail.com> - 2014-09-12 08:30 +0200
      Re: Thread-ID - how much could be? dieter <dieter@handshake.de> - 2014-09-13 08:56 +0200
      Re: Thread-ID - how much could be? Peter Otten <__peter__@web.de> - 2014-09-13 09:48 +0200
      Re: Thread-ID - how much could be? Martin Skjöldebrand <shieldfire@gmail.com> - 2014-09-13 12:09 +0200
      Re: Thread-ID - how much could be? Ervin Hegedüs <airween@gmail.com> - 2014-09-13 12:32 +0200
    Re: Thread-ID - how much could be? Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2014-09-16 10:53 +0200

#77780 — Re: Thread-ID - how much could be?

Hi Peter,

thanks for the reply,

On Thu, Sep 11, 2014 at 09:48:18PM +0200, Peter Otten wrote:
> Ervin Hegedüs wrote:
> 
> > Exception in thread Thread-82:
> > ...
> > My question is: how much thread ID could be totally? Is there any
> > maximum number? And if the thread reached that, what will be
> > done? Overlflowed? Couting from 0 again?
> 
> A quick peak into threading.py reveals
> 
> # Helper to generate new thread names
> _counter = 0
> def _newname(template="Thread-%d"):
>     global _counter
>     _counter += 1
>     return template % _counter
> 
> class Thread:
>     ...
>     def __init__(self, group=None, target=None, name=None,
>                  args=(), kwargs=None, *, daemon=None):
>         ...
>         self._name = str(name or _newname())
> 
> 
> There is no upper limit to the thread name other than that you will 
> eventually run out of memory ;)

thanks - I hope that the memory will not run out by these
threads... :)

Anyway, that means, on my system:

>>> import sys
>>> print sys.maxint
9223372036854775807

the couter could be 9223372036854775807?

And after? :)


Thanks,


a.



 

-- 
I � UTF-8

[toc] | [next] | [standalone]

#77787

Ervin Hegedüs wrote:

[...]
>> > My question is: how much thread ID could be totally? Is there any
>> > maximum number? And if the thread reached that, what will be
>> > done? Overlflowed? Couting from 0 again?
[...]
>> There is no upper limit to the thread name other than that you will
>> eventually run out of memory ;)
> 
> thanks - I hope that the memory will not run out by these
> threads... :)
> 
> Anyway, that means, on my system:
> 
>>>> import sys
>>>> print sys.maxint
> 9223372036854775807
> 
> the couter could be 9223372036854775807?
> 
> And after? :)

Suppose you somehow managed to create 9223372036854775807 threads. If your
computer has 16 GB of RAM available, that means that at most each thread
can use:

py> 16*1024*1024*1024/9223372036854775807
1.862645149230957e-09

bytes. That's not allowing any memory for the rest of your program, the
Python interpreter, other processes, or the operating system. That is less
than a single bit and clearly is impossible. You will run out of memory
*long* before reaching 9223372036854775807 threads running at once.

Suppose you don't have all the threads running at once, but you create and
destroy the threads as quickly as possible, so there is never more than one
thread alive at a time (plus the main interpreter thread). Suppose you have
an extremely fast computer that can create and destroy a billion threads
per second, one thread per nanosecond. Then your program would need to run
for:

py> 9223372036854775807*1e-9/60/60/24/365
292.471208677536

years non-stop before reaching a count of sys.maxint. I know that some Linux
systems can have an uptime over a year, perhaps even two years, but I think
that nearly 300 years is asking a bit much. Your hardware probably won't
keep working that long.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#77788

On Fri, Sep 12, 2014 at 11:29 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I know that some Linux
> systems can have an uptime over a year, perhaps even two years, but I think
> that nearly 300 years is asking a bit much. Your hardware probably won't
> keep working that long.

I've had over two years of uptime. Currently looking at 85 wk 4d
02:11:28 since the UPS and power failed simultaneously, but before
that, over two years.

But what about a 32-bit build? You could blow 1<<31 in about a month
of milliseconds, and I just tried, and on this 32-bit Windows box I
have here, I can start 10K threads in under a second:

>>> def test(n):
    def f(): pass
    start=time.time()
    for i in range(n):
        t=threading.Thread(target=f)
    return (time.time()-start)/n
>>> test(10000)
6.562459468841553e-05
>>> test(100000)
6.906198978424073e-05

So if it's 7e-5 seconds per thread (65-69 microseconds), that'd be
less than two days to blow a 32-bit maxint. You could probably keep
even a Win 98 computer running that long!

ChrisA

[toc] | [prev] | [next] | [standalone]

#77789

On Thu, Sep 11, 2014 at 8:29 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Suppose you somehow managed to create 9223372036854775807 threads. If your
> computer has 16 GB of RAM available, that means that at most each thread
> can use:
>
> py> 16*1024*1024*1024/9223372036854775807
> 1.862645149230957e-09
>
> bytes.

Doesn't that calculation assume that they all have to be alive at the
same time? (Maybe I missed it in earlier posts, but I don't think the
OP indicated they'd have to all be active.)

That said, I think the OP should probably be worrying about other ways
his program could fail besides overflowing some nonexistent max thread
id. :-)

Skip

[toc] | [prev] | [next] | [standalone]

#77795

On 12Sep2014 11:29, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>[...]maxint. I know that some Linux
>systems can have an uptime over a year, perhaps even two years, but I think
>that nearly 300 years is asking a bit much.

2 years is nothing. Unless they have a particularly buggy kernel, most UNIX 
systems, Linux included, will stay up almost indefinitely. We've definitely had 
systems up for well over 2 years.

>Your hardware probably won't
>keep working that long.

300 years? Probably not. Regrettably.

Cheers,
Cameron Simpson <cs@zip.com.au>

We need a taxonomy for 'printing-that-is-no-longer-printing.'
- overhead by WIRED at the Intelligent Printing conference Oct2006

[toc] | [prev] | [next] | [standalone]

#77796

On Fri, Sep 12, 2014 at 1:41 PM, Cameron Simpson <cs@zip.com.au> wrote:
> On 12Sep2014 11:29, Steven D'Aprano <steve+comp.lang.python@pearwood.info>
> wrote:
>>
>> [...]maxint. I know that some Linux
>> systems can have an uptime over a year, perhaps even two years, but I
>> think
>> that nearly 300 years is asking a bit much.
>
>
> 2 years is nothing. Unless they have a particularly buggy kernel, most UNIX
> systems, Linux included, will stay up almost indefinitely. We've definitely
> had systems up for well over 2 years.
>
>> Your hardware probably won't
>> keep working that long.
>
>
> 300 years? Probably not. Regrettably.

Once you get into the counting of years (rather than days), it's all
down to hardware. How long before that hardware needs an upgrade? Does
your incoming power have fail-overs? I don't currently have any
servers with multiple power supplies, so if anything like that goes
bung, my server's down. Doesn't matter how quickly I can bring up an
equivalent on a different hunk of hardware, the uptime's gone.

But yeah. 300 years? Good luck. I don't think anyone's ever going to hit that.

ChrisA

[toc] | [prev] | [next] | [standalone]

#77800

Hi Steven,

On Fri, Sep 12, 2014 at 11:29:56AM +1000, Steven D'Aprano wrote:
> >>>> import sys
> >>>> print sys.maxint
> > 9223372036854775807
> > 
> > the couter could be 9223372036854775807?
> > 
> > And after? :)
> 
> Suppose you somehow managed to create 9223372036854775807 threads. If your
> computer has 16 GB of RAM available, that means that at most each thread
> can use:

so, thanks for your and others answers - this was just a
_theoretical_ question. What is the practice - that's an another
thread. :)

I just simply interested about this theory, not more. I don't
care with-how-many-memories-needs-and-how-many-years-to-overflow
that counter, but many people calculates that - thanks :)


Cheers,


a.

[toc] | [prev] | [next] | [standalone]

#77834

Ervin Hegedüs <airween@gmail.com> writes:
> ...
What is used as thread id is platform dependent. Likely, it depends
on the thread support of the underlying C libary (i.e. the
operating system thread support).

Under Linux, thread ids seem to be addresses - i.e. very large
integers.

[toc] | [prev] | [next] | [standalone]

#77839

dieter wrote:

> Ervin Hegedüs <airween@gmail.com> writes:
>> ...
> What is used as thread id is platform dependent. Likely, it depends
> on the thread support of the underlying C libary (i.e. the
> operating system thread support).
> 
> Under Linux, thread ids seem to be addresses - i.e. very large
> integers.

$ grep "Exception in thread" /usr/lib/python3.4/threading.py 
                    _sys.stderr.write("Exception in thread %s:\n%s\n" %
                            "Exception in thread " + self.name +


I believe the "Thread-ID"s we are talking about are actually user-specified 
names attached Python's threading.Thread instances, not something on the OS 
level. These names default to

$ grep -i -A4 -B1 "def _newname" /usr/lib/python3.4/threading.py
_counter = 0
def _newname(template="Thread-%d"):
    global _counter
    _counter += 1
    return template % _counter

[toc] | [prev] | [next] | [standalone]

#77843

[Multipart message — attachments visible in raw view] — view raw

Unfortunately we will never know 😢

Sent from Blue Mail



On 12 Sep 2014 07:43, at 07:43, Chris Angelico <rosuav@gmail.com> wrote:
>On Fri, Sep 12, 2014 at 1:41 PM, Cameron Simpson <cs@zip.com.au> wrote:
>> On 12Sep2014 11:29, Steven D'Aprano
><steve+comp.lang.python@pearwood.info>
>> wrote:
>>>
>>> [...]maxint. I know that some Linux
>>> systems can have an uptime over a year, perhaps even two years, but
>I
>>> think
>>> that nearly 300 years is asking a bit much.
>>
>>
>> 2 years is nothing. Unless they have a particularly buggy kernel,
>most UNIX
>> systems, Linux included, will stay up almost indefinitely. We've
>definitely
>> had systems up for well over 2 years.
>>
>>> Your hardware probably won't
>>> keep working that long.
>>
>>
>> 300 years? Probably not. Regrettably.
>
>Once you get into the counting of years (rather than days), it's all
>down to hardware. How long before that hardware needs an upgrade? Does
>your incoming power have fail-overs? I don't currently have any
>servers with multiple power supplies, so if anything like that goes
>bung, my server's down. Doesn't matter how quickly I can bring up an
>equivalent on a different hunk of hardware, the uptime's gone.
>
>But yeah. 300 years? Good luck. I don't think anyone's ever going to
>hit that.
>
>ChrisA
>-- 
>https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#77845

On Sat, Sep 13, 2014 at 12:09:28PM +0200, Martin Skjöldebrand wrote:
> Unfortunately we will never know 😢

hehe :), joke of the day :)


thanks,


a.



-- 
I � UTF-8

[toc] | [prev] | [next] | [standalone]

#77936

Am 11.09.2014 23:32 schrieb Ervin Hegedüs:

>> There is no upper limit to the thread name other than that you will
>> eventually run out of memory ;)
>
> thanks - I hope that the memory will not run out by these
> threads... :)
>
> Anyway, that means, on my system:
>
>>>> import sys
>>>> print sys.maxint
> 9223372036854775807
>
> the couter could be 9223372036854775807?
>
> And after? :)

After that, the couter magically turns into a long and it goes on.


Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web