Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #81910 > unrolled thread

time.monotonic() roll over

Back to article view | Back to comp.lang.python

Contents

  time.monotonic() roll over "ast" <nomail@invalid.com> - 2014-12-04 16:25 +0100
    Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-04 17:50 +0200
      Re: time.monotonic() roll over Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-04 09:05 -0700
        Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-04 18:21 +0200
          Re: time.monotonic() roll over Chris Angelico <rosuav@gmail.com> - 2014-12-05 03:27 +1100
            Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-04 20:09 +0200
              Re: time.monotonic() roll over Chris Angelico <rosuav@gmail.com> - 2014-12-05 05:29 +1100
                Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-04 20:44 +0200
                  Re: time.monotonic() roll over Chris Angelico <rosuav@gmail.com> - 2014-12-05 07:02 +1100
              Re: time.monotonic() roll over Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-04 12:29 -0700
              Re: time.monotonic() roll over Akira Li <4kir4.1i@gmail.com> - 2014-12-04 23:24 +0300
              Re: time.monotonic() roll over Chris Angelico <rosuav@gmail.com> - 2014-12-05 07:33 +1100
                Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-04 22:51 +0200
              Re: time.monotonic() roll over Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-04 15:17 -0700
                Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-05 00:44 +0200
                  Re: time.monotonic() roll over Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-05 10:04 -0700
                    Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-05 20:48 +0200
              Re: time.monotonic() roll over Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 11:39 +1100
                Re: time.monotonic() roll over Dave Angel <davea@davea.name> - 2014-12-04 20:21 -0500
                  Re: time.monotonic() roll over Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 13:54 +1100
                    Re: time.monotonic() roll over Dave Angel <davea@davea.name> - 2014-12-05 00:29 -0500
                    Re: time.monotonic() roll over Lele Gaifax <lele@metapensiero.it> - 2014-12-05 08:56 +0100
                      Re: time.monotonic() roll over Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 21:09 +1100
                    Re: time.monotonic() roll over Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-12-05 12:12 -0500
                Re: time.monotonic() roll over Marko Rauhamaa <marko@pacujo.net> - 2014-12-05 08:17 +0200
                  Re: time.monotonic() roll over Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-12-05 12:31 -0500
                Re: time.monotonic() roll over Akira Li <4kir4.1i@gmail.com> - 2014-12-05 12:48 +0300
      Re: time.monotonic() roll over Chris Angelico <rosuav@gmail.com> - 2014-12-05 03:12 +1100
      Re: time.monotonic() roll over random832@fastmail.us - 2014-12-04 11:51 -0500
    Re: time.monotonic() roll over Nobody <nobody@nowhere.invalid> - 2014-12-05 00:54 +0000

Page 2 of 2 — ← Prev page 1 [2]

#81975

On 12/04/2014 09:54 PM, Steven D'Aprano wrote:
> Dave Angel wrote:
>
>> On 12/04/2014 07:39 PM, Steven D'Aprano wrote:
>>> Marko Rauhamaa wrote:
>>>
>>>> So, if I call
>>>>
>>>> time.sleep(86400)
>>>>
>>>> and the program is suspended for 24 hours, should time.sleep() return
>>>> right after it is resumed or after another 24 hours?
>>>
>>> If the program is suspended, then no time should pass for that program.
>>> Since sleep() is given in terms of a duration, not an absolute time
>>> ("sleep until now + 24 hours"), if no time passes for that program,
>>> sleep() should sleep for 24 hours *after the program resumes*.
>>>
>>> Unfortunately a lot of systems get that wrong. E.g. I just ran "sleep 30"
>>> from my Linux shell, immediately paused it using Ctrl-Z, waited a couple
>>> of minutes, and used fg to continue. It returned immediately.
>>>
>>> Why is this behaviour wrong?
>>>
>>> Consider why people might call sleep. They're normally calling it to wait
>>> for something else to complete. Often that something else is an external
>>> process ("wait a few seconds to let the hard drive finish syncing, wait
>>> for the web server to get less busy, wait for the user to catch up...")
>>> but it might be another thread in the same process. If the process is
>>> suspended, that other thread won't get to run, and so even though time on
>>> the outside has continued to move forward, from the perspective of the
>>> process, it hasn't.
>>>
>>> The same applies even more so if the process is running in a virtual
>>> machine which has just been suspended.
>>>
>>> The same applies if the system clock is adjusted mid-sleep. If I call
>>> sleep 60, then immediately adjust the clock forward an hour (say, due to
>>> a daylight savings adjustment), that shouldn't cause sleep to return. It
>>> should still suspend for 60 seconds.
>>
>> And I say you have it exactly backwards.  People should NOT call sleep()
>> to measure something,
>
> I never suggested you call sleep to measure something. You call sleep to
> *wait* for something. I even used the word "wait" four times in the above.

I mentioned measurement because I couldn't imagine any other reason you 
would be arguing the other side.  You're certainly not waiting for the 
event, wleep has no parameters that let you specify what event you might 
be waiting for.


>
>> they should only call it to release the processor
>> for "up to" a specified time.
>
> That's silly. Zero is "up to" any positive value you want. Do you really
> intend to say that this would be an acceptable implementation of sleep()?
>
> def sleep(delay):
>      return
>

Yes, of course.  Not very machine efficient, but perfectly correct. 
Because the testing code will make its check, and turn right around and 
issue another sleep call.  This is called a busy loop, and I mention it 
below.  It is the degenerate (if inefficient) form.


>
>> That time should normally be a major
>> fraction of the time that some external event is expected to take.  Then
>> the code should check the external event and sleep some more if necessary.
>>
>> If efficiency were not an issue, the code would just do a busy loop,
>> checking the external condition repeatedly.
>
> In many cases, you don't have an external condition that can be easily or
> practically checked. So you estimate how long you need to wait, add a
> safety margin, and write "sleep 5" (for some value of 5), and hope.
>
> If you are a sys admin writing a script that runs during boot time, you do
> that a lot. Or at least the ones I work with do :-)
>
>
>> Since the OS has no way of knowing whether the thing being waited for is
>> a thread, another process, a human being, a network operation, or the
>> end of the world, the interpretation of sleep needs to be the most
>> conservative one.
>
> I agree! And returning immediately is not the most conservative one.
>
> The most conservative approach is to assume that while you're suspended,
> *everything else* is suspended too, so when you resume you still have to
> sleep for the full N seconds.
>
>
>> There are many ways of suspending a process, and some
>> of them will also suspend the external event.  Since the OS cannot know
>> which case is significant, it has to return control to the caller at the
>> soonest of the many possible interpretations.
>
> /s/latest/
>

I guess we'll just have to agree to disagree.  I can now see the 
usefulness of a function like you describe, I just can't imagine it 
being the sleep() we've all come to know and love.


-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#81988

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> The most conservative approach is to assume that while you're suspended,
> *everything else* is suspended too, so when you resume you still have to
> sleep for the full N seconds.

That's an intriguing interpretation of what sleep() should do, but I
fail to see *how* the function could even attempt to do that: should it
keep its own notion of the elapsed time?

I mean, one simplicistic, naive (and wrong) implementation of Dave's
(and mine, FWIW) could be:

    def sleep(secs):
        now = time()
        while time() < (now + secs):
            pass # this is wrong, should really be some kind of "yield_cpu()"

How would it become if it should take into account the intervening
"coma"? Maybe something like

    def sleep(secs):
        nmillisleeps = secs * 1000
        for i in range(nmillisleeps):
            now = time()
            while time() < (now + 0.001):
                pass

?

Also, should it also assume that if the system suffers of a slowdown
instead (maybe simply due to another process using the same /inefficient
and wrong/ implementation of sleep() ;-) then everything else gets
slowed down too, and act accordingly?

just-for-the-sake-of-arguing-ly, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

[toc] | [prev] | [next] | [standalone]

#81994

Lele Gaifax wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
> 
>> The most conservative approach is to assume that while you're suspended,
>> *everything else* is suspended too, so when you resume you still have to
>> sleep for the full N seconds.
> 
> That's an intriguing interpretation of what sleep() should do, but I
> fail to see *how* the function could even attempt to do that: should it
> keep its own notion of the elapsed time?

Provided that the operating system can provide a monotonic clock that only
ticks while the process calling it is running, that shouldn't be hard. The
only tricky part is if the monotonic clock rolls over.

I'm not sure how the OS would implement that, but most OSes supply a global
monotonic clock, so it's probably possible. I leave that as an exercise to
the reader :-)

Here's a discussion from the Factor programming language:

http://code-factor.blogspot.com.au/2009/11/monotonic-timers.html



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82007

On Fri, 05 Dec 2014 13:54:51 +1100, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> declaimed the following:

>That's silly. Zero is "up to" any positive value you want. Do you really
>intend to say that this would be an acceptable implementation of sleep()?
>
>def sleep(delay):
>    return
>
	I wouldn't, but only because that doesn't give up the processor

def sleep(delay):
	system.scheduler()		#Whatever one wants to call it...
	return

OTOH -- does at least relinquish the processor... Of course, it does not
meet the (implied) documentation of not returning until at least <delay>
time has passed. That then becomes a documentation flaw.

"""
sleep(delay)

Invokes the OS process scheduler to immediately transfer control to the
next ready task on the system.

The parameter <delay> is a place holder and has no effect; the function
returns immediately upon the process being made active by the OS scheduler.
"""

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#81978

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> Unfortunately a lot of systems get that wrong. E.g. I just ran "sleep
> 30" from my Linux shell, immediately paused it using Ctrl-Z, waited a
> couple of minutes, and used fg to continue. It returned immediately.
>
> Why is this behaviour wrong?

I think the #1 thing is to specify the behavior clearly. I'm not seeing
that so it is impossible to say if the GNU coreutils sleep or
time.sleep() does what it was designed to do. I must admit I have
neglected to document that situation in some of my related designs.

Also, I think what is right or wrong depends on the use case. Ideally,
there are facilities to implement the desired semantics.

If a program spends long periods in an induced coma (which is becoming
more and more common nowadays), the whole design of the program is under
quite a bit of strain. What to do with time-based statistics collection?
Should periodic operations catch up with the lost time by repeating in a
tight loop for a zillion times? What to do with "impossible,"
out-of-order event sequences?


Marko

[toc] | [prev] | [next] | [standalone]

#82009

On Fri, 05 Dec 2014 08:17:53 +0200, Marko Rauhamaa <marko@pacujo.net>
declaimed the following:


>If a program spends long periods in an induced coma (which is becoming
>more and more common nowadays), the whole design of the program is under
>quite a bit of strain. What to do with time-based statistics collection?
>Should periodic operations catch up with the lost time by repeating in a
>tight loop for a zillion times? What to do with "impossible,"
>out-of-order event sequences?
>
	You do what the requirements for the system specify. If the
requirements don't state a behavior for this situation, the design of the
system is faulty and needs to be corrected.

	On my part, every system I've worked on with a "delay", "pause", or
"sleep" operation that takes a relative time as input ("pause" is otherwise
ambiguous -- some languages use it to put a prompt on a console and wait
for operator to resume) have always made the delay relative to the
processor time base. Suspending/blocking a task while the system is still
operating did mean that any pending delay likely expires immediately upon
the task being resumed/unblocked and put on the ready list.

	Now -- what to do in the case of the entire computer being hibernated
is a different matter; synchronizing the "tick" clock used by sleep with a
wall-clock/RTC to account for time spent in hibernation should immediately
expire pending sleeps (or, more correct, expire those that would have
expired if the system had not gone into hibernation).

	Your "time-based statistics collection"? If the time is based on
wall-clock time, and the system is put into hibernation, I'd expect sleeps
to be expired when coming out of hibernation, and one would have a gap in
the collected data time-stamps -- and no sudden looping to collect however
many "missed" intervals took place... The data would collect once, and then
set up for the next standard interval.

	If based upon processor clock tick, independent of wall-clock time, and
the processor clock does not tick during hibernation, then there are no
missed collection intervals from the view of the process. The process does
not know that the world stopped for some period of time.

	If the process needs to be regular with wall-clock time, and missed
intervals are a serious condition --- well; disable hibernation modes.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#81991

Dave Angel <davea@davea.name> writes:

...many words about sleep()...
> Since the OS has no way of knowing whether the thing being waited for
> is a thread, another process, a human being, a network operation, or
> the end of the world, the interpretation of sleep needs to be the most
> conservative one.  There are many ways of suspending a process, and
> some of them will also suspend the external event.  Since the OS
> cannot know which case is significant, it has to return control to the
> caller at the soonest of the many possible interpretations.

That is why there is API (e.g., clock_nanosleep()) that allows us to
choose whether we need a relative delay (e.g., kill a subprocess if it
hasn't finished in 10 seconds) or an absolute deadline (e.g., these
lights should be on 9pm-6am local time).

*Both* use cases are valid.


--
Akira

[toc] | [prev] | [next] | [standalone]

#81916

On Fri, Dec 5, 2014 at 3:05 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Dec 4, 2014 8:56 AM, "Marko Rauhamaa" <marko@pacujo.net> wrote:
>>
>> "ast" <nomail@invalid.com>:
>>
>> > Does any body know when time.monotonic() rolls over ?
>>
>> Never, according to the documentation you linked.
>>
>> Admittedly, the documentation confuses the reader by chatting about some
>> irrelevant internal Windows details.
>
> Not entirely irrelevant. The implication is that if you go more than 49 days
> without calling the function on old Windows systems, rollovers could be
> missed, which is good to know about. The result would still be monotonic,
> but it wouldn't accurately reflect the time elapsed.

I don't know for sure about the newer Windowses, but I believe they
use a 64-bit counter instead of the 32-bit one used in previous
versions, so even if they do roll over, there'll be a much MUCH longer
time scale involved. Even if it stores time in nanoseconds, a 64-bit
counter would allow for hundreds of years between rollovers, which is
reasonably safe - much better than the month-and-a-bit of older ones,
which is shorter than quite a lot of my programs' uptimes. Hence the
lack of information about them; you don't have to worry. (Likewise on
Unix-derived systems, I believe, and for the same reason.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#81920

On Thu, Dec 4, 2014, at 10:50, Marko Rauhamaa wrote:
> That is, the internal integer wrap is not guarded against between the
> calls to time.monotonic(), maybe.

Looking at the code, it looks like it does guard against the rollover,
though if you let your program run for 49.7 days _without_ calling
time.monotonic during that time, it will not realize that it has passed
the original value multiple times. And of course, if it rolls over
before the first time you call monotonic in process B, process B's value
will be less than process A which called it before the rollover.

This is consistent with the documentation, and appears to be the reason
the documentation discusses this issue at all.

[toc] | [prev] | [next] | [standalone]

#81966

On Thu, 04 Dec 2014 16:25:44 +0100, ast wrote:

> There is no roll over problem with time.time() since the very
> first one in planned far in the future, but time.time() can go
> backward when a date update throught NTP server is done.

> time.monotonic() is monotonic but roll over often (every 49.7 
> days)
> 
> So what to do ? 
> 
> Using time.monotonic() and take some actions if a roll over 
> is detected ?

One possibility: use both and monitor the skew (the difference between the
two clocks).

The first time you read them, calculate and store the epoch for
time.monotic(). On subsequent calls, repeat the calculation and check that
the epoch is approximately the same. If it isn't, the difference between
the actual and expected values should be close to an exact multiple of
2**32 milliseconds, which tells you how many times time.monotonic() has
rolled over.

Adjustments to the clock on which time.time() is based shouldn't exceed a
few seconds per year, so there's not much risk that you won't be able to
figure out the correct adjustment.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web