Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92208 > unrolled thread

Function to show time to execute another function

Back to article view | Back to comp.lang.python

Contents

  Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 08:39 +0200
    Re: Function to show time to execute another function Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-07 08:39 +0100
      Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 10:22 +0200
        Re: Function to show time to execute another function Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-06-07 11:06 +0200
          Is it a newsgroup or a list? random832@fastmail.us - 2015-06-07 07:20 -0400
            Re: Is it a newsgroup or a list? Steven D'Aprano <steve@pearwood.info> - 2015-06-07 21:45 +1000
              Re: Is it a newsgroup or a list? Chris Warrick <kwpolska@gmail.com> - 2015-06-07 15:10 +0200
              Re: Is it a newsgroup or a list? Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-06-07 16:12 +0200
          Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 13:53 +0200
          Re: Is it a newsgroup or a list? Tim Golden <mail@timgolden.me.uk> - 2015-06-07 14:57 +0100
        Re: Function to show time to execute another function Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-07 11:16 +0100
          [OT] Re: Function to show time to execute another function Marko Rauhamaa <marko@pacujo.net> - 2015-06-07 14:02 +0300
            Re: [OT] Re: Function to show time to execute another function Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-07 12:12 +0100
          Re: Function to show time to execute another function Steven D'Aprano <steve@pearwood.info> - 2015-06-07 21:29 +1000
          Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 14:03 +0200
        Re: Function to show time to execute another function Tim Golden <mail@timgolden.me.uk> - 2015-06-07 12:05 +0100
          Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 14:43 +0200
        Re: Function to show time to execute another function Laura Creighton <lac@openend.se> - 2015-06-07 15:03 +0200
        Re: Function to show time to execute another function Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-07 20:51 +0200
          Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 22:35 +0200
            Re: Function to show time to execute another function Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-08 07:04 +0200
              Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-08 08:32 +0200
    Re: Function to show time to execute another function Steven D'Aprano <steve@pearwood.info> - 2015-06-07 19:28 +1000
      Re: Function to show time to execute another function Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-06-07 11:44 +0200
      Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 14:14 +0200
      Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 14:58 +0200
    Re: Function to show time to execute another function Steven D'Aprano <steve@pearwood.info> - 2015-06-07 19:51 +1000
      Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 14:27 +0200
    Re: Function to show time to execute another function Cecil Westerhof <Cecil@decebal.nl> - 2015-06-07 11:31 +0200
    Re: Is it a newsgroup or a list? Gene Heskett <gheskett@wdtv.com> - 2015-06-07 11:49 -0400
    Re: Is it a newsgroup or a list? Larry Martell <larry.martell@gmail.com> - 2015-06-07 11:56 -0400

Page 2 of 2 — ← Prev page 1 [2]

#92326

On 07.06.2015 22:35, Cecil Westerhof wrote:

>> And you also posted your solution. I fail to find any question in
>> your original posting at all.
> 
> That is because there was no question: I just wanted to share
> something I thought that could be useful. If you would have taken the
> trouble to read a little, you would have known that.

You actually didn't write that. You wrote "I have a problem. Here is my
solution. This is how it works." Okay, so what? If you would have taken
the time to write "Just wanted to share this" your intention would have
been obvious.

>>> Sadly the quality of the answers on this list is going down. 
>>
>> Maybe you should start asking questions that people are able to
>> comprehend so that you get an answer that you like.
> 
> I do not want answer I like, but an answer that is useful.

Sorry to break it to you, but this group does not revolve around you. An
answer that might not be useful to you might be useful for someone else
and it surely is useful to get a discussion.

>>> Here I get an alternative that does only half what I want and when
>>> writing an alternative for ‘!find’ I am told I could use ‘!find’
>>> (which only works in ipython, not python and which also not works
>>> with Windows).
>>
>> Protip: Ditch the shitty attitude. I don't know if you're a jerk or
>> not but I know for sure that you sound like one. Makes it also much
>> less likely to get the answers you'd like.
> 
> So if someone gives an answer that is completely useless it is a
> shitty attitude when I point this out? Very interesting indeed.

Precisely. Especially the tone you used is proof of your really shitty
attitude.

> I do not think I have to expect something useful from you, it looks
> like you prefer name calling. Luckily there are a lot of people that
> have a different attitude.

Indeed you can expect nothing more from me and I purposely omitted the
way I do timing from this discussion. I do not aid people who fail to
recognize their major social dysfunction. Not even when they are coding
geniuses. Which, judging from your code snippet, you clearly aren't.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#92328

On Monday  8 Jun 2015 07:04 CEST, Johannes Bauer wrote:

> On 07.06.2015 22:35, Cecil Westerhof wrote:
>
>>> And you also posted your solution. I fail to find any question in
>>> your original posting at all.
>>
>> That is because there was no question: I just wanted to share
>> something I thought that could be useful. If you would have taken
>> the trouble to read a little, you would have known that.
>
> You actually didn't write that. You wrote "I have a problem. Here is
> my solution. This is how it works." Okay, so what? If you would have
> taken the time to write "Just wanted to share this" your intention
> would have been obvious.

Again: if you just had read other posts you would have found that I
acknowledged that.


>>>> Sadly the quality of the answers on this list is going down. 
>>>
>>> Maybe you should start asking questions that people are able to
>>> comprehend so that you get an answer that you like.
>>
>> I do not want answer I like, but an answer that is useful.
>
> Sorry to break it to you, but this group does not revolve around
> you. An answer that might not be useful to you might be useful for
> someone else and it surely is useful to get a discussion.

Where do you find that I think that the group revolves around me?

Well it could be useful for someone to write something about how to
pay less taxes. But I do not think it should be posted in this group.


>>>> Here I get an alternative that does only half what I want and
>>>> when writing an alternative for ‘!find’ I am told I could use
>>>> ‘!find’ (which only works in ipython, not python and which also
>>>> not works with Windows).
>>>
>>> Protip: Ditch the shitty attitude. I don't know if you're a jerk
>>> or not but I know for sure that you sound like one. Makes it also
>>> much less likely to get the answers you'd like.
>>
>> So if someone gives an answer that is completely useless it is a
>> shitty attitude when I point this out? Very interesting indeed.
>
> Precisely. Especially the tone you used is proof of your really
> shitty attitude.

Do you ever look into a mirror?


>> I do not think I have to expect something useful from you, it looks
>> like you prefer name calling. Luckily there are a lot of people
>> that have a different attitude.
>
> Indeed you can expect nothing more from me and I purposely omitted
> the way I do timing from this discussion. I do not aid people who
> fail to recognize their major social dysfunction. Not even when they
> are coding geniuses. Which, judging from your code snippet, you
> clearly aren't.

And again the name calling. You think you are very superior, don't
you? You have a problem with my tone, well I do not even get near you:
you clearly are the master. If I would like to become dysfunctional I
should treasure your postings.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92217

On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I wrote
> the following function:
>     def time_test(function, *args):
>         startTime   = time.time()
>         results     = function(*args)
>         endTime     = time.time()
>         print('It took {0} seconds'.format(endTime - startTime))
>         return results


There is a lot of subtlety in timing functions on modern day computers. What
you measure there includes the time taken to lookup the names "function"
and "args" (although that ought to be very quick) and to expand out *args
(not quite so quick). 

More importantly, it will also be less accurate on Windows systems, and may
include time during which the operating system is running other background
tasks. It makes no attempt to allow for whether code is in the CPU cache or
not. And what if the garbage collector happens to run in the middle of your
test?

Timing code these days is subtle and complicated!

Depending on what function actually does, those complications may, or may
not, make a real difference. If function() does a lot of work (say, at
least one second) then what I have said is probably completely irrelevant
and you can ignore it. For those cases, your function is perfectly fine,
although I prefer to use a with statement interactively. Of course I can
still call a function, but I don't *have* to call a function.

Here is a simple example:

http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/



But if your function takes less than, say, 1 millisecond, then your timing
results are probably just meaningless random numbers, affected more by the
other ten thousand processes running on your computer than by the Python
code itself.

In that case, you should learn how to use the timeit module. It's a little
complex, but worth it for timing small code snippets.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#92219

Il 07/06/2015 11:28, Steven D'Aprano ha scritto:

>
> But if your function takes less than, say, 1 millisecond, then your timing
> results are probably just meaningless random numbers, affected more by the
> other ten thousand processes running on your computer than by the Python
> code itself.

That's a good point. IMHO test the time execution of a function makes 
sense only if all these assetions are true:

- the function takes a long time to be executed;
- the execution is iterated a certain (large) number of times.

-- 
Ciao!
Luca

[toc] | [prev] | [next] | [standalone]

#92239

On Sunday  7 Jun 2015 11:28 CEST, Steven D'Aprano wrote:

> On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:
>
>> Sometimes I just want to know how much time a function takes, but
>> at the same time I also want the result of the function. For this I
>> wrote the following function: def time_test(function, *args):
>> startTime = time.time() results = function(*args) endTime =
>> time.time() print('It took {0} seconds'.format(endTime -
>> startTime)) return results
>
>
> There is a lot of subtlety in timing functions on modern day
> computers. What you measure there includes the time taken to lookup
> the names "function" and "args" (although that ought to be very
> quick) and to expand out *args (not quite so quick).
>
> More importantly, it will also be less accurate on Windows systems,
> and may include time during which the operating system is running
> other background tasks. It makes no attempt to allow for whether
> code is in the CPU cache or not. And what if the garbage collector
> happens to run in the middle of your test?
>
> Timing code these days is subtle and complicated!
>
> Depending on what function actually does, those complications may,
> or may not, make a real difference. If function() does a lot of work
> (say, at least one second) then what I have said is probably
> completely irrelevant and you can ignore it. For those cases, your
> function is perfectly fine, although I prefer to use a with
> statement interactively. Of course I can still call a function, but
> I don't *have* to call a function.

Next time I should give more background. ;-)

This function is only mend to get an indication of the needed time and
in my opinion is only useful from ate least 5 seconds. If you want
real measurements you should use timeit I think.


> Here is a simple example:
>
> http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/

That looks quite interesting. I think I will use that in my function.
I still think the function is handy, because you need less code with
it.


> But if your function takes less than, say, 1 millisecond, then your
> timing results are probably just meaningless random numbers,
> affected more by the other ten thousand processes running on your
> computer than by the Python code itself.
>
> In that case, you should learn how to use the timeit module. It's a
> little complex, but worth it for timing small code snippets.

I knew that. It is just to use before using the heavy guns (and also
wanting the result of the function). But I should be more explicit
next time.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92251

On Sunday  7 Jun 2015 11:28 CEST, Steven D'Aprano wrote:

> Here is a simple example:
>
> http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/
I use that now in my function:
#-----------------------------------------------------------------------
def time_test(function, arguments, print_time = True):
    t = Timer()
    with t:
        results     = function(*arguments)
    used_time = t.interval
    if print_time:
        print('It took {0} seconds'.format(used_time))
    else:
        results = (used_time, results)
    return results
#-----------------------------------------------------------------------

I find that neater. It also looks like the time value is a little
less, but I did not really measure it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92222

On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I wrote
> the following function:
>     def time_test(function, *args):
>         startTime   = time.time()
>         results     = function(*args)
>         endTime     = time.time()
>         print('It took {0} seconds'.format(endTime - startTime))
>         return results

Oh, I forgot to mention... 

This has the disadvantage that it only prints the time, you cannot collect
it for further processing. (Well, not easily -- you could always shadow the
print built-in, then parse the string... ugggh, that's horrible.)

Using the Timer with statement I linked to earlier, you can easily collect
both the time and the calculated result:

with Timer(verbose=False) as t:
    do_this()
    do_that()
    result = something_else()

time_taken = t.interval
print(result, "took", time_taken)

Even the verbose=False flag is optional, if you leave it out the timing
result will be printed as usual, and you can still capture the time using
t.interval afterwards.

If you are interested, I have a more advanced version of that Timer which I
use almost every day. It works in Python 2.4 through 3.3, I expect it to
work in 3.4 and 3.5 as well, and is extensively tested in CPython and seems
to work in Jython and IronPython as well.


Oh, and to give you an idea of just how large the individual timing
fluctuations of small code snippets can be, here is an example running
under IronPython 2.6:

>>> from timer import Stopwatch
>>> with Stopwatch():
...     x = 12
...     y = x/3
...
time taken: 0.019135 seconds
>>> with Stopwatch():
...     x = 12
...     y = x/3
...
elapsed time is very small; consider using timeit.Timer for micro-timings of
small code snippets
time taken: 0.000114 seconds


The first run was over 160 times slower than the second run.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#92240

On Sunday  7 Jun 2015 11:51 CEST, Steven D'Aprano wrote:

> On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:
>
>> Sometimes I just want to know how much time a function takes, but
>> at the same time I also want the result of the function. For this I
>> wrote the following function: def time_test(function, *args):
>> startTime = time.time() results = function(*args) endTime =
>> time.time() print('It took {0} seconds'.format(endTime -
>> startTime)) return results
>
> Oh, I forgot to mention... 
>
> This has the disadvantage that it only prints the time, you cannot
> collect it for further processing. (Well, not easily -- you could
> always shadow the print built-in, then parse the string... ugggh,
> that's horrible.)

I already found this an omission and posted a better version.


> Using the Timer with statement I linked to earlier, you can easily
> collect both the time and the calculated result:
>
> with Timer(verbose=False) as t:
> do_this()
> do_that()
> result = something_else()
>
> time_taken = t.interval
> print(result, "took", time_taken)

Personally I find:
    time_test(test_random, (100, 10 ** 5))
better as:
    with Timer() as t:
        test_random(100, 10 ** 5)
also because I will do it in a loop with different values for the
second (and maybe the first) parameter. But that is probably personal
preference.


> If you are interested, I have a more advanced version of that Timer
> which I use almost every day. It works in Python 2.4 through 3.3, I
> expect it to work in 3.4 and 3.5 as well, and is extensively tested
> in CPython and seems to work in Jython and IronPython as well.

I would certainly be interested. I work with 3.4, so I could test it
for you. And maybe even build a 3.5 version and test it there also.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92223

On Sunday  7 Jun 2015 08:39 CEST, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I
> wrote the following function: def time_test(function, *args):
> startTime = time.time() results = function(*args) endTime =
> time.time() print('It took {0} seconds'.format(endTime - startTime))
> return results
>
> I can do:
> time_test(test_random, 100, 10 ** 5)
> This outputs:
> It took 17.01685857772827 seconds
> and returns:
> (98592, 100833, 0.977775133140936)
>
> When executing:
> time_test(test_random, 100, 10 ** 6)
> it outputs:
> It took 165.26371836662292 seconds
> and returns:
> (997103, 1002009, 0.9951038363926871)

I improved a little on the function:
    def time_test(function, arguments, print_time = True):
        start_time  = time.time()
        results     = function(*arguments)
        end_time    = time.time()
        used_time   = end_time - start_time
        if print_time:
            print('It took {0} seconds'.format(used_time))
        else:
            results = (used_time, results)
        return results

Default the function still prints the time, but it also possible to
return the time with the results.

Also the arguments are given as a tuple now.

With the following function:
    def test_random(length, multiplier = 10000):
        number_list = length * [0]
        for i in range(length * multiplier):
            number_list[random.randint(0, length - 1)] += 1
        minimum = min(number_list)
        maximum = max(number_list)
        return (minimum, maximum, minimum / maximum)

The call:
    time_test(test_random, (100, 10 ** 5))
outputs:
    It took 15.95521855354309 seconds
and gives:
    (98870, 100810, 0.9807558773931158)

And the call:
    time_test(test_random, (100, 10 ** 5), False)
does not output anything and gives
    (15.926506280899048, (99146, 100779, 0.9837962273886425))

Of-course a new run will give a different result.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92262 — Re: Is it a newsgroup or a list?

On Sunday 07 June 2015 09:57:55 Tim Golden wrote:
> On 07/06/2015 12:20, random832@fastmail.us wrote:
> > On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
> >> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
> >>   > That only times the function. I explicitly mentioned I want
> >>   > both the needed time AND the output.
> >>   >
> >>   > Sadly the quality of the answers on this list is going down....
> >>
> >> First of all, thank God it's a newsgroup, not a list.
> >
> > ...
> >
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> >
> > I've always thought of this as a list that is mirrored to a
> > newsgroup, not a newsgroup that is mirrored to a list. What is the
> > true nature of python-list (and others such as python-ideas)? What
> > is it primarily, what was it first, etc?
>
> It's both: they're mirrored to each other and I consider both to be
> first class citizens. In strict chronology I believe comp.lang.python
> existed first, but python-list has been around for long enough that
> the distinction now is really quite academic.

And I should point out that streaming to a newsgroup server at your local 
ISP, is something not often justified as it can quite easily use up the 
bandwidth your local ISP is buying.  Mine is one such as the newsgroups 
in total are often tying up 20+Gb/sec just for that one service.

In contrast, subbing to 50 mailing lists per customer translates to about 
1% of the bandwidth they are provisioned for.

If you were an ISP, how would you handle it? The mailing list version is 
a no brainer.

> I would not be surprised to find that a fair majority of people told
> off on this list for abusing newsgroup etiquette in one way or another
> not only have no idea that this is (also) a newsgroup but don't even
> know what a newsgroup *is*.

This is true. But newsgroups I believe predate a mailing list by perhaps 
6 months. I sat and watched the net come to life, dialing up Delphi at 
300 baud in the beginning, running up an LD bill often north of $100/mo 
because the access number was not a local call.  On a TRS-80 Color 
Computer in those days.

> They may also not know that it's a mailing list, nor what that is,
> because they're coming via Google Groups -- which uses the Usenet
> gateway -- or through gmane -- which comes in through the mailing list
> -- or some other web mirror.

You mentioned GoogleGroups, now go warsh yur mouth out with some of 
Grandma's Lye soap.  This list is 500% easier to read when they are 
filtered out. I still see the responses but they are at least formatted 
for readability.

> TJG

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#92264 — Re: Is it a newsgroup or a list?

On Sun, Jun 7, 2015 at 11:49 AM, Gene Heskett <gheskett@wdtv.com> wrote:
> You mentioned GoogleGroups, now go warsh yur mouth out with some of
> Grandma's Lye soap.  This list is 500% easier to read when they are
> filtered out. I still see the responses but they are at least formatted
> for readability.

Gene is very wise. Google Groups is evil.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web