Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #20159 > unrolled thread

log and figure out what bits are slow and optimize them.

Back to article view | Back to comp.lang.python

Contents

  log and figure out what bits are slow and optimize them. sajuptpm <sajuptpm@gmail.com> - 2012-02-10 04:30 -0800
    Re: log and figure out what bits are slow and optimize them. Arnaud Delobelle <arnodel@gmail.com> - 2012-02-10 12:38 +0000
      Re: log and figure out what bits are slow and optimize them. sajuptpm <sajuptpm@gmail.com> - 2012-02-10 04:56 -0800
        Re: log and figure out what bits are slow and optimize them. Kev Dwyer <kevin.p.dwyer@gmail.com> - 2012-02-10 22:37 +0000
          Re: log and figure out what bits are slow and optimize them. John Gordon <gordon@panix.com> - 2012-02-10 23:06 +0000
            Re: log and figure out what bits are slow and optimize them. sajuptpm <sajuptpm@gmail.com> - 2012-02-10 23:53 -0800
    Re: log and figure out what bits are slow and optimize them. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-02-10 14:25 +0000

#20159 — log and figure out what bits are slow and optimize them.

Hi,

I want to log time taken to complete database requests inside a method/
function using decorator .  is it possible ????
I think, i have to inject log code inside the method/fuctions or
modify it.
I wrote a decorator to log taken by a method/function to complete it
execution and its working well.

My requirement : log everything and figure out what bits are slow and
optimize them.

What are your suggestions ??

[toc] | [next] | [standalone]

#20160

On 10 February 2012 12:30, sajuptpm <sajuptpm@gmail.com> wrote:
> Hi,
>
> I want to log time taken to complete database requests inside a method/
> function using decorator .  is it possible ????
> I think, i have to inject log code inside the method/fuctions or
> modify it.
> I wrote a decorator to log taken by a method/function to complete it
> execution and its working well.
>
> My requirement : log everything and figure out what bits are slow and
> optimize them.
>
> What are your suggestions ??

Are you familiar with this?

http://docs.python.org/library/profile.html

-- 
Arnaud

[toc] | [prev] | [next] | [standalone]

#20161

Hi,

Yes i saw profile module,
I think i have to do function call via

cProfile.run('foo()')

I know, we can debug this way.

But, i need a fixed logging system and want to use it in production.
      I think, we can't permanently include profile's debugging code
in source code,
     will cause any performance issue ??

[toc] | [prev] | [next] | [standalone]

#20199

sajuptpm wrote:

> Hi,
> 
> Yes i saw profile module,
> I think i have to do function call via
> 
> cProfile.run('foo()')
> 
> I know, we can debug this way.
> 
> But, i need a fixed logging system and want to use it in production.
>       I think, we can't permanently include profile's debugging code
> in source code,
>      will cause any performance issue ??

*Any* instrumentation code is going to affect performance.

It's a trade-off that you need to analyse and manage in the context of your 
application.

[toc] | [prev] | [next] | [standalone]

#20203

In <mailman.5685.1328913452.27778.python-list@python.org> Kev Dwyer <kevin.p.dwyer@gmail.com> writes:

> *Any* instrumentation code is going to affect performance.

Funny story about that...

I wanted to profile some code of mine, and a colleague recommended the
'hotshot' module.

It's pretty easy to use: there are functions to start profiling, stop
profiling and print results.

So I added the function calls and ran my code.... and it took a really
long time.  I mean a REALLY long time.  In fact I eventually had to kill
the process.

I briefly wondered if my coworker was playing a prank on me... then I
realized that I had neglected to call the function to stop profiling!

So when I went to print the results, it was still profiling... endlessly.

(Okay, maybe it wasn't that funny.)

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon@panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"

[toc] | [prev] | [next] | [standalone]

#20215

I decided to create a decorator like.

import cProfile
def debug_time(method):
    def timed(*args, **kw):
        prof = cProfile.Profile()
        prof.enable(subcalls=False, builtins=False)
        result = prof.runcall(method, *args, **kw)
        #prof.print_stats()
        msg = "\n\n\n\n#######################################"
        msg += "\n\nURL : %s" %(tg.request.url)
        msg += "\nMethod: %r" %(method.__name__)
        print "--ddd--------", type(prof.getstats())
        msg += "\n\nStatus : %s" %(prof.print_stats())
        msg += "\n\n#######################################"
        print msg
        LOGGER.info(msg)
        return result
    return timed

Ref : http://stackoverflow.com/questions/5375624/a-decorator-that-profiles-a-method-call-and-logs-the-profiling-result


I want to log it in existing log file in my project,
so i tried prof.print_stats() and prof.getstats(). prof.getstats()
will need extra loop for fetch data.
prof.print_stats() will log library calls also.

Please suggest a better way to log profiler output.

[toc] | [prev] | [next] | [standalone]

#20164

Please don't top post.

On 10/02/2012 12:59, Saju M wrote:
> Yes i saw profile module,
> I think, i have to do function call via
> cProfile.run('foo()')
> I know, we can debug this way.
> But, I need a fixed logging system and want to use it in production.
>        I think, we can't permanently include profile's debugging code in
> source code,
>       will cause any performance issue ??
>

How about http://docs.python.org/library/logging.html ?

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web