Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85488 > unrolled thread

line_profiler: what am I doing wrong?

Back to article view | Back to comp.lang.python

Contents

  line_profiler: what am I doing wrong? Neal Becker <ndbecker2@gmail.com> - 2015-02-10 19:06 -0500
    Re: line_profiler: what am I doing wrong? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-11 11:27 +1100
      Re: line_profiler: what am I doing wrong? Neal Becker <ndbecker2@gmail.com> - 2015-02-10 19:45 -0500
        Re: line_profiler: what am I doing wrong? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-11 12:17 +1100
          Re: line_profiler: what am I doing wrong? Robert Kern <robert.kern@gmail.com> - 2015-02-11 15:42 +0000
            Re: line_profiler: what am I doing wrong? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-12 10:02 +1100
          Re: line_profiler: what am I doing wrong? Neal Becker <ndbecker2@gmail.com> - 2015-02-13 08:33 -0500

#85488 — line_profiler: what am I doing wrong?

I inserted 
@profile
def run(...)

into a module-level global function called 'run'.  Something is very wrong here.
1. profile results were written before anything even ran
2. profile is not defined?

 kernprof -l ./test_unframed.py --lots --of --args ...

Wrote profile results to test_unframed.py.lprof
Traceback (most recent call last):
  File "/home/nbecker/.local/bin/kernprof", line 9, in <module>
    load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')()
  File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, 
in main
    execfile(script_file, ns, ns)
  File "./test_unframed.py", line 721, in <module>
    @profile
NameError: name 'profile' is not defined

[toc] | [next] | [standalone]

#85493

Neal Becker wrote:

> I inserted
> @profile
> def run(...)
> 
> into a module-level global function called 'run'.  Something is very wrong
> here. 1. profile results were written before anything even ran
> 2. profile is not defined?

Well, is it defined? Where does it come from?

If you defined it yourself, it needs to be defined before you can use it.
This won't work:


@profile
def run(...)

def profile(func): ...


Swap the order of profile and run and it should work. (Give or take any
additional bugs in your code.)


If you've imported it from an external module, how did you import it?


import some_module

@some_module.profile
def run(...)


should work. So will this:


from some_module import profile

@profile
def run(...)


But this won't:


import some_module

@profile
def run(...)


and will fail with NameError, exactly as you are experiencing.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85494

Steven D'Aprano wrote:

> Neal Becker wrote:
> 
>> I inserted
>> @profile
>> def run(...)
>> 
>> into a module-level global function called 'run'.  Something is very wrong
>> here. 1. profile results were written before anything even ran
>> 2. profile is not defined?
> 
> Well, is it defined? Where does it come from?
> 
> If you defined it yourself, it needs to be defined before you can use it.
> This won't work:
> 
> 
> @profile
> def run(...)
> 
> def profile(func): ...
> 
> 
> Swap the order of profile and run and it should work. (Give or take any
> additional bugs in your code.)
> 
> 
> If you've imported it from an external module, how did you import it?
> 
> 
> import some_module
> 
> @some_module.profile
> def run(...)
> 
> 
> should work. So will this:
> 
> 
> from some_module import profile
> 
> @profile
> def run(...)
> 
> 
> But this won't:
> 
> 
> import some_module
> 
> @profile
> def run(...)
> 
> 
> and will fail with NameError, exactly as you are experiencing.
> 
> 
> 
> 

To quote from https://pypi.python.org/pypi/line_profiler/

$ kernprof -l script_to_profile.py
kernprof will create an instance of LineProfiler and insert it into the 
__builtins__ namespace with the name profile. It has been written to be used as 
a decorator, so in your script, you decorate the functions you want to profile 
with @profile.

@profile
def slow_function(a, b, c):
    ...

I've used it before (maybe 1 year ago), don't know why it isn't working now.
-- 
-- Those who don't understand recursion are doomed to repeat it

[toc] | [prev] | [next] | [standalone]

#85495

Neal Becker wrote:


> To quote from https://pypi.python.org/pypi/line_profiler/
> 
> $ kernprof -l script_to_profile.py
> kernprof will create an instance of LineProfiler and insert it into the
> __builtins__ namespace with the name profile. 

Ewww!!!! What a Ruby-esque interface, that makes me sad :-( And what if you
have your own profile global name?

And *wrong* too. `__builtins__` is a private CPython implementation detail.
The way to monkey-patch the built-ins in Python 2 is to inject the object
into `__builtin__` (no s), or `builtins` in Python 3. Seeing as
line_profiler is written in C, perhaps the author (Robert Kern) doesn't
care about supporting Jython or IronPython, but there may be Python
implementations (PyPy perhaps?) which can run C code but don't have
__builtins__.


In any case, it sounds like it isn't working. Try creating this simple file:

# demo.py
print(profile)  # Does profile actually get injected into the builtins?


then running it:

kernprof -l demo.py


If that still gives a NameError, you have a bug in kernprof. Or possibly you
have broken it -- perhaps you are accidentally shadowing one of its modules
with a module of your own?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85535

On 2015-02-11 01:17, Steven D'Aprano wrote:
> Neal Becker wrote:
>
>
>> To quote from https://pypi.python.org/pypi/line_profiler/
>>
>> $ kernprof -l script_to_profile.py
>> kernprof will create an instance of LineProfiler and insert it into the
>> __builtins__ namespace with the name profile.
>
> Ewww!!!! What a Ruby-esque interface, that makes me sad :-(

<shrug> This is not a production library. It's a development tool designed to 
help developers shorten the cycle time for investigating these kinds of issues. 
Well, *a* developer; i.e. me. If it helps anyone else, yahtzee!

> And what if you
> have your own profile global name?

Then you can pull it out from __builtin__ with a different name and use that 
other name.

> And *wrong* too. `__builtins__` is a private CPython implementation detail.
> The way to monkey-patch the built-ins in Python 2 is to inject the object
> into `__builtin__` (no s), or `builtins` in Python 3.

And indeed that is how it is implemented. Referring to that namespace as the 
"`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking I've 
implemented it one particular way, if you are desperate to find a nit to pick.

> Seeing as
> line_profiler is written in C, perhaps the author (Robert Kern) doesn't
> care about supporting Jython or IronPython,  but there may be Python
> implementations (PyPy perhaps?) which can run C code but don't have
> __builtins__.

Indeed, I do not care about any of them. PyPy does not implement CPython's 
tracing API:

https://bitbucket.org/pypy/pypy/src/2b2163d65ee437646194a1ceb2a3153db24c5f7e/pypy/module/cpyext/stubs.py?at=default#cl-1286

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

[toc] | [prev] | [next] | [standalone]

#85556

Robert Kern wrote:

> Referring to that namespace as the
> "`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking 
> I've implemented it one particular way

Well it certainly mislead me :-)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85640

Robert Kern wrote:

> On 2015-02-11 01:17, Steven D'Aprano wrote:
>> Neal Becker wrote:
>>
>>
>>> To quote from https://pypi.python.org/pypi/line_profiler/
>>>
>>> $ kernprof -l script_to_profile.py
>>> kernprof will create an instance of LineProfiler and insert it into the
>>> __builtins__ namespace with the name profile.
>>
>> Ewww!!!! What a Ruby-esque interface, that makes me sad :-(
> 
> <shrug> This is not a production library. It's a development tool designed to
> help developers shorten the cycle time for investigating these kinds of
> issues. Well, *a* developer; i.e. me. If it helps anyone else, yahtzee!
> 
>> And what if you
>> have your own profile global name?
> 
> Then you can pull it out from __builtin__ with a different name and use that
> other name.
> 
>> And *wrong* too. `__builtins__` is a private CPython implementation detail.
>> The way to monkey-patch the built-ins in Python 2 is to inject the object
>> into `__builtin__` (no s), or `builtins` in Python 3.
> 
> And indeed that is how it is implemented. Referring to that namespace as the
> "`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking
> I've implemented it one particular way, if you are desperate to find a nit to
> pick.
> 
>> Seeing as
>> line_profiler is written in C, perhaps the author (Robert Kern) doesn't
>> care about supporting Jython or IronPython,  but there may be Python
>> implementations (PyPy perhaps?) which can run C code but don't have
>> __builtins__.
> 
> Indeed, I do not care about any of them. PyPy does not implement CPython's
> tracing API:
> 
> 
https://bitbucket.org/pypy/pypy/src/2b2163d65ee437646194a1ceb2a3153db24c5f7e/pypy/module/cpyext/stubs.py?at=default#cl-1286
> 

Hi Robert, any idea why line_profiler is not working?  I've used it fine in the 
past.

-- 
-- Those who don't understand recursion are doomed to repeat it

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web