Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #20623 > unrolled thread

Re: #line in python (dirty tricks)

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: #line in python (dirty tricks) Ross Boylan <ross@biostat.ucsf.edu> - 2012-02-20 07:08 -0800
    Re: #line in python (dirty tricks) Duncan Booth <duncan.booth@invalid.invalid> - 2012-02-20 18:54 +0000

#20623 — Re: #line in python (dirty tricks)

Duncan Booth wrote
________________________________________________________________________
> Ross Boylan <ross at biostat.ucsf.edu> wrote:
> 
> > As an extension or alternate, could there be a decorator like
> > @source_line(lineno, filename)
> > for classes and methods that could do the conversion on the fly?  I
> > don't know if there's a way to go from the function (or class) object
> > the decorator receives to the AST.
> > 
> No [easy] way to go from bytecodes back to AST, but I see no reason why you 
> can't create a new code object with your filename and line numbers and then 
> create a new function using your modified code object.
Could you elaborate?  I don't understand what you are suggesting.
> 
> If you don't have a 1:1 correspondence of lines then you'll need to pick 
> out all the existing line numbers from the code object co_lnotab and modify 
> them: see dis.py findlinestarts() for how to do this.
> 
> Classes would be harder: the decorator doesn't run until after the class 
> body has executed, so you can't change the line numbers that way until it's 
> too late. The only thing I can think would be to put all of the generated 
> code inside a function and fix up that function with a decorator that scans 
> the bytecode to find all contained classes and fix them up.
> 
> Or you could generate a .pyc file and then fix up line numbers in the whole 
> file: see 
> http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html for 
> some code that shows you what's in a .pyc
> 
My latest concept is to produce code that rewrites itself.  Suppose the
naive file would be
----------- mycode.py (naive) --------------------------------
class SomeClass:
    "class comment"

    def some_function(self, bar):
        pass
--------- end -------------------------------

Protect that code by putting  an "if 0:" in front of it and indenting
each line one space.  Than prepend a bit of code to do the rewriting,
and add indicators of the original line numbers.

----------- mycode.py (after wrapping) -------------
from detangle import detangle
detangle("mycode.py", "mycode.nw")
if 0:
 # original code goes here
 class SomeClass:
    "class comment"
 #and when line numbering changes
 #line 35
    def some_function(self, bar):
       pass
------------- end -------------------
I would write detangle so that it scans through the file in which it
appears (named in the first argument), rewriting so that it appears to
come from the original file (mycode.nw) given in the second argument.

The scanning would look for the "if 0:" in the file.  At that point it
would accumulate code by reading lines and stripping the leading space.
If it found a #line directive it would remember it and then remove it
from the string it was accumulating.  Finally, detangle would would
pass the string of code to ast.compile, catching any syntax errors and
rewriting the file and line number (I might rewrite columns too with an
extension) and then rethrowing them.

If compilation succeeded detangle could rewrite the AST and then exec
it.

Ross

[toc] | [next] | [standalone]

#20631

Ross Boylan <ross@biostat.ucsf.edu> wrote:

>> No [easy] way to go from bytecodes back to AST, but I see no reason
>> why you can't create a new code object with your filename and line
>> numbers and then create a new function using your modified code
>> object. 
> Could you elaborate?  I don't understand what you are suggesting.

This (written for Python 3.x, needs some attribute names changing for 
Python 2.x:

import functools
def source_line(lineno, filename = None):
    def decorator(f):
        c = f.__code__
        code = type(c)(
            c.co_argcount, c.co_kwonlyargcount, c.co_nlocals, 
c.co_stacksize, c.co_flags, c.co_code,
            c.co_consts, c.co_names, c.co_varnames,
            c.co_filename if filename is None else filename,
            c.co_name,
            lineno,
            c.co_lnotab, c.co_freevars, c.co_cellvars)
        f.__code__ = code
        return f
    return decorator


@source_line(43, 'foo.bar')
def foo():
    """This is foo"""
    for i in range(10):
        bar()

@source_line(665, 'evil.txt')
def bar():
    raise RuntimeError("oops")

if __name__=='__main__':
    import traceback
    try:
        foo()
    except RuntimeError as ex:
        traceback.print_exc()

When you run it the output looks something like this (if you create a 
file evil.txt with 667 lines):

Traceback (most recent call last):
  File "C:\temp\foo.py", line 30, in <module>
    foo()
  File "foo.bar", line 47, in foo
  File "evil.txt", line 667, in bar
    Evil line 667
RuntimeError: oops

-- 
Duncan Booth http://kupuguy.blogspot.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web