Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11543 > unrolled thread

Idea for pure-python templates using AST.

Back to article view | Back to comp.lang.python

Contents

  Idea for pure-python templates using AST. "Paul Wray" <paul.wray@det.nsw.edu.au> - 2011-08-16 21:33 +1000
    Re: Idea for pure-python templates using AST. aspineux <aspineux@gmail.com> - 2011-08-16 05:32 -0700
      Re: Idea for pure-python templates using AST. anand jeyahar <anand.ibmgsi@gmail.com> - 2011-08-16 21:46 +0530
      Re: Idea for pure-python templates using AST. Paul Wray <paulwray111111@gmail.com> - 2011-08-16 18:10 -0700
    Re: Idea for pure-python templates using AST. Terry Reedy <tjreedy@udel.edu> - 2011-08-16 11:47 -0400
    Re: Idea for pure-python templates using AST. Chris Angelico <rosuav@gmail.com> - 2011-08-16 17:14 +0100
      Re: Idea for pure-python templates using AST. Paul Wray <paulwray111111@gmail.com> - 2011-08-16 16:57 -0700
        Re: Idea for pure-python templates using AST. Chris Angelico <rosuav@gmail.com> - 2011-08-17 01:03 +0100
    Re: Idea for pure-python templates using AST. Irmen de Jong <irmen@-NOSPAM-xs4all.nl> - 2011-08-16 21:23 +0200
      Re: Idea for pure-python templates using AST. Paul Wray <paulwray111111@gmail.com> - 2011-08-16 18:02 -0700
    Re: Idea for pure-python templates using AST. Tim Roberts <timr@probo.com> - 2011-08-16 17:39 -0700

#11543 — Idea for pure-python templates using AST.

Hello all

Ive had what I think is a great idea for pure-python templates (I can almost 
hear the groans, bear with me...)

For the impatient, proof of concept is at http://pastie.org/2379978 
demonstrating simple substitution, balanced tags using context manager, 
subtemplates,  and template inheritance.

I'm posting here to get opinions on:
* the merits of the idea, (or otherwise!)
* whether there are any established/mature templating systems that use this 
approach, or whether its come up before,
* ideas for improvements and advice on other aspects such as sandboxing
* of course, to share the idea in case others want to use it

Background: Ive been working on an application that recursively renders 
html/xml documents from heterogenoeus trees, with separate classes for each 
document component. First I built my own renderer but was dissatisfied with 
the repetitive code. Then looked at Mako and Jinja, and used Jinja but was 
still disatisfied, because I still needed a render method in each class to 
do preparation, and also the template which was centrally managed by the 
Jinja loader and environment. I found a way to call templates recursively 
via Jinja filters, but was not sure that it wouldnt blow up in my face, so I 
also had separate calls to render the children of each node, and then feed 
the value to the parent template. I kept thinking that there must be a way 
to get the brevity and clarity of declarative templates, plus the simplicity 
of pure python loops, tests and function calls.

The idea:
Python syntax allows a statement to be a bare literal or identifier. These 
have no effect on the program.

So the function below is legal python:

def myFunc():
    'a'
    x = 45
    'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
        '<h1>'; self.heading; '</h1>'
        '<ul>'
        for itm in self.items:
            '<li>'; itm; '</li>'
        '</ul>'

The idea is simply to use python ASTs to transform this code so that it 
accumulates the values of the bare expressions.

I think this give the best of both worlds - declarative syntax for the 
template literals, but in a pure python context, giving you all the power of 
python control statements, classes etc.

For application contexts outside pure python code (ie user-created 
templates) , it would be simple to extend the technique to creating template 
functions from strings, and insert the resulting methods into a namespace 
for execution.)

I think, given the technique is already working with the AST, that 
sandboxing should not be too hard either - advice on this aspect would be 
appreciated.

Paul Wray 

[toc] | [next] | [standalone]

#11547

On Aug 16, 1:33 pm, "Paul Wray" <paul.w...@det.nsw.edu.au> wrote:
> Hello all
>
> Ive had what I think is a great idea for pure-python templates (I can almost
> hear the groans, bear with me...)
>
> For the impatient, proof of concept is athttp://pastie.org/2379978
> demonstrating simple substitution, balanced tags using context manager,
> subtemplates,  and template inheritance.

You code fail, see below for other comment

Traceback (most recent call last):
  File "Download/pastie-2379978.rb", line 108, in <module>
    make_template(template1)
  File "Download/pastie-2379978.rb", line 60, in make_template
    ast.fix_missing_locations(astFromSrc)
  File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
    _fix(node, 1, 0)
  File "/usr/lib/python2.6/ast.py", line 132, in _fix
    _fix(child, lineno, col_offset)
  File "/usr/lib/python2.6/ast.py", line 132, in _fix
    _fix(child, lineno, col_offset)
  File "/usr/lib/python2.6/ast.py", line 121, in _fix
    if 'lineno' in node._attributes:
AttributeError: 'arguments' object has no attribute '_attributes'




>
> I'm posting here to get opinions on:
> * the merits of the idea, (or otherwise!)
> * whether there are any established/mature templating systems that use this
> approach, or whether its come up before,
> * ideas for improvements and advice on other aspects such as sandboxing
> * of course, to share the idea in case others want to use it

This is very original ! First time I see it. I like it.
But how to debug large template ?
How to find/detect a missing </TAG> ?
This is very important. This is one big advantage of Genshi over Kid
How to report the correct error at the correct line ?
How to find/get nice editor to edit large template ?


>
> Background: Ive been working on an application that recursively renders
> html/xml documents from heterogenoeus trees, with separate classes for each
> document component. First I built my own renderer but was dissatisfied with
> the repetitive code. Then looked at Mako and Jinja, and used Jinja but was
> still disatisfied, because I still needed a render method in each class to
> do preparation, and also the template which was centrally managed by the
> Jinja loader and environment. I found a way to call templates recursively
> via Jinja filters, but was not sure that it wouldnt blow up in my face, so I
> also had separate calls to render the children of each node, and then feed
> the value to the parent template. I kept thinking that there must be a way
> to get the brevity and clarity of declarative templates, plus the simplicity
> of pure python loops, tests and function calls.
>
> The idea:
> Python syntax allows a statement to be a bare literal or identifier. These
> have no effect on the program.
>
> So the function below is legal python:
>
> def myFunc():
>     'a'
>     x = 45
>     'b'; 'c'; x
>
> So is this (within the appropriate class context of course):
>
> def body(self, r):
>         '<h1>'; self.heading; '</h1>'
>         '<ul>'
>         for itm in self.items:
>             '<li>'; itm; '</li>'
>         '</ul>'
>
> The idea is simply to use python ASTs to transform this code so that it
> accumulates the values of the bare expressions.

You could call it PHP :-)

>
> I think this give the best of both worlds - declarative syntax for the
> template literals, but in a pure python context, giving you all the power of
> python control statements, classes etc.
>
> For application contexts outside pure python code (ie user-created
> templates) , it would be simple to extend the technique to creating template
> functions from strings, and insert the resulting methods into a namespace
> for execution.)
>
> I think, given the technique is already working with the AST, that
> sandboxing should not be too hard either - advice on this aspect would be
> appreciated.


Maybe a good idea.
But still a lot of work to get the level of existing libraries.

Maybe better if mixed with other tools like Genshi to create widget.
Maybe next toscawidget could use such a technique.


Thanks for sharing

>
> Paul Wray

[toc] | [prev] | [next] | [standalone]

#11576

Hi all,
   I did it. Finally managed to port mysqltuner.pl to python. Was a
real pain in the butt doing it from bottom up manually, without ever
really learing perl syntax. But i finally got it done. Now i need help
testing it. find it here.
 git@github.com:anandjeyahar/mysqlDbAdmin-python.git.

Also i never really thought about design. Just blindly/mechanically,
translated from perl to python. So criticize and let me know how i can
improve this.

Thanks and Regards,
==============================================
Anand Jeyahar
https://sites.google.com/site/anandjeyahar
==============================================
The man who is really serious,
with the urge to find out what truth is,
has no style at all. He lives only in what is.
                  ~Bruce Lee

Love is a trade with lousy accounting policies.
                 ~Aang Jie




On Tue, Aug 16, 2011 at 18:02, aspineux <aspineux@gmail.com> wrote:
> On Aug 16, 1:33 pm, "Paul Wray" <paul.w...@det.nsw.edu.au> wrote:
>> Hello all
>>
>> Ive had what I think is a great idea for pure-python templates (I can almost
>> hear the groans, bear with me...)
>>
>> For the impatient, proof of concept is athttp://pastie.org/2379978
>> demonstrating simple substitution, balanced tags using context manager,
>> subtemplates,  and template inheritance.
>
> You code fail, see below for other comment
>
> Traceback (most recent call last):
>  File "Download/pastie-2379978.rb", line 108, in <module>
>    make_template(template1)
>  File "Download/pastie-2379978.rb", line 60, in make_template
>    ast.fix_missing_locations(astFromSrc)
>  File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
>    _fix(node, 1, 0)
>  File "/usr/lib/python2.6/ast.py", line 132, in _fix
>    _fix(child, lineno, col_offset)
>  File "/usr/lib/python2.6/ast.py", line 132, in _fix
>    _fix(child, lineno, col_offset)
>  File "/usr/lib/python2.6/ast.py", line 121, in _fix
>    if 'lineno' in node._attributes:
> AttributeError: 'arguments' object has no attribute '_attributes'
>
>
>
>
>>
>> I'm posting here to get opinions on:
>> * the merits of the idea, (or otherwise!)
>> * whether there are any established/mature templating systems that use this
>> approach, or whether its come up before,
>> * ideas for improvements and advice on other aspects such as sandboxing
>> * of course, to share the idea in case others want to use it
>
> This is very original ! First time I see it. I like it.
> But how to debug large template ?
> How to find/detect a missing </TAG> ?
> This is very important. This is one big advantage of Genshi over Kid
> How to report the correct error at the correct line ?
> How to find/get nice editor to edit large template ?
>
>
>>
>> Background: Ive been working on an application that recursively renders
>> html/xml documents from heterogenoeus trees, with separate classes for each
>> document component. First I built my own renderer but was dissatisfied with
>> the repetitive code. Then looked at Mako and Jinja, and used Jinja but was
>> still disatisfied, because I still needed a render method in each class to
>> do preparation, and also the template which was centrally managed by the
>> Jinja loader and environment. I found a way to call templates recursively
>> via Jinja filters, but was not sure that it wouldnt blow up in my face, so I
>> also had separate calls to render the children of each node, and then feed
>> the value to the parent template. I kept thinking that there must be a way
>> to get the brevity and clarity of declarative templates, plus the simplicity
>> of pure python loops, tests and function calls.
>>
>> The idea:
>> Python syntax allows a statement to be a bare literal or identifier. These
>> have no effect on the program.
>>
>> So the function below is legal python:
>>
>> def myFunc():
>>     'a'
>>     x = 45
>>     'b'; 'c'; x
>>
>> So is this (within the appropriate class context of course):
>>
>> def body(self, r):
>>         '<h1>'; self.heading; '</h1>'
>>         '<ul>'
>>         for itm in self.items:
>>             '<li>'; itm; '</li>'
>>         '</ul>'
>>
>> The idea is simply to use python ASTs to transform this code so that it
>> accumulates the values of the bare expressions.
>
> You could call it PHP :-)
>
>>
>> I think this give the best of both worlds - declarative syntax for the
>> template literals, but in a pure python context, giving you all the power of
>> python control statements, classes etc.
>>
>> For application contexts outside pure python code (ie user-created
>> templates) , it would be simple to extend the technique to creating template
>> functions from strings, and insert the resulting methods into a namespace
>> for execution.)
>>
>> I think, given the technique is already working with the AST, that
>> sandboxing should not be too hard either - advice on this aspect would be
>> appreciated.
>
>
> Maybe a good idea.
> But still a lot of work to get the level of existing libraries.
>
> Maybe better if mixed with other tools like Genshi to create widget.
> Maybe next toscawidget could use such a technique.
>
>
> Thanks for sharing
>
>>
>> Paul Wray
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#11642

> You code fail, see below for other comment
>
> Traceback (most recent call last):
>   File "Download/pastie-2379978.rb", line 108, in <module>
>     make_template(template1)
>   File "Download/pastie-2379978.rb", line 60, in make_template
>     ast.fix_missing_locations(astFromSrc)
>   File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
>     _fix(node, 1, 0)
>   File "/usr/lib/python2.6/ast.py", line 132, in _fix
>     _fix(child, lineno, col_offset)
>   File "/usr/lib/python2.6/ast.py", line 132, in _fix
>     _fix(child, lineno, col_offset)
>   File "/usr/lib/python2.6/ast.py", line 121, in _fix
>     if 'lineno' in node._attributes:
> AttributeError: 'arguments' object has no attribute '_attributes'
>

Sorry, should have specified Python 2.7.
Also Pastie thinks its ruby code so the donwloaded file is confusingly
___.rb

[toc] | [prev] | [next] | [standalone]

#11569

On 8/16/2011 7:33 AM, Paul Wray wrote:
> Hello all
>
> Ive had what I think is a great idea for pure-python templates (I can
> almost hear the groans, bear with me...)
>
> For the impatient, proof of concept is at http://pastie.org/2379978
> demonstrating simple substitution, balanced tags using context manager,
> subtemplates, and template inheritance.
>
> I'm posting here to get opinions on:
> * the merits of the idea, (or otherwise!)
> * whether there are any established/mature templating systems that use
> this approach, or whether its come up before,
> * ideas for improvements and advice on other aspects such as sandboxing
> * of course, to share the idea in case others want to use it
>
> Background: Ive been working on an application that recursively renders
> html/xml documents from heterogenoeus trees, with separate classes for
> each document component. First I built my own renderer but was
> dissatisfied with the repetitive code. Then looked at Mako and Jinja,
> and used Jinja but was still disatisfied, because I still needed a
> render method in each class to do preparation, and also the template
> which was centrally managed by the Jinja loader and environment. I found
> a way to call templates recursively via Jinja filters, but was not sure
> that it wouldnt blow up in my face, so I also had separate calls to
> render the children of each node, and then feed the value to the parent
> template. I kept thinking that there must be a way to get the brevity
> and clarity of declarative templates, plus the simplicity of pure python
> loops, tests and function calls.
>
> The idea:
> Python syntax allows a statement to be a bare literal or identifier.
> These have no effect on the program.

More generally, Python has expression statements, with the result of the 
expression ignored. These are usually function calls with side-effects. 
"print('x')" has no effect on the program and the return value is 
usually ignored.

> So the function below is legal python:
>
> def myFunc():
> 'a'
> x = 45
> 'b'; 'c'; x
>
> So is this (within the appropriate class context of course):
>
> def body(self, r):
> '<h1>'; self.heading; '</h1>'
> '<ul>'
> for itm in self.items:
> '<li>'; itm; '</li>'
> '</ul>'
>
> The idea is simply to use python ASTs to transform this code so that it
> accumulates the values of the bare expressions.

Interesting idea, though I have no experience for comparison.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#11575

On Tue, Aug 16, 2011 at 12:33 PM, Paul Wray <paul.wray@det.nsw.edu.au> wrote:
> The idea is simply to use python ASTs to transform this code so that it
> accumulates the values of the bare expressions.

That'd be similar to what the interactive loop does. Are you aware,
though, that docstrings are bare expressions? You may have a syntactic
collision there.

ChrisA

[toc] | [prev] | [next] | [standalone]

#11627

On Aug 17, 2:14 am, Chris Angelico <ros...@gmail.com> wrote:
> On Tue, Aug 16, 2011 at 12:33 PM, Paul Wray <paul.w...@det.nsw.edu.au> wrote:
> > The idea is simply to use python ASTs to transform this code so that it
> > accumulates the values of the bare expressions.
>
> That'd be similar to what the interactive loop does. Are you aware,
> though, that docstrings are bare expressions? You may have a syntactic
> collision there.
>
> ChrisA

Thanks yes ama aware of docstrings but did not consider.
They are easy to strip out though.

[toc] | [prev] | [next] | [standalone]

#11629

On Wed, Aug 17, 2011 at 12:57 AM, Paul Wray <paulwray111111@gmail.com> wrote:
> Thanks yes ama aware of docstrings but did not consider.
> They are easy to strip out though.
>

Maybe. You'd have to take notice of what's a docstring and what's the
first element to be outputted. Or alternatively, just forbid
docstrings on those functions.

ChrisA

[toc] | [prev] | [next] | [standalone]

#11605

On 16-08-11 13:33, Paul Wray wrote:

> The idea:
> Python syntax allows a statement to be a bare literal or identifier.
> These have no effect on the program.
>
> So the function below is legal python:
>
> def myFunc():
> 'a'
> x = 45
> 'b'; 'c'; x
>
> So is this (within the appropriate class context of course):
>
> def body(self, r):
> '<h1>'; self.heading; '</h1>'
> '<ul>'
> for itm in self.items:
> '<li>'; itm; '</li>'
> '</ul>'
>

Looks very similar to PTL what Quixote uses:
http://www.quixote.ca/overview/paper.html

(never used it though, and I doubt Quixote used ASTs)

Irmen

[toc] | [prev] | [next] | [standalone]

#11641

On Aug 17, 5:23 am, Irmen de Jong <ir...@-NOSPAM-xs4all.nl> wrote:
> On 16-08-11 13:33, Paul Wray wrote:
>
>
>
>
>
>
>
>
>
> > The idea:
> > Python syntax allows a statement to be a bare literal or identifier.
> > These have no effect on the program.
>
> > So the function below is legal python:
>
> > def myFunc():
> > 'a'
> > x = 45
> > 'b'; 'c'; x
>
> > So is this (within the appropriate class context of course):
>
> > def body(self, r):
> > '<h1>'; self.heading; '</h1>'
> > '<ul>'
> > for itm in self.items:
> > '<li>'; itm; '</li>'
> > '</ul>'
>
> Looks very similar to PTL what Quixote uses:http://www.quixote.ca/overview/paper.html
>
> (never used it though, and I doubt Quixote used ASTs)
>
> Irmen

Thanks. Yes it looks much the same, and it does use AST. This from the
ptl_compile.py module:

"""Compile a PTL template.

First template function names are mangled, noting the template type.
Next, the file is parsed into a parse tree.  This tree is converted
into
a modified AST.  It is during this state that the semantics are
modified
by adding extra nodes to the tree.  Finally bytecode is generated
using
the compiler package.
"""

[toc] | [prev] | [next] | [standalone]

#11637

"Paul Wray" <paul.wray@det.nsw.edu.au> wrote:
>
>Ive had what I think is a great idea for pure-python templates (I can almost 
>hear the groans, bear with me...)
>...
>The idea:
>Python syntax allows a statement to be a bare literal or identifier. These 
>have no effect on the program.
>...
>So is this (within the appropriate class context of course):
>
>def body(self, r):
>        '<h1>'; self.heading; '</h1>'
>        '<ul>'
>        for itm in self.items:
>            '<li>'; itm; '</li>'
>        '</ul>'

This is essentially how the original CherryPy version 1 web framework
worked.    In the end, I think it was decided that this represented too
much of a mix of processing and presentation, and CherryPy 2 and 3 use a
different scheme.
-- 
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web