Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #109671 > unrolled thread

Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

Back to article view | Back to comp.lang.python

Contents

  Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy rocky <rocky@gnu.org> - 2016-06-08 03:38 -0700
    Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy Robin Becker <robin@reportlab.com> - 2016-06-08 17:50 +0100
      Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy rocky <rocky@gnu.org> - 2016-06-08 11:32 -0700
        Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy Robin Becker <robin@reportlab.com> - 2016-06-10 09:33 +0100
          Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy rocky <rocky@gnu.org> - 2016-06-10 17:56 -0700
      Re: Want to play with or learn a parser system including a grammar   for Python? See spark_parser on pypy Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-06-09 10:22 +1200
    Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-06-08 22:36 -0700
      Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy rocky <rocky@gnu.org> - 2016-06-10 18:00 -0700

#109671 — Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

For those who are interested in experimenting with parser systems in Python, there has been one around for a long while. But in my opinion it was a bit lacking in graded example demonstrating how to use it.

So in the recently in spark_parser 1.3.0 [1], I've beefed up the examples a little. In addition to the example programs which give the classic arithmetic expression evaluator, I now include the beginnings of a full Python 2.6 language. The semantic routines just reformat the input which was parsed into a tree and spit that back out formatted according to predefined rules. Included is the grammar file from the 2.6.9 distribution.

For those who want to go further and see a full production-like example, see the uncompyle6 github repository[2] which uses this. It reads bytecode from Python versions 2.3 to 3.5 or so and produces the corresponding Python 2 or Python 3 source code.

Alternatively if you want to understand how uncompyle6 works, such as to improve it or fix bugs, you'll probably need to understand how the parser system it uses works.

[1] https://pypi.python.org/pypi/spark_parser/1.3.0
[2] https://github.com/rocky/python-uncompyle6

[toc] | [next] | [standalone]

#109682

On 08/06/2016 11:38, rocky wrote:
...........
> [1] https://pypi.python.org/pypi/spark_parser/1.3.0
.......
the page above shows one can implement a time travel machine as it boldly states

"The original version of this was written by John Aycock and was described in 
his 1988 paper: “Compiling Little Languages in Python” at the 7th International 
Python Conference."

I suppose those early primitive pythons had some issue with integers as 
wackypedia states that

"Python was conceived in the late 1980s[1] and its implementation was started in 
December 1989[2] by Guido van Rossum at CWI in the Netherlands"

so that Aycocks's paper must have been at the -1st Python Conference
-parallely yrs-
Robin Becker

[toc] | [prev] | [next] | [standalone]

#109685

On Wednesday, June 8, 2016 at 12:50:57 PM UTC-4, Robin Becker wrote:
> On 08/06/2016 11:38, rocky wrote:
> ...........
> > [1] https://pypi.python.org/pypi/spark_parser/1.3.0
> .......
> the page above shows one can implement a time travel machine as it boldly states
> 
> "The original version of this was written by John Aycock and was described in 
> his 1988 paper: “Compiling Little Languages in Python” at the 7th International 
> Python Conference."
> 
> I suppose those early primitive pythons had some issue with integers as 
> wackypedia states that
> 
> "Python was conceived in the late 1980s[1] and its implementation was started in 
> December 1989[2] by Guido van Rossum at CWI in the Netherlands"
> 
> so that Aycocks's paper must have been at the -1st Python Conference
> -parallely yrs-
> Robin Becker

Sorry that should have been 1998 which would make more sense for the 7th  conference if the 1st one was around 2001. I've corrected the date in [1] https://pypi.python.org/pypi/spark_parser/1.3.0 

The automated tests in the package just don't  catch stuff like this. But I'm sure there are also other mistakes as well in there so feel free to let me know.

[toc] | [prev] | [next] | [standalone]

#109776

On 08/06/2016 19:32, rocky wrote:
..........
>
> Sorry that should have been 1998 which would make more sense for the 7th  conference if the 1st one was around 2001. I've corrected the date in [1] https://pypi.python.org/pypi/spark_parser/1.3.0
>
> The automated tests in the package just don't  catch stuff like this. But I'm sure there are also other mistakes as well in there so feel free to let me know.
>
not a big deal; I like the spark parser :)
-- 
Robin Becker

[toc] | [prev] | [next] | [standalone]

#109807

On Friday, June 10, 2016 at 4:33:28 AM UTC-4, Robin Becker wrote:
> On 08/06/2016 19:32, rocky wrote:
> ..........
> >
> > Sorry that should have been 1998 which would make more sense for the 7th  conference if the 1st one was around 2001. I've corrected the date in [1] https://pypi.python.org/pypi/spark_parser/1.3.0
> >
> > The automated tests in the package just don't  catch stuff like this. But I'm sure there are also other mistakes as well in there so feel free to let me know.
> >
> not a big deal; I like the spark parser :)
> -- 
> Robin Becker

It is cool for how small in code size, how general, (more general than LL or LR), and how fast it works for well-crafted grammars. Surprisingly little was needed to make it work on Python 3 (as well as addition on Python 2).

It has however long been neglected. John stopped working on it around 2002 or so without having put it on Pypy. Since it wasn't it's own package, so you had to find some other project (with possibly varying versions of the program) and copy the spark.py file from that or download the "0.7 pre-alpha" version of 2002 from John's website and possibly modify it if you have Python 3. 

Ok. So I've got that addressed by putting it in pypy. 

But the other basic area of neglect as documentation. John's paper I find woefully inadequate. Most of the projects on John's site that say they use SPARK are now dead links. And using the various uncompyle/decompyle programs as a tutorial for the parser is a bit whacky too as these are very complicated, and not intended as a tutorial.

I've only started to address this by putting together some simple examples. And on http://github.com/rocky/python-spark there is now a wiki to try to address some of this as well. In the wiki for http://github.com/rocky/python-uncompyle6 I've extracted the documentation for the table-driven formatting, but more is needed here as well.

So if you or other like this and know to to use it, please consider beefing up the wiki. Thanks.

[toc] | [prev] | [next] | [standalone]

#109691 — Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

Robin Becker wrote:
> "Python was conceived in the late 1980s[1] and its implementation was 
> started in December 1989[2] by Guido van Rossum at CWI in the Netherlands"
> 
> so that Aycocks's paper must have been at the -1st Python Conference

When the time machine was invented, Guido thought it would be
fun to go back and hold a Python conference a little before
Python actually existed, to mess with future historians
trying to put together a timeline for Python development.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#109697

On Wednesday, June 8, 2016 at 10:39:00 PM UTC+12, rocky wrote:

> In addition to the example programs which give the classic arithmetic
> expression evaluator, I now include the beginnings of a full Python 2.6
> language.

Does anybody bother with LR(k) parsers any more?

[toc] | [prev] | [next] | [standalone]

#109808

On Thursday, June 9, 2016 at 1:36:56 AM UTC-4, Lawrence D’Oliveiro wrote:
> On Wednesday, June 8, 2016 at 10:39:00 PM UTC+12, rocky wrote:
> 
> > In addition to the example programs which give the classic arithmetic
> > expression evaluator, I now include the beginnings of a full Python 2.6
> > language.
> 
> Does anybody bother with LR(k) parsers any more?

I don't understand. Python itself I think does. And By the way, SPARK is an Early-Algorithm parser [1], so it is a little more general than LL or LR parsers. 

[1] https://en.wikipedia.org/wiki/Earley_parser

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web