Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95784 > unrolled thread

Why Python is not both an interpreter and a compiler?

Back to article view | Back to comp.lang.python

Contents

  Why Python is not both an interpreter and a compiler? Mahan Marwat <mahanmarwat@gmail.com> - 2015-08-31 01:35 -0700
    Re: Why Python is not both an interpreter and a compiler? Chris Angelico <rosuav@gmail.com> - 2015-08-31 18:49 +1000
    Re: Why Python is not both an interpreter and a compiler? Cameron Simpson <cs@zip.com.au> - 2015-08-31 19:59 +1000
    Re: Why Python is not both an interpreter and a compiler? Ben Finney <ben+python@benfinney.id.au> - 2015-08-31 21:41 +1000
      Re: Why Python is not both an interpreter and a compiler? Marko Rauhamaa <marko@pacujo.net> - 2015-08-31 16:39 +0300
        Re: Why Python is not both an interpreter and a compiler? Mahan Marwat <mahanmarwat@gmail.com> - 2015-08-31 10:48 -0700
          Re: Why Python is not both an interpreter and a compiler? Emile van Sebille <emile@fenx.com> - 2015-08-31 10:56 -0700
          Re: Why Python is not both an interpreter and a compiler? Ben Finney <ben+python@benfinney.id.au> - 2015-09-01 09:15 +1000
          Re: Why Python is not both an interpreter and a compiler? Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-09-01 07:45 +0200
            Re: Why Python is not both an interpreter and a compiler? Ian Kelly <ian.g.kelly@gmail.com> - 2015-09-01 11:21 -0600
          Re: Why Python is not both an interpreter and a compiler? Laura Creighton <lac@openend.se> - 2015-09-01 09:51 +0200
          Re: Why Python is not both an interpreter and a compiler? Steven D'Aprano <steve@pearwood.info> - 2015-09-02 01:20 +1000
            Re: Why Python is not both an interpreter and a compiler? Marko Rauhamaa <marko@pacujo.net> - 2015-09-01 19:20 +0300
              Re: Why Python is not both an interpreter and a compiler? Chris Angelico <rosuav@gmail.com> - 2015-09-02 03:06 +1000
              Re: Why Python is not both an interpreter and a compiler? Laura Creighton <lac@openend.se> - 2015-09-01 19:33 +0200
                Re: Why Python is not both an interpreter and a compiler? Marko Rauhamaa <marko@pacujo.net> - 2015-09-01 23:08 +0300
                  Re: Why Python is not both an interpreter and a compiler? Chris Angelico <rosuav@gmail.com> - 2015-09-02 10:50 +1000
                  Re: Why Python is not both an interpreter and a compiler? Laura Creighton <lac@openend.se> - 2015-09-02 09:09 +0200
                    Re: Why Python is not both an interpreter and a compiler? wxjmfauth@gmail.com - 2015-09-02 01:25 -0700
              Re: Why Python is not both an interpreter and a compiler? Laura Creighton <lac@openend.se> - 2015-09-01 19:52 +0200
              Re: Why Python is not both an interpreter and a compiler? Steven D'Aprano <steve@pearwood.info> - 2015-09-02 12:49 +1000
                Re: Why Python is not both an interpreter and a compiler? Marko Rauhamaa <marko@pacujo.net> - 2015-09-02 06:58 +0300
                Re: Why Python is not both an interpreter and a compiler? Jussi Piitulainen <harvesting@makes.address.invalid> - 2015-09-02 08:59 +0300
    Re: Why Python is not both an interpreter and a compiler? Michael Torrie <torriem@gmail.com> - 2015-09-01 21:46 -0600

Page 2 of 2 — ← Prev page 1 [2]

#95851

On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:

> Steven D'Aprano <steve@pearwood.info>:
> 
>> I believe that Marko is wrong. It is not so easy to compile Python to
>> machine language for real machines. That's why the compiler targets a
>> virtual machine instead.
> 
> Somehow Guile manages it even though Scheme is at least as dynamic a
> language as Python.

It's not about the dynamicism precisely, it's about what counts as primitive
data types and operations.

What are the primitives in Scheme? In Python, the primitives are pretty
complex. I don't know how accurate this page is:

https://en.wikibooks.org/wiki/Scheme_Programming/Scheme_Datatypes

but it suggests that Scheme primitives are quite close to the machine. That
may keep the runtime small.


> I never said a compiler would translate Python to (analogous) machine
> language. I said you could easily turn CPython into a dynamic library
> (run-time environment) and write a small bootstrapper that you package
> into an executable archive with the Python code (whether .py or .pyc).
> What results is a single executable that you can run analogously to any
> other command.

Provided you have the correct version of the dynamic library installed in
the correct place. 

But this doesn't solve the problem of being able to distribute a single
executable file that requires no pre-installed libraries, which is the
problem cx_freeze and pytoexe are made to solve. They solve the case when
you can't assume that there is a Python run-time environment available.

If you are going to require a Python run-time environment, let's call
it "pylib", then you might as well require the python compiler and standard
library be installed. (In the case of C, that's not the case, a distinct
run-time environment makes sense, as the compile-time and run-time
environments are sharply delineated in C. One certainly wouldn't want to
have to re-compile the average C application each and every time you run
it.)

Maybe you could separate the REPL and remove it from pylib, but that's
probably quite small, it might not make that much of a difference to the
total size.

Maybe you could build a pylib that was significantly smaller than the Python
interpreter plus stdlib. That's fine, I'm not arguing it can't be done. I'm
arguing that it's not *easy*, if it were easy somebody likely would have
done it by now.

I don't know the state of the art here. Does Cython work in this space? How
about Nuitka?


> In fact, the shebang notation turns any single .py file into such an
> executable.

I trust that you're not actually arguing that distributing .py files meets
the requirement for a standalone application.


> The problem is if you break your program into modules. Java, 
> of course, solved a similar problem with .jar files (but still wouldn't
> jump over the final hurdle of making the .jar files executable).

You can distribute your Python app as a zip file, except of course you still
need the Python interpreter to be installed.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#95853

Steven D'Aprano <steve@pearwood.info>:

> On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:
>> I never said a compiler would translate Python to (analogous) machine
>> language. I said you could easily turn CPython into a dynamic library
>> (run-time environment) and write a small bootstrapper that you
>> package into an executable archive with the Python code (whether .py
>> or .pyc). What results is a single executable that you can run
>> analogously to any other command.
>
> Provided you have the correct version of the dynamic library installed
> in the correct place.

Yes, virtually all Linux software builds upon dynamic libraries that
have been introduced to the linker via ldconfig.

> But this doesn't solve the problem of being able to distribute a single
> executable file that requires no pre-installed libraries, which is the
> problem cx_freeze and pytoexe are made to solve.

I wasn't trying to solve that particular problem. I was simply stating
you could compile (translate, turn) a Python program into a single,
executable file.

> I trust that you're not actually arguing that distributing .py files
> meets the requirement for a standalone application.

If your application consists of a single .py file, why not?

>> The problem is if you break your program into modules. Java, of
>> course, solved a similar problem with .jar files (but still wouldn't
>> jump over the final hurdle of making the .jar files executable).
>
> You can distribute your Python app as a zip file, except of course you
> still need the Python interpreter to be installed.

Again, having a Python interpreter around is not the issue I'm talking
about. I'm talking about the possibility to compile (translate, turn) a
Python program into a single, executable file.

Now, even C programs can suffer from the module problem: you sometimes
need to ship extra dynamic libraries ("modules") with your binary.


Marko

[toc] | [prev] | [next] | [standalone]

#95854

Steven D'Aprano writes:
> On Wed, 2 Sep 2015 02:20 am, Marko Rauhamaa wrote:
>> Steven D'Aprano:
>> 
>>> I believe that Marko is wrong. It is not so easy to compile Python
>>> to machine language for real machines. That's why the compiler
>>> targets a virtual machine instead.
>> 
>> Somehow Guile manages it even though Scheme is at least as dynamic a
>> language as Python.
>
> It's not about the dynamicism precisely, it's about what counts as
> primitive data types and operations.
>
> What are the primitives in Scheme? In Python, the primitives are
> pretty complex. I don't know how accurate this page is:
>
> https://en.wikibooks.org/wiki/Scheme_Programming/Scheme_Datatypes
>
> but it suggests that Scheme primitives are quite close to the
> machine. That may keep the runtime small.

I think I spotted a tiny inaccuracy, but the real problem with that page
is that it's *very* incomplete: no mention of exact integers and
rationals, strings appear in text but not as an entry, were vectors even
mentioned?, nothing about procedures!, let alone reified continuations,
nothing about input/output ports. Also records (definable types with
named fields) are now officially in for some time already, so that's
another kind of structured types. Many implementations have object
systems, possibly modules as objects.

It's quite analogous to Python's objects.

Eval in Scheme is more restricted. Procedure internals are not
accessible at all, and implementations typically offer ways to declare
that standard names in a compilation unit indeed have their standard
meaning and variables will not be assigned outside the given compilation
unit, so the compiler can propagate information about expected argument
types around, infer more types, inline code, and lot's of things that
are way beyond me.

Some Scheme implementations use quite aggressive compilation techniques.
Guile is one, I think. Gambit and Larceny are two others.

[- -]

[toc] | [prev] | [next] | [standalone]

#95852

On 08/31/2015 02:35 AM, Mahan Marwat wrote:
> What I know about an interpreter and a compiler is: they both convert
> source code to machine code and the only difference is, an
> interpreter convert it, line by line while compiler convert the whole
> source file. Now if we compile a C source file on C compiler, it will
> produce a small executable file. But if we compile (freeze) a Python
> source file on Python interpreter (using cx_freeze), it will produce
> a big executable file. Now the thing is C compiler does not embed a C
> compiler inside our program, while the Python interpreter (cx_freeze,
> pytoexe) embed Python interpreter inside our program, what is the
> reason? The question is, why cx_freeze etc... embed Python
> interpreter inside our programs, they are unable to produce machine
> code like C compiler do? Cant we program a language, which is both
> interpreted and compiled? The core developer cant add the compiling
> functionality to Python?

It think your questions have been well answered by others.  But there
are several attempts at making an actual python compiler.  Often this
involve less-dynamic subset of Python.  For example, pypy has a dialect
called rpython which compiles straight to C++ code, and then to machine
code.

Another subset compiler is cython, which is somewhat of a specialized
compiler. It compiles a subset of Python to a binary shared library that
can be imported into a Python program running in the normal interpreter.

The dynamic nature of Python means that probably your best route to
speed is going to be through just-in-time compilation. The pypy project
is an attempt to do JIT with Python. So far the results are very
promising.  Pretty cool since pypy is written in Python and bootstraps
from the standard python interpreter.

Lastly, one attempt at a compiler is nuitka (google it).  It produces
self-contained executables.  Nuitka compiles what it can, and interprets
the rest (if I understand it correctly) by embedding libpython itself in
the executable.  At this time, nuitka isn't focusing on performance,
more correctness.  GvR doesn't really think much of nuitka, but I think
it's a cool project and the developer is a nice guy.  Maybe have its uses.

So far I haven't had a use for nuikta; cPython is enough for me, with
cython for compiling functions that need some more raw speed.  I tend to
use more conventional optimization techniques that work just fine with
the interpreter.  And often the results are fast enough.  For example I
implemented a simple memoization wrapper for a particularly expensive
function that was called a lot, often over the same inputs several
times.  Runtime went from 10 seconds to less than 1.  Enough for me!

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web