Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!eternal-september.org!feeder.eternal-september.org!border1.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'anyway.': 0.05; 'interpreter': 0.05; 'lemburg': 0.05; 'startup': 0.05; 'subject:Python': 0.06; '(using': 0.07; 'amtsgericht': 0.07; 'binary': 0.07; 'germany.': 0.07; 'hrb': 0.07; 'marc-andre': 0.07; 'subject:ANN': 0.07; 'subject:file': 0.07; 'm.-a.': 0.09; 'parsed': 0.09; 'run,': 0.09; 'worse': 0.09; 'cc:addr:python- list': 0.11; 'python': 0.11; 'itself.': 0.14; '(#1,': 0.16; '2.1.3': 0.16; '46611': 0.16; 'adapter': 0.16; 'assumptions': 0.16; 'awesome.': 0.16; 'd-40764': 0.16; 'dipl.-math.': 0.16; 'duesseldorf:': 0.16; 'egenix.com': 0.16; 'included,': 0.16; 'langenfeld,': 0.16; 'mxdatetime,': 0.16; 'mxodbc,': 0.16; 'mxtexttools': 0.16; 'stdlib.': 0.16; 'sys.path': 0.16; 'sysconfig': 0.16; 'to:addr:mal': 0.16; 'to:name:m.-a. lemburg': 0.16; 'url:egenix': 0.16; 'apps': 0.16; 'skip:_ 70': 0.16; 'wrote:': 0.18; 'app': 0.19; 'projects,': 0.19; 'seems': 0.21; '>>>': 0.22; 'memory': 0.22; 'community.': 0.22; 'gmbh': 0.22; 'cc:addr:python.org': 0.22; '>>>': 0.24; 'byte': 0.24; 'certainly': 0.24; 'guys': 0.24; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; 'source': 0.25; '>': 0.26; 'header:In-Reply- To:1': 0.27; 'external': 0.29; 'scanned': 0.29; 'mode': 0.30; 'message-id:@mail.gmail.com': 0.30; 'code': 0.31; 'software,': 0.31; 'probably': 0.32; "we're": 0.32; 'run': 0.32; 'open': 0.33; 'url:python': 0.33; 'running': 0.33; 'ceo': 0.33; 'guess': 0.33; 'could': 0.34; 'tool': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'add': 0.35; 'really': 0.36; 'url:zope': 0.36; 'next': 0.36; "didn't": 0.36; 'possible': 0.36; 'should': 0.36; 'application': 0.37; 'so,': 0.37; 'performance': 0.37; 'being': 0.38; 'files': 0.38; 'embedded': 0.39; 'days': 0.60; 'even': 0.60; 'read': 0.60; 'most': 0.60; 'simply': 0.61; 'complete': 0.62; 'zip': 0.64; 'more': 0.64; 'total': 0.65; 'forward': 0.65; 'services': 0.66; 'here': 0.66; 'subject:One': 0.74; 'try,': 0.84; 'reside': 0.91; 'archive.': 0.93; 'advertise': 0.95 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=1sF7uyeA78emCodqzBkL3gIal3z8Ae+TB2sr/h5JdHQ=; b=Bcz9CG0AbuJPWzmLsJcsdSazWFrhtpxbt97Znr11T/9DnvjOMJ678w9I288s6L4X3o CYmG8tkR2MNDmBAT0nfhuqcFdlrq4i3XZdpkGzOutAL6emThb9sDIT2DihdJDetKaci7 xIPkX9kD02c9yHy1ubqQ6VT23zod/OuKriZHpKxJi1yRs3JpKPOa2qrPbAXOejE2COZD 6ROPRHERMPgr3lVVKYXLy5jARH7VzJjk7xEG+CFGxs2kYeUW6LARJQoQnQW310EAenwo 34LG3rxqBDvZr59dOJsBXdpzZWChHVTsQr8LQWQNoPqEHxgYxZzLvx7WASvR8bytFeMh jawg== X-Received: by 10.202.12.142 with SMTP id 136mr2769731oim.30.1431527682861; Wed, 13 May 2015 07:34:42 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <55535FE6.2090007@egenix.com> References: <555346E3.4030400@egenix.com> <55535FE6.2090007@egenix.com> From: Cristiano Cortezia Date: Wed, 13 May 2015 11:34:22 -0300 Subject: Re: ANN: eGenix PyRun - One file Python Runtime 2.1.0 To: "M.-A. Lemburg" Cc: python-list@python.org Content-Type: multipart/alternative; boundary=001a113d1cec05f2080515f783d7 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.20+ Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 190 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1431527692 news.xs4all.nl 2838 [2001:888:2000:d::a6]:40135 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:90553 --001a113d1cec05f2080515f783d7 Content-Type: text/plain; charset=UTF-8 > > In one of the next releases we'll probably add a tool to bundle > complete applications together with pyrun, perhaps even by > recompiling it to include the application byte code files > right in the binary like we do for the stdlib. Well, that would be simply awesome. Looking forward to it. PS: you guys should definitely advertise this work on the embedded software community. 2015-05-13 11:29 GMT-03:00 M.-A. Lemburg : > On 13.05.2015 16:09, Cristiano Cortezia wrote: > > Well I gave it a try, and it seems my assumptions were *somehow* true. > > Here is what I got when running one of my apps in single shot mode (load, > > run, terminate): > > > > *default python distribution* > > total time 9.022s > > ENOENT's count 7377 > > > > *pyrun* > > total time 8.455s > > ENOENT's count 3064 > > > > So, it indeed failed much less to open files, but I guess this didn't > make > > that much difference after all (500ms). > > PyRun has the advantage of being able to read the byte code > directly from the binary (using memory mapping). However, > it still needs to run the same startup machinery as Python > itself. > > Note that startup time for Python was a lot worse before > Python used the same approach as PyRun to compile in the > parsed sysconfig data. > > > Perhaps it is because this app has some external dependencies (22 to be > > precise) not bundled on pyrun that had to be scanned by the interpreter > > anyway. If by any means we could bundle them all the same way, than it > > could bring a much higher performance gain. But I guess it is not really > > safe-feasible. > > It's certainly possible to use the pyrun build system to create > bundles with more packages and tools included. > > The one we're shipping has most of the stdlib included, > but leaves all the application code to reside on the > sys.path or in a ZIP archive. > > In one of the next releases we'll probably add a tool to bundle > complete applications together with pyrun, perhaps even by > recompiling it to include the application byte code files > right in the binary like we do for the stdlib. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, May 13 2015) > >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ > >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2015-05-13: Released mxODBC Connect 2.1.3 ... http://egenix.com/go75 > 2015-05-11: Released eGenix PyRun 2.1.0 ... http://egenix.com/go74 > 2015-05-25: PyWaw Summit 2015, Warsaw, Poland ... 12 days to go > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > --001a113d1cec05f2080515f783d7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
In one of the next releases we'll probably add a tool to bundle
complete applications together with pyrun, perhaps even by<= /span>
recompiling it to include the application byte code file= s
right in the binary like we do for the stdlib.<= /blockquote>

Well, that would be simply awesome. Looking= forward to it.=C2=A0

PS: you guys should definite= ly advertise this work on the embedded software community.


2015-05= -13 11:29 GMT-03:00 M.-A. Lemburg <mal@egenix.com>:
On 13.05.2015 16:09, Cristiano Cortezia wrote:
> Well I gave it a try, and it seems my assumptions were *somehow* true.=
> Here is what I got when running one of my apps in sin= gle shot mode (load,
> run, terminate):
>
> *default python distribution*
> total time 9.022s
> ENOENT's count 7377
>
> *pyrun*
> total time 8.455s
> ENOENT's count 3064
>
> So, it indeed failed much less to open files, but I guess this didn= 9;t make
> that much difference after all (500ms).

PyRun has the advantage of being able to read the byte code
directly from the binary (using memory mapping). However,
it still needs to run the same startup machinery as Python
itself.

Note that startup time for Python was a lot worse before
Python used the same approach as PyRun to compile in the
parsed sysconfig data.

> Perhaps it is because this app has some external dependencies (22 to b= e
> precise) not bundled on pyrun that had to be scanned by the interprete= r
> anyway. If by any means we could bundle them all the same way, than it=
> could bring a much higher performance gain. But I guess it is not real= ly
> safe-feasible.

It's certainly possible to use the pyrun build system to create<= br> bundles with more packages and tools included.

The one we're shipping has most of the stdlib included,
but leaves all the application code to reside on the
sys.path or in a ZIP archive.

In one of the next releases we'll probably add a tool to bundle
complete applications together with pyrun, perhaps even by
recompiling it to include the application byte code files
right in the binary like we do for the stdlib.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source=C2=A0 (#1, May 13 201= 5)
>>> Python Projects, Coaching and Consulting ...=C2=A0 http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...=C2=A0 =C2=A0 =C2=A0 =C2= =A0http://zope.egenix= .com/
>>> mxODBC, mxDateTime, mxTextTools ...=C2=A0 =C2=A0 =C2=A0 =C2=A0= http://python.egen= ix.com/
________________________________________________________________________ 2015-05-13: Released mxODBC Connect 2.1.3 ...=C2=A0 =C2=A0 =C2=A0http://egenix.com/go75
2015-05-11: Released eGenix PyRun 2.1.0 ...=C2=A0 =C2=A0 =C2=A0 =C2=A0<= a href=3D"http://egenix.com/go74 2015-05-25" target=3D"_blank">http://egenix.com/go74
2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 12 days to go

=C2=A0 =C2=A0eGenix.com Software, Skills and Services GmbH=C2=A0 Pastor-Loe= h-Str.48
=C2=A0 =C2=A0 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lembu= rg
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Registered at Amtsgericht Duesseld= orf: HRB 46611
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0http://www.egenix.com/comp= any/contact/

--001a113d1cec05f2080515f783d7--