Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python.': 0.02; 'scripts': 0.03; 'programmer': 0.03; 'string.': 0.05; 'string': 0.09; 'builtin': 0.09; 'lookup': 0.09; 'pep': 0.09; 'subject:files': 0.09; 'subject:modules': 0.09; 'used.': 0.09; 'python': 0.11; 'mostly': 0.14; "(i'm": 0.16; '04:01': 0.16; 'bit.': 0.16; 'c/c++': 0.16; 'different?': 0.16; 'easier.': 0.16; 'filename:fname piece:signature': 0.16; 'finder': 0.16; 'first:': 0.16; 'in-memory': 0.16; 'machinery.': 0.16; 'object).': 0.16; 'received:192.168.1.20': 0.16; 'situation.': 0.16; 'slight': 0.16; 'url:ch': 0.16; 'url:peps': 0.16; 'pushed': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'module': 0.19; 'implementing': 0.19; 'properly': 0.19; 'stefan': 0.19; '>>>': 0.22; 'code,': 0.22; 'memory': 0.22; 'import': 0.22; 'load': 0.23; 'header:User-Agent:1': 0.23; 'byte': 0.24; 'script.': 0.24; 'simpler': 0.24; 'url:dev': 0.24; 'looks': 0.24; '(or': 0.24; "i've": 0.25; 'source': 0.25; 'script': 0.25; 'skip:" 20': 0.27; 'header:In-Reply-To:1': 0.27; 'fixed': 0.29; 'rest': 0.29; 'skip:p 30': 0.29; "doesn't": 0.30; 'said,': 0.30; "i'm": 0.30; 'work.': 0.31; 'code': 0.31; 'getting': 0.31; 'url:wiki': 0.31; '>>>>': 0.31; 'loading': 0.31; 'though.': 0.31; 'stuff': 0.32; 'supposed': 0.32; 'interface': 0.32; 'regular': 0.32; 'run': 0.32; 'url:python': 0.33; 'running': 0.33; 'totally': 0.33; 'subject:from': 0.34; 'could': 0.34; 'problem': 0.35; "can't": 0.35; 'classes': 0.35; 'anybody': 0.35; 'good.': 0.35; 'but': 0.35; 'add': 0.35; 'there': 0.35; 'version': 0.36; 'c++': 0.36; 'module.': 0.36; 'doing': 0.36; 'method': 0.36; "i'll": 0.36; 'hi,': 0.36; 'similar': 0.36; 'url:org': 0.36; 'integration': 0.37; 'too': 0.37; 'implement': 0.38; 'url:library': 0.38; 'to:addr:python-list': 0.38; 'files': 0.38; 'pm,': 0.38; 'does': 0.39; 'embedded': 0.39; 'to:addr:python.org': 0.39; 'called': 0.40; 'und': 0.40; 'how': 0.40; 'solve': 0.60; 'url:3': 0.61; 'from:charset:utf-8': 0.61; 'you.': 0.62; 'name': 0.63; 'sincerely': 0.63; 'skip:n 10': 0.64; 'map': 0.64; 'finally': 0.65; 'land': 0.65; 'within': 0.65; 'between': 0.67; 'url:4': 0.69; 'evaluate': 0.72; 'export': 0.74; 'years:': 0.84; 'url:php': 0.85; 'yours': 0.88; 'involved.': 0.91; 'same,': 0.91; 'step.': 0.91; 'differences': 0.93 Date: Sat, 17 May 2014 17:28:05 +0200 From: =?UTF-8?B?Um9sYW5kIFBsw7xzcw==?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: python-list@python.org Subject: Re: Loading modules from files through C++ References: <5376ACE7.8030706@rptd.ch> <53775D68.2010009@rptd.ch> <537768FB.7060303@rptd.ch> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2c6vjUCEQtig1cdXKvap0HVFb4mp72Kxk" X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 124 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1400340488 news.xs4all.nl 2919 [2001:888:2000:d::a6]:48707 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:71696 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --2c6vjUCEQtig1cdXKvap0HVFb4mp72Kxk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 05/17/2014 04:01 PM, Stefan Behnel wrote: > Hi, > > please avoid top-posting. > > > Roland Pl=C3=BCss, 17.05.2014 15:49: >> On 05/17/2014 03:26 PM, Stefan Behnel wrote: >>> Roland Pl=C3=BCss, 17.05.2014 15:00: >>>> On 05/17/2014 01:58 PM, Stefan Behnel wrote: >>>>> Roland Pl=C3=BCss, 17.05.2014 02:27: >>>>>> I'm using Python in an embedded situation. In particular I have to= load >>>>>> python scripts through a memory interface so regular python module= >>>>>> loading can not be used. I got working so far a module loader obje= ct >>>>>> I've added using C++ to sys.meta_path . Now I'm totally stuck at t= he >>>>>> finally loading step. >>>>>> >>>>>> I've got this a C++ loader method "load_module(fullname)" which do= es >>>>>> load the requested module script files into a null-terminated stri= ng. I >>>>>> know that "load_module" has to return the module PyObject*. But I = can't >>>>>> get the python source in the c-string into a module PyObject*. >>>>>> [...] >>>>>> Can anybody help how in gods name one is supposed to create a modu= le >>>>>> from an in-memory c-string when called from within load_module (or= >>>>>> anywhere)? >>>>> Looks like you want to implement a SourceLoader: >>>>> >>>>> https://docs.python.org/3.4/library/importlib.html#importlib.abc.So= urceLoader >>>>> >>>>> I recommend implementing this in Python code instead of C code, tho= ugh. >>>>> Much easier. Cython can help with the integration between both. >>>> That doesn't work in 2.x, doesn't it? >>> Is there a reason you have to use Py2? >>> >>> Anyway, PEP 302 predates Py3 by a couple of years: >>> >>> http://legacy.python.org/dev/peps/pep-0302/ >> I'm willing to go to Py3 but only if the solution to the problem is >> simpler than getting it fixed in Py2. So some questions first: >> >> - does this importlib stuff you showed there apply to C++ land (I need= >> to fully drive it from C++ not Python code)? > As I said, implementing this in Python code is much simpler than doing = it > in C/C++ code. Basically, stop where you got the C string and do the re= st > in Python. All your C code has to do is to take a module lookup request= > from your custom Python module Finder and return a byte string with the= > code. Then let your Python code wrap that in a Loader and return it to = the > import machinery. I don't get how this is supposed to work. I'm running it as fully embedded Python. There is no main script. The builtin modules are added as C++ bound classes and a user made main script is loaded but not run directly (I'm hooking into a create object). For this purpose I load the script module using C++ code using PyImport_ImportModule(moduleName). At this time the module loading code has to kick in already (I've added this one by C++ too before). The problem is now that in this call I end up in my C++ loader version where there is no Python script involved. I came to the conclusion that I can solve this only by having the C++ end properly load the module. I could add Python code with PyRun_SimpleString but then I'm down to the same problem as before: how to evaluate code so it is attached to a module or type-class? As I understand it the problem is the same as before just pushed around a bit.= >> - is the C++ land of Py3 similar to Py2 or totally different? > Much the same, just slight differences. As I said, Cython can help with= the > integration and would mostly cover the differences for you. Sounds good. I'll give it a try > > Stefan > > --=20 Yours sincerely Pl=C3=BCss Roland Leader and Head Programmer - Game: Epsylon ( http://www.indiedb.com/games/epsylon ) - Game Engine: Drag[en]gine ( http://www.indiedb.com/engines/dragengine , http://dragengine.rptd.ch/wiki ) - Normal Map Generator: DENormGen ( http://epsylon.rptd.ch/denormgen.php = ) - As well as various Blender export scripts und game tools --2c6vjUCEQtig1cdXKvap0HVFb4mp72Kxk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlN3gA0ACgkQjKGu9ysRWfpCrgCcD6uevS0N1z9+5DzEK2phfPAz pn4AnRxTGgOHQWmRfVOWDhMdAswHpTfh =zgqk -----END PGP SIGNATURE----- --2c6vjUCEQtig1cdXKvap0HVFb4mp72Kxk--