Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <05D7659F-9D0B-466B-833E-8454DCF0F57A@gmail.com>
References: <05D7659F-9D0B-466B-833E-8454DCF0F57A@gmail.com>
Date: Sun, 23 Nov 2014 20:24:36 +1100
Subject: Re: Embedded python 'scripting engine' inside Python app
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.16174.1416734680.18130.python-list@python.org>
Lines: 60
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:81350

On Sun, Nov 23, 2014 at 4:48 PM, Patrick Stinson <patrickkidd@gmail.com> wr=
ote:
> I am writing a python app (using PyQt, but that=E2=80=99s not important h=
ere), and want my users to be able to write their own scripts to automate t=
he app=E2=80=99s functioning using an engine API hat I expose. I have exten=
sive experience doing this in a C++ app with the CPython api, but have no i=
dea how to do this outside of calling exec() from with in Python :)
>
> Ideally their script would compile when the source changes and retain it=
=E2=80=99s state and respond to callbacks from the api object. It appears t=
his won=E2=80=99t work with exec() because the script=E2=80=99s definitions=
 and state disappear as soon as the exec() call is complete, and the script=
 doesn=E2=80=99t seem to be able to access it=E2=80=99s own defined functio=
ns and classes.
>
> Thoughts? Fun stuff!

First off, a cautionary note: Security-wise, this is absolutely
equivalent to your users editing your source code. Be aware that
you're giving them complete control.

What you should be able to do is exec the script in a specific global
dictionary. Here's some example code (Python 3.4):

>>> script =3D """
def init():
    print("Initializing")

def on_some_event(status):
    trigger_some_action("Status is now "+status)
"""
>>> def trigger_some_action(msg):
    print("Action triggered.",msg)
>>> globl =3D {"trigger_some_action":trigger_some_action}
>>> exec(script,globl)
>>> globl["init"]()
Initializing
>>> globl["on_some_event"]("Something happened")
Action triggered. Status is now Something happened

You can provide globals like this, or you can create an importable
module for the scripts to call on. (Or both. Create a module, and
pre-import it automatically.) The script defines functions with
specific names and/or calls your functions to register hooks; you can
reach into the globals to trigger functions.

One way to handle updates to the code would be to exec it in the same
globals dictionary. That has its complexities (for instance, if you
rename a function, the old version will still exist under the old name
unless you explicitly del it), but it can be very convenient.
Alternatively, you could have the new version run in a new dictionary,
but important state can be kept in separate dictionaries. That's how I
manage things with a Pike program - all code gets reloaded cleanly,
but retained state is stored separately in a mutable object that gets
passed around.

There are several ways this sort of thing can be done. It's reasonably
easy, as long as you take a bit of care across the reload boundaries.

ChrisA