Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!eternal-september.org!feeder.eternal-september.org!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'scripts': 0.03; 'ideally': 0.04; 'cpython': 0.05; 'explicitly': 0.05; 'subject:Python': 0.06; '"""': 0.07; '(python': 0.07; '(using': 0.07; 'automate': 0.07; 'definitions': 0.07; 'rename': 0.07; "subject:' ": 0.07; 'augmented': 0.09; 'boundaries.': 0.09; 'classes.': 0.09; 'defines': 0.09; 'exec': 0.09; 'function,': 0.09; 'functioning': 0.09; 'separately': 0.09; 'try:': 0.09; 'api': 0.11; 'cc:addr :python-list': 0.11; 'python': 0.11; 'def': 0.12; 'stored': 0.12; 'changes': 0.15; "'__doc__',": 0.16; '23,': 0.16; 'api,': 0.16; 'dict': 0.16; 'dictionary,': 0.16; 'dictionary.': 0.16; 'exception:': 0.16; 'globals': 0.16; 'it),': 0.16; 'mutable': 0.16; 'pyqt,': 0.16; 'reasonably': 0.16; 'reload': 0.16; 'retained': 0.16; 'skip:{ 40': 0.16; 'stuff!': 0.16; 'symbols': 0.16; 'thoughts?': 0.16; 'wrote:on': 0.16; 'subject:python': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'bit': 0.19; 'app': 0.19; 'module': 0.19; 'patrick': 0.19; 'appears': 0.22; 'example': 0.22; 'import': 0.22; 'email addr:gmail.com>': 0.22; 'separate': 0.22; 'cc:addr:python.org': 0.22; 'instance,': 0.24; 'module,': 0.24; '(or': 0.24; 'cc:2**0': 0.24; 'sort': 0.25; 'source': 0.25; 'script': 0.25; 'equivalent': 0.26; '(for': 0.26; 'defined': 0.27; 'gets': 0.27; 'header:In-Reply-To:1': 0.27; 'idea': 0.28; 'chris': 0.29; 'am,': 0.29; 'skip:g 30': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'reply.': 0.31; "skip:' 10": 0.31; '>>>>': 0.31; 'complete,': 0.31; 'easy,': 0.31; 'object.': 0.31; 'thanks!': 0.32; 'run': 0.32; 'url:python': 0.33; 'could': 0.34; 'except': 0.35; 'skip:s 30': 0.35; 'something': 0.35; 'done.': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'version': 0.36; 'c++': 0.36; 'functions.': 0.36; 'url:listinfo': 0.36; 'doing': 0.36; 'thanks': 0.36; 'url:org': 0.36; 'should': 0.36; 'clear': 0.37; 'skip:o 20': 0.38; 'message-id:@gmail.com': 0.38; 'hat': 0.38; 'nov': 0.38; 'handle': 0.38; 'pm,': 0.38; 'skip:p 20': 0.39; 'users': 0.40; 'url:mail': 0.40; 'how': 0.40; 'even': 0.60; 'around.': 0.60; 'deleting': 0.60; 'skip:t 30': 0.61; 'new': 0.61; "you're": 0.61; 'first': 0.61; 'complete': 0.62; 'reach': 0.63; 'header:Message-Id:1': 0.63; 'name': 0.63; 'soon': 0.63; 'provide': 0.64; 'more': 0.64; 'great': 0.65; 'kept': 0.65; 'to:addr:gmail.com': 0.65; 'effectively': 0.66; 'note:': 0.66; 'secure': 0.71; '2014,': 0.84; 'dict.': 0.84; 'disappear': 0.84; 'it\xe2\x80\x99s': 0.84; 'off,': 0.84; 'pike': 0.84; 'absolutely': 0.87; 'clearing': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=JRoyLf4uRgH8E2QmxfajD9dKN1p44tfAXIOMTaMSijY=; b=SZvlj5zKSKk7NPPbPrlDhTjHt8g4fbzgI2U0QWEwflGono2reQpXlYPjb9BVjfCwGf ecnPODUv2oUXx1mirUigA7Y2RL+cfnTyjmjBtcyUYJsChdnP7ZtCVQqH0u0Oc9N0W8v0 cSsz6dEu6mGm/IazH+rCOv3Z3tmwO4xUBOQEhz8TrCgdesNnxo8OiDbPCXU3vhHOagxD REbpSI+wokrv0KCflkI0HPOfjanZvv/CPy4DxRhXu35Ge36Tqtqxc6z8fUQUlS2jv/7F dn1JzM0z+H3fr946BLXUL+zW59kqsH5ul5ch+xO3Qmf/pPeqHgbkuee6+FbmIksSh1Gf 1x9A== X-Received: by 10.70.89.129 with SMTP id bo1mr23364264pdb.23.1416738529284; Sun, 23 Nov 2014 02:28:49 -0800 (PST) Content-Type: multipart/alternative; boundary="Apple-Mail=_B33F8527-2A95-450D-BC25-50436B5C8D37" Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Embedded python 'scripting engine' inside Python app From: Patrick Stinson In-Reply-To: Date: Sun, 23 Nov 2014 01:28:47 -0900 References: <05D7659F-9D0B-466B-833E-8454DCF0F57A@gmail.com> To: Chris Angelico X-Mailer: Apple Mail (2.1993) X-Mailman-Approved-At: Mon, 24 Nov 2014 09:16:01 +0100 Cc: "python-list@python.org" X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 266 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1416816962 news.xs4all.nl 2870 [2001:888:2000:d::a6]:57328 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:81443 --Apple-Mail=_B33F8527-2A95-450D-BC25-50436B5C8D37 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thanks for your great reply. I even augmented the reloading with the = same dict by clearing all of the non-standard symbols from the dict. = This effectively resets the dict: # try to clear out the module by deleting all global refs = =20 d =3D self.module.__dict__ for k in dict(d).keys(): if not k in ['__spec__', '__name__', '__loader__', = '__package__', '__doc__', '__builtins__']: del d[k] self.module.__dict__['sendMessage'] =3D self.sendMessage try: exec(self.source, self.module.__dict__) except Exception: import traceback traceback.print_exc(file=3Dsys.stdout) Is there a better and more secure way to do the python-within-python in = order allow users to automate your app? Thanks! > On Nov 23, 2014, at 12:24 AM, Chris Angelico wrote: >=20 > On Sun, Nov 23, 2014 at 4:48 PM, Patrick Stinson = wrote: >> I am writing a python app (using PyQt, but that=E2=80=99s not = important here), and want my users to be able to write their own scripts = to automate the app=E2=80=99s functioning using an engine API hat I = expose. I have extensive experience doing this in a C++ app with the = CPython api, but have no idea how to do this outside of calling exec() = from with in Python :) >>=20 >> Ideally their script would compile when the source changes and retain = it=E2=80=99s state and respond to callbacks from the api object. It = appears this won=E2=80=99t work with exec() because the script=E2=80=99s = definitions and state disappear as soon as the exec() call is complete, = and the script doesn=E2=80=99t seem to be able to access it=E2=80=99s = own defined functions and classes. >>=20 >> Thoughts? Fun stuff! >=20 > First off, a cautionary note: Security-wise, this is absolutely > equivalent to your users editing your source code. Be aware that > you're giving them complete control. >=20 > What you should be able to do is exec the script in a specific global > dictionary. Here's some example code (Python 3.4): >=20 >>>> script =3D """ > def init(): > print("Initializing") >=20 > def on_some_event(status): > trigger_some_action("Status is now "+status) > """ >>>> def trigger_some_action(msg): > print("Action triggered.",msg) >>>> globl =3D {"trigger_some_action":trigger_some_action} >>>> exec(script,globl) >>>> globl["init"]() > Initializing >>>> globl["on_some_event"]("Something happened") > Action triggered. Status is now Something happened >=20 > You can provide globals like this, or you can create an importable > module for the scripts to call on. (Or both. Create a module, and > pre-import it automatically.) The script defines functions with > specific names and/or calls your functions to register hooks; you can > reach into the globals to trigger functions. >=20 > One way to handle updates to the code would be to exec it in the same > globals dictionary. That has its complexities (for instance, if you > rename a function, the old version will still exist under the old name > unless you explicitly del it), but it can be very convenient. > Alternatively, you could have the new version run in a new dictionary, > but important state can be kept in separate dictionaries. That's how I > manage things with a Pike program - all code gets reloaded cleanly, > but retained state is stored separately in a mutable object that gets > passed around. >=20 > There are several ways this sort of thing can be done. It's reasonably > easy, as long as you take a bit of care across the reload boundaries. >=20 > ChrisA > --=20 > https://mail.python.org/mailman/listinfo/python-list --Apple-Mail=_B33F8527-2A95-450D-BC25-50436B5C8D37 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Thanks for your great reply. I even augmented = the reloading with the same dict by clearing all of the non-standard = symbols from the dict. This effectively resets the dict:

      =   # try to clear out the module by deleting all global = refs                   =                     =                  
      =   d =3D self.module.__dict__
        for k in dict(d).keys():
      =       if not k in ['__spec__', '__name__', '__loader__', = '__package__', '__doc__', '__builtins__']:
      =           del = d[k]
  =       self.module.__dict__['sendMessage'] =3D self.sendMessage
        try:
            exec(self.source, = self.module.__dict__)
      =   except Exception:
            import traceback
            = traceback.print_exc(file=3Dsys.stdout)


Is = there a better and more secure way to do the python-within-python in = order allow users to automate your app?

Thanks!


On Nov 23, 2014, at 12:24 AM, Chris Angelico = <rosuav@gmail.com> wrote:

On Sun, Nov 23, 2014 = at 4:48 PM, Patrick Stinson <patrickkidd@gmail.com> wrote:
I am writing a python app (using PyQt, but = that=E2=80=99s not important here), and want my users to be able to = write their own scripts to automate the app=E2=80=99s functioning using = an engine API hat I expose. I have extensive experience doing this in a = C++ app with the CPython api, but have no idea how to do this outside of = calling exec() from with in Python :)

Ideally= their script would compile when the source changes and retain it=E2=80=99= s state and respond to callbacks from the api object. It appears this = won=E2=80=99t work with exec() because the script=E2=80=99s definitions = and state disappear as soon as the exec() call is complete, and the = script doesn=E2=80=99t seem to be able to access it=E2=80=99s own = defined functions and classes.

Thoughts? = Fun stuff!

First off, a = cautionary note: Security-wise, this is absolutely
equivalent to your users editing your source code. Be aware = that
you're giving them complete control.

What you should be able to do is exec the script in a = specific global
dictionary. Here's some example code = (Python 3.4):

script =3D """
def init():
   print("Initializing")

def on_some_event(status):
=    trigger_some_action("Status is now "+status)
"""
def trigger_some_action(msg):
=    print("Action triggered.",msg)
globl =3D = {"trigger_some_action":trigger_some_action}
exec(script,globl)
globl["init"]()
Initializing
globl["on_some_event"]("Something happened")
Action triggered. = Status is now Something happened

You can = provide globals like this, or you can create an importable
module for the scripts to call on. (Or both. Create a module, = and
pre-import it automatically.) The script defines = functions with
specific names and/or calls your functions = to register hooks; you can
reach into the globals to = trigger functions.

One way to handle = updates to the code would be to exec it in the same
globals = dictionary. That has its complexities (for instance, if you
rename a function, the old version will still exist under the = old name
unless you explicitly del it), but it can be very = convenient.
Alternatively, you could have the new version = run in a new dictionary,
but important state can be kept = in separate dictionaries. That's how I
manage things with = a Pike program - all code gets reloaded cleanly,
but = retained state is stored separately in a mutable object that gets
passed around.

There are several = ways this sort of thing can be done. It's reasonably
easy, = as long as you take a bit of care across the reload boundaries.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

= --Apple-Mail=_B33F8527-2A95-450D-BC25-50436B5C8D37--