Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #31628 > unrolled thread

use of exec()

Back to article view | Back to comp.lang.python

Contents

  use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-18 04:41 -0700
    Re: use of exec() Chris Angelico <rosuav@gmail.com> - 2012-10-18 22:49 +1100
      Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-18 07:07 -0700
        Re: use of exec() Chris Angelico <rosuav@gmail.com> - 2012-10-19 01:29 +1100
          Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-18 08:00 -0700
            Re: use of exec() Chris Angelico <rosuav@gmail.com> - 2012-10-19 02:16 +1100
              Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-19 16:43 -0700
                Re: use of exec() Chris Angelico <rosuav@gmail.com> - 2012-10-20 13:00 +1100
                  Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-20 03:41 -0700
                  Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-20 03:41 -0700
              Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-19 16:43 -0700
          Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-18 08:00 -0700
      Re: use of exec() lars van gemerden <lars@rational-it.com> - 2012-10-18 07:07 -0700

#31628 — use of exec()

I am trying to implement a way to let users give a limited possibility to define functions in text, that wille be stored and executed at a later time. I use exec() to transform the text to a function. The code is as follows:

class code(str):
    def __call__(self, *args):
        try:
            return self._func_(*args)
        except AttributeError:
            self._func_ = self._creat_func_()
            return self._func_(*args)
    def __getstate__(self):
        return {}

class _lambdacode(code):
    def _creat_func_(self):
        return eval("lambda %s: %s" % (", ".join(type(self).args), self),{},{})
    
class _functioncode(code):
    def _creat_func_(self):
        exec("def function(%s):\n\t%s" % (", ".join(type(self).args), 
                                          "\n\t".join(self.split('\n'))))
        return function

def lambdatype(*argnames):
    return type('lambdacode', (_lambdacode,),{'args': argnames}) 

def functiontype(*argnames):
    return type('functioncode', (_functioncode,),{'args': argnames})
    
if __name__ == '__main__':
    f = lambdatype('a', 'b')('a ** b')
    print f
    print f(3, 4)
    print f(4, 3)
    
    g = functiontype('a', 'b')('a, b = a/b, a*b\nreturn a ** b')
    print g
    print g(3.0, 4.0) 
    print g(4.0, 3.0)

This code works. But if I replace _functioncode with:

class _functioncode(code):
    def _creat_func_(self):
        exec("def function(%s):\n\t%s" % (", ".join(type(self).args), 
                                          "\n\t".join(self.split('\n'))),{})
        return function

or

class _functioncode(code):
    def _creat_func_(self):
        exec("def function(%s):\n\t%s" % (", ".join(type(self).args), 
                                          "\n\t".join(self.split('\n'))),{},{})
        return function

to restrict access to global variables, similar to the lambda version, i get the error:

Traceback (most recent call last):
  File "D:\Documents\Code\Eclipse\workspace\FlowWare_1\toolshed\tests\mini_test.py", line 41, in <module>
    print g(3.0, 4.0)
  File "D:\Documents\Code\Eclipse\workspace\FlowWare_1\toolshed\tests\mini_test.py", line 13, in __call__
    self._func_ = self._creat_func_()
  File "D:\Documents\Code\Eclipse\workspace\FlowWare_1\toolshed\tests\mini_test.py", line 25, in _creat_func_
    return function
NameError: name 'function' is not defined

which seems an odd error, but i think some global variable is necessary for this to work (if i put in globals() instead of {}, it works).

My question is which variable or if that is not the problem, what is and how can i restrict access the user code has.

Cheers, Lars

[toc] | [next] | [standalone]

#31629

On Thu, Oct 18, 2012 at 10:41 PM, lars van gemerden
<lars@rational-it.com> wrote:
> NameError: name 'function' is not defined
>
> which seems an odd error, but i think some global variable is necessary for this to work (if i put in globals() instead of {}, it works).

The def statement simply adds a name to the current namespace. This
should work (untested):

class _functioncode(code):
    def _creat_func_(self):
        ns={}
        exec("def function(%s):\n\t%s" % (", ".join(type(self).args),
                                          "\n\t".join(self.split('\n'))),ns,ns)
        return ns.function

But it's going to be eternally plagued by security issues. You may
want, instead, to look at literal_eval from the ast module; but that
won't work if you need anything other than, as the name suggests,
literals.

ChrisA

[toc] | [prev] | [next] | [standalone]

#31640

On Thursday, October 18, 2012 1:49:35 PM UTC+2, Chris Angelico wrote:
> On Thu, Oct 18, 2012 at 10:41 PM, lars van gemerden
> 
> <lars@rational-it.com> wrote:
> 
> > NameError: name 'function' is not defined
> 
> >
> 
> > which seems an odd error, but i think some global variable is necessary for this to work (if i put in globals() instead of {}, it works).
> 
> 
> 
> The def statement simply adds a name to the current namespace. This
> 
> should work (untested):
> 
> 
> 
> class _functioncode(code):
> 
>     def _creat_func_(self):
> 
>         ns={}
> 
>         exec("def function(%s):\n\t%s" % (", ".join(type(self).args),
> 
>                                           "\n\t".join(self.split('\n'))),ns,ns)
> 
>         return ns.function
> 
> 
> 
> But it's going to be eternally plagued by security issues. You may
> 
> want, instead, to look at literal_eval from the ast module; but that
> 
> won't work if you need anything other than, as the name suggests,
> 
> literals.
> 
> 
> 
> ChrisA

Thanks, Chris,

That works like a charm (after replacig "return ns.function" with "return ns['function']" ;-) ).

About the security, i noticed you can still import and use modules within the exec'ed code. Is there a way to prevent this or otherwise make this approach more secure.

I should say that the users that will be able to make custom functions, are not end-users, but authenticated designers, however i would like to close a backdoor to the whole framework.

Cheers, Lars 

[toc] | [prev] | [next] | [standalone]

#31643

On Fri, Oct 19, 2012 at 1:07 AM, lars van gemerden <lars@rational-it.com> wrote:
> Thanks, Chris,
>
> That works like a charm (after replacig "return ns.function" with "return ns['function']" ;-) ).

Err, yes, I forget sometimes that Python doesn't do that. JavaScript
and Pike both let you (though Pike uses -> instead of . for that
operator). Yes, Python has real methods on dictionary objects :)

> About the security, i noticed you can still import and use modules within the exec'ed code. Is there a way to prevent this or otherwise make this approach more secure.

Basically no, there's no real way to make it secure. Without
eliminating exec/eval, destroying insecurity is the hopeless work of a
wasted life, as the oracle said to Alice.

> I should say that the users that will be able to make custom functions, are not end-users, but authenticated designers, however i would like to close a backdoor to the whole framework.

You have to decide one thing: Will you permit them to execute
untrusted code on your system? If so, go ahead (and just warn them
that things like import shouldn't be done, as they can cause other
messes). I run a server that I build with the help of another guy (I
do the code, he does the bulk of the content - descriptions and
stuff), and I'm happy to trust him to not be malicious, so the purpose
of "embedded code in loci" is to make it easier to write tiny bits of
code, without any security requirement. But if you need security,
don't use eval. AT ALL.

There may be a brand new service coming along, though. The ast module
I think is getting a new evaluator that allows a little more
functionality than literal_eval, while still not permitting most
things. But you then have the question of performance, since you
effectively interpret the code at a high level.

ChrisA

[toc] | [prev] | [next] | [standalone]

#31647

On Thursday, October 18, 2012 4:29:45 PM UTC+2, Chris Angelico wrote:
> On Fri, Oct 19, 2012 at 1:07 AM, lars van gemerden <lars@rational-it.com> wrote:
> 
> > Thanks, Chris,
> 
> >
> 
> > That works like a charm (after replacig "return ns.function" with "return ns['function']" ;-) ).
> 
> 
> 
> Err, yes, I forget sometimes that Python doesn't do that. JavaScript
> 
> and Pike both let you (though Pike uses -> instead of . for that
> 
> operator). Yes, Python has real methods on dictionary objects :)
> 
> 
> 
> > About the security, i noticed you can still import and use modules within the exec'ed code. Is there a way to prevent this or otherwise make this approach more secure.
> 
> 
> 
> Basically no, there's no real way to make it secure. Without
> 
> eliminating exec/eval, destroying insecurity is the hopeless work of a
> 
> wasted life, as the oracle said to Alice.
> 
> 
> 
> > I should say that the users that will be able to make custom functions, are not end-users, but authenticated designers, however i would like to close a backdoor to the whole framework.
> 
> 
> 
> You have to decide one thing: Will you permit them to execute
> 
> untrusted code on your system? If so, go ahead (and just warn them
> 
> that things like import shouldn't be done, as they can cause other
> 
> messes). I run a server that I build with the help of another guy (I
> 
> do the code, he does the bulk of the content - descriptions and
> 
> stuff), and I'm happy to trust him to not be malicious, so the purpose
> 
> of "embedded code in loci" is to make it easier to write tiny bits of
> 
> code, without any security requirement. But if you need security,
> 
> don't use eval. AT ALL.
> 
> 
> 
> There may be a brand new service coming along, though. The ast module
> 
> I think is getting a new evaluator that allows a little more
> 
> functionality than literal_eval, while still not permitting most
> 
> things. But you then have the question of performance, since you
> 
> effectively interpret the code at a high level.
> 
> 
> 
> ChrisA

I get your point, since in this case having the custom code option makes the system a whole lot less complex and flexible, i will leave the option in. The future customer will be informed that they should handle the security around the designers as if they were programmers. Aditionally i will probably add some screening for unwanted keywords (like 'import') and securely log any new/changed custom code including the designer account (must do that for other actions anyway).

Thanks again, Lars

[toc] | [prev] | [next] | [standalone]

#31652

On Fri, Oct 19, 2012 at 2:00 AM, lars van gemerden <lars@rational-it.com> wrote:
> I get your point, since in this case having the custom code option makes the system a whole lot less complex and flexible, i will leave the option in. The future customer will be informed that they should handle the security around the designers as if they were programmers. Aditionally i will probably add some screening for unwanted keywords (like 'import') and securely log any new/changed custom code including the designer account (must do that for other actions anyway).

That sounds like a reasonable implementation of Layer Eight security.
As long as everyone understands that this code can do ANYTHING, you'll
be fine.

You may want to add some other programmatic checks, though; for
instance, a watchdog timer in case the code gets stuck in an infinite
loop, or a memory usage limit, or somesuch. Since you're no longer
worrying about security, this sort of thing will be fairly easy, and
will be just to help catch common errors.

ChrisA

[toc] | [prev] | [next] | [standalone]

#31783

On Thursday, October 18, 2012 5:16:50 PM UTC+2, Chris Angelico wrote:
> On Fri, Oct 19, 2012 at 2:00 AM, lars van gemerden <lars@rational-it.com> wrote:
> 
> > I get your point, since in this case having the custom code option makes the system a whole lot less complex and flexible, i will leave the option in. The future customer will be informed that they should handle the security around the designers as if they were programmers. Aditionally i will probably add some screening for unwanted keywords (like 'import') and securely log any new/changed custom code including the designer account (must do that for other actions anyway).
> 
> 
> 
> That sounds like a reasonable implementation of Layer Eight security.
> 
> As long as everyone understands that this code can do ANYTHING, you'll
> 
> be fine.
> 
> 
> 
> You may want to add some other programmatic checks, though; for
> 
> instance, a watchdog timer in case the code gets stuck in an infinite
> 
> loop, or a memory usage limit, or somesuch. Since you're no longer
> 
> worrying about security, this sort of thing will be fairly easy, and
> 
> will be just to help catch common errors.
> 
> 
> 
> ChrisA

Do you have any ideas about to what extend the "lambda" version of the code (custom code is only the 'body' of the lambda function) has the same issues?

Cheers, Lars 

[toc] | [prev] | [next] | [standalone]

#31789

On Sat, Oct 20, 2012 at 10:43 AM, lars van gemerden
<lars@rational-it.com> wrote:
> Do you have any ideas about to what extend the "lambda" version of the code (custom code is only the 'body' of the lambda function) has the same issues?

The lambda version definitely has the same issues. You can do pretty
much anything with a single expression.

ChrisA

[toc] | [prev] | [next] | [standalone]

#31806

On Saturday, October 20, 2012 4:00:55 AM UTC+2, Chris Angelico wrote:
> On Sat, Oct 20, 2012 at 10:43 AM, lars van gemerden
> 
> <lars@rational-it.com> wrote:
> 
> > Do you have any ideas about to what extend the "lambda" version of the code (custom code is only the 'body' of the lambda function) has the same issues?
> 
> 
> 
> The lambda version definitely has the same issues. You can do pretty
> 
> much anything with a single expression.
> 
> 
> 
> ChrisA

Thanks a lot Chris, I will return later to the watchdog/memory limit ideas (no idea how to do that yet).

[toc] | [prev] | [next] | [standalone]

#31807

On Saturday, October 20, 2012 4:00:55 AM UTC+2, Chris Angelico wrote:
> On Sat, Oct 20, 2012 at 10:43 AM, lars van gemerden
> 
> <lars@rational-it.com> wrote:
> 
> > Do you have any ideas about to what extend the "lambda" version of the code (custom code is only the 'body' of the lambda function) has the same issues?
> 
> 
> 
> The lambda version definitely has the same issues. You can do pretty
> 
> much anything with a single expression.
> 
> 
> 
> ChrisA

Thanks a lot Chris, I will return later to the watchdog/memory limit ideas (no idea how to do that yet).

[toc] | [prev] | [next] | [standalone]

#31784

On Thursday, October 18, 2012 5:16:50 PM UTC+2, Chris Angelico wrote:
> On Fri, Oct 19, 2012 at 2:00 AM, lars van gemerden <lars@rational-it.com> wrote:
> 
> > I get your point, since in this case having the custom code option makes the system a whole lot less complex and flexible, i will leave the option in. The future customer will be informed that they should handle the security around the designers as if they were programmers. Aditionally i will probably add some screening for unwanted keywords (like 'import') and securely log any new/changed custom code including the designer account (must do that for other actions anyway).
> 
> 
> 
> That sounds like a reasonable implementation of Layer Eight security.
> 
> As long as everyone understands that this code can do ANYTHING, you'll
> 
> be fine.
> 
> 
> 
> You may want to add some other programmatic checks, though; for
> 
> instance, a watchdog timer in case the code gets stuck in an infinite
> 
> loop, or a memory usage limit, or somesuch. Since you're no longer
> 
> worrying about security, this sort of thing will be fairly easy, and
> 
> will be just to help catch common errors.
> 
> 
> 
> ChrisA

Do you have any ideas about to what extend the "lambda" version of the code (custom code is only the 'body' of the lambda function) has the same issues?

Cheers, Lars 

[toc] | [prev] | [next] | [standalone]

#31648

On Thursday, October 18, 2012 4:29:45 PM UTC+2, Chris Angelico wrote:
> On Fri, Oct 19, 2012 at 1:07 AM, lars van gemerden <lars@rational-it.com> wrote:
> 
> > Thanks, Chris,
> 
> >
> 
> > That works like a charm (after replacig "return ns.function" with "return ns['function']" ;-) ).
> 
> 
> 
> Err, yes, I forget sometimes that Python doesn't do that. JavaScript
> 
> and Pike both let you (though Pike uses -> instead of . for that
> 
> operator). Yes, Python has real methods on dictionary objects :)
> 
> 
> 
> > About the security, i noticed you can still import and use modules within the exec'ed code. Is there a way to prevent this or otherwise make this approach more secure.
> 
> 
> 
> Basically no, there's no real way to make it secure. Without
> 
> eliminating exec/eval, destroying insecurity is the hopeless work of a
> 
> wasted life, as the oracle said to Alice.
> 
> 
> 
> > I should say that the users that will be able to make custom functions, are not end-users, but authenticated designers, however i would like to close a backdoor to the whole framework.
> 
> 
> 
> You have to decide one thing: Will you permit them to execute
> 
> untrusted code on your system? If so, go ahead (and just warn them
> 
> that things like import shouldn't be done, as they can cause other
> 
> messes). I run a server that I build with the help of another guy (I
> 
> do the code, he does the bulk of the content - descriptions and
> 
> stuff), and I'm happy to trust him to not be malicious, so the purpose
> 
> of "embedded code in loci" is to make it easier to write tiny bits of
> 
> code, without any security requirement. But if you need security,
> 
> don't use eval. AT ALL.
> 
> 
> 
> There may be a brand new service coming along, though. The ast module
> 
> I think is getting a new evaluator that allows a little more
> 
> functionality than literal_eval, while still not permitting most
> 
> things. But you then have the question of performance, since you
> 
> effectively interpret the code at a high level.
> 
> 
> 
> ChrisA

I get your point, since in this case having the custom code option makes the system a whole lot less complex and flexible, i will leave the option in. The future customer will be informed that they should handle the security around the designers as if they were programmers. Aditionally i will probably add some screening for unwanted keywords (like 'import') and securely log any new/changed custom code including the designer account (must do that for other actions anyway).

Thanks again, Lars

[toc] | [prev] | [next] | [standalone]

#31641

On Thursday, October 18, 2012 1:49:35 PM UTC+2, Chris Angelico wrote:
> On Thu, Oct 18, 2012 at 10:41 PM, lars van gemerden
> 
> <lars@rational-it.com> wrote:
> 
> > NameError: name 'function' is not defined
> 
> >
> 
> > which seems an odd error, but i think some global variable is necessary for this to work (if i put in globals() instead of {}, it works).
> 
> 
> 
> The def statement simply adds a name to the current namespace. This
> 
> should work (untested):
> 
> 
> 
> class _functioncode(code):
> 
>     def _creat_func_(self):
> 
>         ns={}
> 
>         exec("def function(%s):\n\t%s" % (", ".join(type(self).args),
> 
>                                           "\n\t".join(self.split('\n'))),ns,ns)
> 
>         return ns.function
> 
> 
> 
> But it's going to be eternally plagued by security issues. You may
> 
> want, instead, to look at literal_eval from the ast module; but that
> 
> won't work if you need anything other than, as the name suggests,
> 
> literals.
> 
> 
> 
> ChrisA

Thanks, Chris,

That works like a charm (after replacig "return ns.function" with "return ns['function']" ;-) ).

About the security, i noticed you can still import and use modules within the exec'ed code. Is there a way to prevent this or otherwise make this approach more secure.

I should say that the users that will be able to make custom functions, are not end-users, but authenticated designers, however i would like to close a backdoor to the whole framework.

Cheers, Lars 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web