Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #91220 > unrolled thread

Creating a reliable sandboxed Python environment

Back to article view | Back to comp.lang.python

Contents

  Creating a reliable sandboxed Python environment davidfstr@gmail.com - 2015-05-25 19:24 -0700
    Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-26 12:44 +1000
    Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-25 23:17 -0700
    Re: Creating a reliable sandboxed Python environment Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-26 17:10 +1000
      Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-26 09:53 +0200
      Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-26 10:02 +0200
    Re: Creating a reliable sandboxed Python environment Ned Batchelder <ned@nedbatchelder.com> - 2015-05-26 03:21 -0700
    Re: Creating a reliable sandboxed Python environment marco.nawijn@colosso.nl - 2015-05-26 05:01 -0700
    Re: Creating a reliable sandboxed Python environment davidfstr@gmail.com - 2015-05-28 09:34 -0700
      Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-30 20:13 -0700
    Re: Creating a reliable sandboxed Python environment Stefan Behnel <stefan_ml@behnel.de> - 2015-05-28 20:41 +0200
    Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-29 04:51 +1000
      Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-29 11:30 -0700
        Re: Creating a reliable sandboxed Python environment Marko Rauhamaa <marko@pacujo.net> - 2015-05-29 22:12 +0300
          Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-29 13:15 -0700
    Re: Creating a reliable sandboxed Python environment Stefan Behnel <stefan_ml@behnel.de> - 2015-05-29 08:18 +0200
    Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-29 17:41 +1000
      Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-29 11:33 -0700
        Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-30 08:49 +1000
          Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-29 18:28 -0700
            Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-30 12:42 +1000
              Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-29 21:48 -0700
                Re: Creating a reliable sandboxed Python environment Steven D'Aprano <steve@pearwood.info> - 2015-05-30 19:00 +1000
                  Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-30 13:24 +0200
                    Re: Creating a reliable sandboxed Python environment Steven D'Aprano <steve@pearwood.info> - 2015-05-31 09:52 +1000
                      Re: Creating a reliable sandboxed Python environment Modulok <modulok@gmail.com> - 2015-05-30 19:08 -0600
                      Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-31 08:14 +0200
                  Re: Creating a reliable sandboxed Python environment Stefan Behnel <stefan_ml@behnel.de> - 2015-05-30 20:42 +0200
                  Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-30 13:00 -0700
                    Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-31 08:20 +1000
                      Re: Creating a reliable sandboxed Python environment Paul Rubin <no.email@nospam.invalid> - 2015-05-30 15:36 -0700
                  Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-30 22:54 +0200
          Re: Creating a reliable sandboxed Python environment BartC <bc@freeuk.com> - 2015-05-30 13:06 +0100
            Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-30 22:37 +1000
    Re: Creating a reliable sandboxed Python environment Stefan Behnel <stefan_ml@behnel.de> - 2015-05-29 11:23 +0200
    Re: Creating a reliable sandboxed Python environment Chris Angelico <rosuav@gmail.com> - 2015-05-29 19:38 +1000
    Re: Creating a reliable sandboxed Python environment Laura Creighton <lac@openend.se> - 2015-05-29 11:56 +0200

Page 1 of 2  [1] 2  Next page →

#91220 — Creating a reliable sandboxed Python environment

I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.

Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.

For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
* reading from stdin & writing to stdout;
* reading from files, within a set of whitelisted directories;
* pure Python computation.

In particular all other operations available through system calls are banned. This includes, but is not limited to:
* writing to files;
* manipulating network sockets;
* communicating with other processes.

I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.

It might be necessary to isolate the Python process at the operating system level.
* A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
* No obvious way to block socket creation. Again this would be just a blacklist.
* No obvious way to detect unapproved system calls and block them.

In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.

Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.

- David

[toc] | [next] | [standalone]

#91222

On Tue, May 26, 2015 at 12:24 PM,  <davidfstr@gmail.com> wrote:
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
>
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
>
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
>
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.

Yes, I had a project along similar lines to yours, a few years back.
We wanted to let our end users customize our service using a Python
script. Our conclusions were:

1) As you say, it is fundamentally not possible to make this work at
the Python level.
2) It's extremely difficult to do at any other level, too.
3) Python is a great language, despite my then-boss's dislike of it.
4) Lua isn't as great a language, but it's much easier to sandbox.
5) Unicode is important, even if my then-boss took a lot of convincing
on that one. (Was a big point in Python's favour, and against Lua.)
6) Efficient transfer of complex structured data across a process
boundary is difficult.
7) Letting end users script your system safely is a fundamentally hard problem.

We ended up abandoning Python altogether and using ECMAScript (with
Google's V8 interpreter) as our scripting language, and even then, we
had to do all sorts of things to make it safe. (And I wouldn't bet my
life on it being safe even now. Not even sure I'd bet my data or
uptime on it being safe, either.)

My recommendation to you: If you absolutely have to run untrusted
Python code, don't concern yourself with *anything* that the Python
code can and can't do. You'll end up making gross and ugly hacks that
stop people from doing legitimate things, in an attempt to prevent
abuses. Instead, *just* guard yourself at the OS level - a chroot jail
to protect what matters, iptables rules to prevent anything going to
the outside world, run as a non-significant user with minimal
permissions, ulimit everything so they can't hurt you. Whatever it
takes, make it so that you could protect C code, because trust me,
it'll be less headaches than trying to sandbox anything at the Python
level. Or, worse, you won't get headaches, you'll just have a flawed
security model that eventually gets exploited.

There are a couple of alternatives. You could go for a really extreme
protection system and actually spin up a virtual machine, where
they're welcome to do whatever they like, and it'll run inside X
amount of memory and Y amount of CPU. Pretty costly (the overhead of a
full OS for every client), but it'll work. Or you could go to the
other extreme, and instead of actually permitting arbitrary Python
code, you instead allow a "Python-like syntax" wherein people can
manipulate the input. You'd need to then create some special hacks to
allow file I/O, so this probably wouldn't work for your scenario, but
imagine writing a sed-like program that accepts Python code. You could
do something like this:

for line in input:
    print(evaluate_user_code(line), file=output)

where evaluate_user_code() is a protected evaluator, like
ast.literal_eval() but additionally allowing access to one name
"line", which obviously would be the line in question.

But for your case, I think that'd require too many hacks to be useful.

ChrisA

[toc] | [prev] | [next] | [standalone]

#91233

davidfstr@gmail.com writes:
> Has anyone on this list attempted to sandbox Python programs in a
> serious fashion? I'd be interested to hear your approach.

There is something like that for C++ and it is quite complicated:

https://github.com/Eelis/geordi

I expect that for Python you'd have to do most of the same stuff.

[toc] | [prev] | [next] | [standalone]

#91235

On Tuesday 26 May 2015 12:24, davidfstr@gmail.com wrote:

> I am writing a web service that accepts Python programs as input, runs the
> provided program with some profiling hooks, and returns various
> information about the program's runtime behavior. To do this in a safe
> manner, I need to be able to create a sandbox that restricts what the
> submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet
> involves selectively blacklisting functionality that gives access to
> system resources, such as trying to hide the "open" builtin to restrict
> access to file I/O. All such approaches are doomed to fail because you can
> always find a way around a blacklist.

It's not so much that you can find your way around a blacklist, but that a 
blacklist only bans things which you have thought of. Perhaps the attacker 
has thought of something else.

Ideally, a sandbox will whitelist functions which you know are safe, with a 
"default deny" policy. That requires building your own parser which only 
allows code that passes your whitelist. Even then, the problem is that 
perhaps there is an attack vector you didn't think of: something you thought 
was safe, actually is not.

Have you read Tav's admirable but failed attempt to sandbox file IO?

http://tav.espians.com/a-challenge-to-break-python-security.html

http://tav.espians.com/paving-the-way-to-securing-the-python-
interpreter.html

http://tav.espians.com/update-on-securing-the-python-interpreter.html

Also google for "Capabilities Python" or CapPython.

My sense is that the only way to safely sandbox Python is to create your own 
Python implementation designed with security in mind. You can't get there 
starting from CPython. Maybe Jython?


> For my particular sandbox, I wish to allow *only* the following kinds of
> actions (in a whitelist): * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.

Pure Python computation can be used to DOS your machine, e.g. 
(100**100)**100 will, I think, do it. (I'm not about to try it.)



-- 
Steve

[toc] | [prev] | [next] | [standalone]

#91238

In a message of Tue, 26 May 2015 17:10:30 +1000, "Steven D'Aprano" writes:
>My sense is that the only way to safely sandbox Python is to create your own 
>Python implementation designed with security in mind. You can't get there 
>starting from CPython. Maybe Jython?

You get there starting with PyPy.
see: http://pypy.readthedocs.org/en/latest/sandbox.html

Note: this is not very heavily used.  You may find bugs we don't
know about yet.

Laura

[toc] | [prev] | [next] | [standalone]

#91240

In a message of Tue, 26 May 2015 09:53:56 +0200, Laura Creighton writes:
>In a message of Tue, 26 May 2015 17:10:30 +1000, "Steven D'Aprano" writes:
>>My sense is that the only way to safely sandbox Python is to create your own 
>>Python implementation designed with security in mind. You can't get there 
>>starting from CPython. Maybe Jython?
>
>You get there starting with PyPy.
>see: http://pypy.readthedocs.org/en/latest/sandbox.html
>
>Note: this is not very heavily used.  You may find bugs we don't
>know about yet.
>
>Laura

Also, the place to discuss this is pypy-dev@python.org or the #pypy
channel on freenode.

Laura

[toc] | [prev] | [next] | [standalone]

#91241

On Monday, May 25, 2015 at 10:24:32 PM UTC-4, davi...@gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David

When we needed this at edX, we wrote CodeJail (https://github.com/edx/codejail).
It's a wrapper around AppArmor to provide OS-level protection of code
execution in subprocesses.  It has Python-specific features, but because it
is based on AppArmor, can sandbox any process, so long as it's configured
properly.

--Ned.

[toc] | [prev] | [next] | [standalone]

#91242

On Tuesday, May 26, 2015 at 4:24:32 AM UTC+2, davi...@gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David

What about launching the Python process in a Docker container?
Spinning up a new container is pretty quick and it might provide
you with enough isolation. Probably not a perfect solution, but
I do believe that it would be easier than trying to sandbox Python
itself.

Marco

[toc] | [prev] | [next] | [standalone]

#91377

Thanks for the responses folks. I will briefly summarize them:

> As you say, it is fundamentally not possible to make this work at 
the Python level.

This is pretty effectively demonstrated by "Tav's admirable but failed attempt to sandbox file IO":
* http://tav.espians.com/a-challenge-to-break-python-security.html

Wow there are some impressive ways to confuse the system. I particularly like overriding str's equality function to defeat mode checking code when opening files.

> When we needed this at edX, we wrote CodeJail (https://github.com/edx/codejail). 
It's a wrapper around AppArmor to provide OS-level protection of code 
execution in subprocesses.  It has Python-specific features, but because it 
is based on AppArmor, can sandbox any process, so long as it's configured 
properly. 

This looks promising. I will take a closer look.

> What about launching the Python process in a Docker container?

This may work in combination with other techniques. Certainly faster than spinning up a new VM or snapshot-restoring a fixed VM on a repeated basis. Would need to see whether CPU, Memory, and Disk usage could be constrained at the level of a container.

- David


On Monday, May 25, 2015 at 7:24:32 PM UTC-7, davi...@gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David

[toc] | [prev] | [next] | [standalone]

#91558

davidfstr@gmail.com writes:
> Thanks for the responses folks. I will briefly summarize them:...

I do think you should look at Geordi (the C++ IRC bot) that I linked.
It seems to have changed its implementation to use Docker, but either
way, lots of the the stuff it did was language independent.

[toc] | [prev] | [next] | [standalone]

#91381

davidfstr@gmail.com schrieb am 26.05.2015 um 04:24:
> Has anyone on this list attempted to sandbox Python programs in a
> serious fashion? I'd be interested to hear your approach.

Not quite sandboxing Python, but I've seen people use my Lupa [1] library
for this. They're writing all their code in Python, and then let users
embed their own Lua code into it to script their API. The Lua runtime is
apparently quite good at sandboxing, and it's really small, just some 600KB
or so. Lupa then lets you easily control the access to your Python code at
a whitelist level by intercepting all Python attribute lookups.

It doesn't add much to your application to embed Lua (or even LuaJIT) in
Python, and it gives users a nicely object oriented language to call and
orchestrate your Python objects.

Stefan


[1] https://pypi.python.org/pypi/lupa

[toc] | [prev] | [next] | [standalone]

#91384

On Fri, May 29, 2015 at 4:41 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
> davidfstr@gmail.com schrieb am 26.05.2015 um 04:24:
>> Has anyone on this list attempted to sandbox Python programs in a
>> serious fashion? I'd be interested to hear your approach.
>
> Not quite sandboxing Python, but I've seen people use my Lupa [1] library
> for this. They're writing all their code in Python, and then let users
> embed their own Lua code into it to script their API. The Lua runtime is
> apparently quite good at sandboxing, and it's really small, just some 600KB
> or so. Lupa then lets you easily control the access to your Python code at
> a whitelist level by intercepting all Python attribute lookups.
>
> It doesn't add much to your application to embed Lua (or even LuaJIT) in
> Python, and it gives users a nicely object oriented language to call and
> orchestrate your Python objects.

Lua's a much weaker language than Python is, though. Can it handle
arbitrary-precision integers? Unicode? Dare I even ask,
arbitrary-precision rationals (fractions.Fraction)? Security comes at
a price, I guess.

ChrisA

[toc] | [prev] | [next] | [standalone]

#91493

Chris Angelico <rosuav@gmail.com> writes:
>> It doesn't add much to your application to embed Lua
> Lua's a much weaker language than Python is, though.  Can it handle
> arbitrary-precision integers? Unicode? Dare I even ask,
> arbitrary-precision rationals (fractions.Fraction)? Security comes at
> a price, I guess.

The language features are an orthogonal issue to embeddability.  Lua is
easier to embed securely because its embedding interface was designed
for that.  It's also easy to call C functions from Lua, so if you want
arbitrary precision integers or rationals, you can link GMP into your
application use it from your Lua scripts.  

As another example, Javascript is as powerful as Python (though worse in
many ways due to misdesign), but also by now supports reasonably secure
embedding (or else it wouldn't be usable in browsers).

See
http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/#the-damn-interpreter
for Armin Ronacher's comments on CPython's not-so-great embedding
interface.  The interface he contrasts it with is essentially how Lua
and Javascript (at least in some implementations) work.  I haven't
looked at MicroPython or PyPy.

[toc] | [prev] | [next] | [standalone]

#91496

Paul Rubin <no.email@nospam.invalid>:

> The language features are an orthogonal issue to embeddability.

I doubt that. Guile is designed for embedding but it is a full-fledged
Scheme implementation.

> Lua is easier to embed securely because its embedding interface was
> designed for that.

I have very little experience with Lua. What surprises me is that it is
not as elementary as it could be: I don't really see the value of
metatables and weak tables.


Marko

[toc] | [prev] | [next] | [standalone]

#91499

Marko Rauhamaa <marko@pacujo.net> writes:
>> The language features are an orthogonal issue to embeddability.
> I doubt that. Guile is designed for embedding but it is a full-fledged
> Scheme implementation.

Orthogonal means independent, not opposing.

> I have very little experience with Lua. What surprises me is that it is
> not as elementary as it could be: I don't really see the value of
> metatables and weak tables.

It lets you implement various flavors of OOP etc. without different
language mechanisms.

[toc] | [prev] | [next] | [standalone]

#91431

Chris Angelico schrieb am 28.05.2015 um 20:51:
> On Fri, May 29, 2015 at 4:41 AM, Stefan Behnel wrote:
>> davidfstr@gmail.com schrieb am 26.05.2015 um 04:24:
>>> Has anyone on this list attempted to sandbox Python programs in a
>>> serious fashion? I'd be interested to hear your approach.
>>
>> Not quite sandboxing Python, but I've seen people use my Lupa [1] library
>> for this. They're writing all their code in Python, and then let users
>> embed their own Lua code into it to script their API. The Lua runtime is
>> apparently quite good at sandboxing, and it's really small, just some 600KB
>> or so. Lupa then lets you easily control the access to your Python code at
>> a whitelist level by intercepting all Python attribute lookups.
>>
>> It doesn't add much to your application to embed Lua (or even LuaJIT) in
>> Python, and it gives users a nicely object oriented language to call and
>> orchestrate your Python objects.
> 
> Lua's a much weaker language than Python is, though. Can it handle
> arbitrary-precision integers? Unicode? Dare I even ask,
> arbitrary-precision rationals (fractions.Fraction)?

All of those and way more, as long as you use it embedded in Python.


> Security comes at a price, I guess.

Sure, but features aren't the price here.

Stefan

[toc] | [prev] | [next] | [standalone]

#91433

On Fri, May 29, 2015 at 4:18 PM, Stefan Behnel <stefan_ml@behnel.de> wrote:
>> Lua's a much weaker language than Python is, though. Can it handle
>> arbitrary-precision integers? Unicode? Dare I even ask,
>> arbitrary-precision rationals (fractions.Fraction)?
>
> All of those and way more, as long as you use it embedded in Python.

Okay, so how would you go about using Lua-embedded-in-Python to
manipulate Unicode text? I'm not talking about things like the
unicodedata module and the name/codepoint lookups, I'm talking about
the basics of working with international text and the fundamental need
for your native string type to cope with that. Do you have to keep
bouncing back and forth between Python and Lua? And if you do
arithmetic in the single most obvious way, what happens?

http://www.lua.org/pil/2.3.html

Looks to me as if Lua doesn't have integers at all, and so the obvious
form of arithmetic will be equivalent to doing everything in Python
floats. That's not arbitrary precision. There's also a comment that
you can use some other type for numbers, but I suspect that's still
"some other C type", so you still can't do arbitrary precision. (Plus
you get one type for ints and floats, so even if you did select some
magic type that bounces through to a Python int, it'd stop you from
doing any non-integral arithmetic at all.)

http://www.lua.org/pil/2.4.html

Likewise, eight-bit strings, not Unicode. While I could accept some
sort of inter-language thunk for something uncommon, like
fractions.Fraction, forcing people to thunk back and forth for basic
arithmetic and string manipulation is way too much hassle - which
means they won't do it, which means strings will be eight-bit and
numbers will be doubles. And eight-bit strings might be treated as
UTF-8, or might be treated as some arbitrary codepage, or might be
both at once in different contexts, so you can't really depend on them
being anything more than ASCII.

Or can you change all this when you embed Lua?

ChrisA

[toc] | [prev] | [next] | [standalone]

#91495

Chris Angelico <rosuav@gmail.com> writes:
> Looks to me as if Lua doesn't have integers at all

They fixed that in Lua 5.3:

  http://www.lua.org/manual/5.3/readme.html#changes

> Likewise, eight-bit strings, not Unicode. 

Also fixed in 5.3 (basic utf-8 support added, per above).

[toc] | [prev] | [next] | [standalone]

#91507

On Sat, May 30, 2015 at 4:33 AM, Paul Rubin <no.email@nospam.invalid> wrote:
> Chris Angelico <rosuav@gmail.com> writes:
>> Looks to me as if Lua doesn't have integers at all
>
> They fixed that in Lua 5.3:
>
>   http://www.lua.org/manual/5.3/readme.html#changes

That's 64-bit integers, not arbitrary-precision, but that's something
at least. You do still need to worry about what happens when your
numbers get too big; in Python, you simply don't. So it's still not
quite there in terms of functionality.

>> Likewise, eight-bit strings, not Unicode.
>
> Also fixed in 5.3 (basic utf-8 support added, per above).

This is definitely NOT what I'm talking about. From what I can see,
5.3 introduces a new library for working with strings of bytes as if
they were UTF-8. That's on par with PHP's Unicode support - basically,
nothing that isn't explicitly coded. It makes Unicode something that
you tack on with resignation, rather than something that's just an
automatic part of text handling. So, a long way from being there in
functionality.

Do you see what I mean about functionality being sacrificed for
security? There is no way that this could be called fully functional
by comparison with Python.

ChrisA

[toc] | [prev] | [next] | [standalone]

#91511

Chris Angelico <rosuav@gmail.com> writes:
> Do you see what I mean about functionality being sacrificed for
> security? 

No I don't.  Lua has less functionality because it was designed to have
a small embedding footprint.  Python is much bigger because it was
mostly designed to run as a standalone interpreter.  That has nothing to
do with security.  You haven't shown the slightest connection between
Lua's lower functionality and its higher sandbox security, because there
is none.  The lower functionality is because of a totally independent
reason, namely the desire to make the interpreter smaller.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web