Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Date: Thu, 12 Jan 2012 15:45:48 -0500
Subject: Symbolic expressions (or: partials and closures from the inside out)
From: Nathan Rice <nathan.alexander.rice@gmail.com>
To: python-list <python-list@python.org>, python-ideas <python-ideas@python.org>
Content-Type: text/plain; charset=ISO-8859-1
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.4692.1326401152.27778.python-list@python.org>
Lines: 94
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:18882

Greetings,

I have been writing a lot of code lately that involves creating
symbolic expressions of one form or another, which are then fully
evaluated at a later time.  Examples of this include Elementwise,
where I create expressions that act on every member of an iterable
(there is a much improved new version coming soon, by the way), and a
design by contract/validation lib I'm working on (which shall remain
nameless :D) that uses symbolic expressions in the *args of the
metaclass __new__ method to generate a constraint class which
validates input using __instancecheck__.  I do most of this with
lambdas, a little hacking with closures and FunctionType(), and
chainable objects.  I am very impressed that python is this flexible,
but there are some issues with the approach that I would like to
rectify, namely:

1.  Because of the early binding behavior of most things in Python, if
I want to include isinstance(X, someclass) in a symbolic expression, I
have to wrap it in a lambda (or use .apply(), in the case of
Elementwise).  This is not a huge deal for me, but it forces me to
create wrappers for lots of functions (e.g. isinstance_(X, someclass))
and/or have users wrap every such function they want to use in a
symbolic expression.   Having to do this also bloats the code a lot;
the github version of Elementwise is over 3,000 LoC at this point
(including prodigious documentation, but still...).

2.  Python expects that certain functions, such as int(), str(), etc,
will have a specific return type.  While in general I agree with this,
it makes Elementwise somewhat inconsistent (and it will do the same to
anything else that wants to work with symbolic expressions).

I'm interested in fixing both issues. I believe both issues I've had
could be solved by having a robust "symbolic object".  These objects
would basically usable like ordinary objects, however upon any
attribute access or other form of interaction, the object would
basically short circuit the calling function, and return a symbolic
object directly to the outer scope.  The symbolic object would behave
like a generator function frozen at the point of attribute access, and
upon send()-ing (or whatever method), it would behave exactly as if
the values sent had been the ones passed in originally (ideally
without consuming the generator).

I have thought about ways to approximate this behavior python
currently, and while I could hack something together using inspect to
pull relevant info from the stack, then break out using a special
exception (potentially passing a generator with state as an exception
arg), this approach strikes me as VERY brittle, implementation
dependent, ugly and difficult to work with.  Additionally, you would
need to catch the special exception somewhere in the stack, so this
trick wouldn't work on the first thing in an expression to be
evaluated.

As an aside, I'd like to solicit some feedback on the validation
syntax I've been working on.  Currently, I have code that support
things like:

X = SymbolicObject()

const = Constraints(X * 2 + 1 >= 5, X % 2 != 0)
const2 = Constraints(X[-1] == "h")
const3 = Constraints(X[-1].upper() == "H")

>>> print isinstance(3, const)
True

>>> print isinstance(2, const)
False

>>> print isinstance(1, const)
False

>>> print isinstance("bleh", const2)
True

>> print isinstance("bleh", const3)
True

Callables are supported as well, so if you wanted to do something like:

Constraints(isinstance(X.attr, someclass), somefunc(X[-2].attr, args))

You could approximate that with:

Constraints(lambda x: isinstance(x.attr, someclass), lambda x:
somefunc(x[-2].attr, args))

As I mentioned in the first paragraph, Constraints is a metaclass, so
your validations are checked using __instancecheck__.  I'm also
considering having __init__ generate mock objects (for certain
straight-forward cases, anyhow).

Thanks for your time,

Nathan