Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #75468 > unrolled thread

dict to boolean expression, how to?

Back to article view | Back to comp.lang.python

Contents

  dict to boolean expression, how to? Alex van der Spek <zdoor@xs4all.nl> - 2014-08-01 12:45 +0000
    Re: dict to boolean expression, how to? Chris Angelico <rosuav@gmail.com> - 2014-08-01 23:04 +1000
    Re: dict to boolean expression, how to? Roy Smith <roy@panix.com> - 2014-08-01 09:24 -0400
    Re: dict to boolean expression, how to? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-01 13:28 +0000
      Re: dict to boolean expression, how to? Roy Smith <roy@panix.com> - 2014-08-01 09:32 -0400
        Re: dict to boolean expression, how to? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-01 15:02 +0000
          Re: dict to boolean expression, how to? Terry Reedy <tjreedy@udel.edu> - 2014-08-01 18:16 -0400
      Re: dict to boolean expression, how to? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-08-01 16:50 +0100
        Re: dict to boolean expression, how to? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-01 16:35 +0000
    Re: dict to boolean expression, how to? Alex van der Spek <zdoor@xs4all.nl> - 2014-08-01 14:26 +0000
      Re: dict to boolean expression, how to? marco.nawijn@colosso.nl - 2014-08-01 08:07 -0700
      Re: dict to boolean expression, how to? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-01 15:24 +0000
        Re: dict to boolean expression, how to? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-08-01 17:03 +0100
      Re: dict to boolean expression, how to? Peter Otten <__peter__@web.de> - 2014-08-01 17:44 +0200
        eval [was Re: dict to boolean expression, how to?] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-02 02:59 +0000
          Re: eval [was Re: dict to boolean expression, how to?] Peter Otten <__peter__@web.de> - 2014-08-02 09:43 +0200
          Re: eval [was Re: dict to boolean expression, how to?] Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-08-02 08:59 +0100
          Re: eval [was Re: dict to boolean expression, how to?] Duncan Booth <duncan.booth@invalid.invalid> - 2014-08-05 11:57 +0000
            Re: eval [was Re: dict to boolean expression, how to?] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-06 02:25 +1000
      Re: dict to boolean expression, how to? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-08-01 17:12 +0100
      Re: dict to boolean expression, how to? Peter Pearson <ppearson@nowhere.invalid> - 2014-08-01 17:00 +0000
    Re: dict to boolean expression, how to? Ned Batchelder <ned@nedbatchelder.com> - 2014-08-01 10:29 -0400

Page 1 of 2  [1] 2  Next page →

#75468 — dict to boolean expression, how to?

With a dict like so:

cond = {'a': 1, 'b': 1, 'c': 1,
        'A': 0, 'B', 0, 'C':0}

how would you make a boolean expression like this:

bool = (('a' == 1) & ('A' == 0) |
        ('b' == 1) & ('B' == 0) |
        ('c' == 1) & ('C' == 0))

The fact that lowercase and uppercase keys are stringed together with & is 
intentional albeit the actual condition is a bit more tricky.

I've tried several approaches using eval() on a string built from the dict 
but landed with just spelling it out literally.


Any pointers welcome.
Alex

[toc] | [next] | [standalone]

#75469

On Fri, Aug 1, 2014 at 10:45 PM, Alex van der Spek <zdoor@xs4all.nl> wrote:
> how would you make a boolean expression like this:
>
> bool = (('a' == 1) & ('A' == 0) |
>         ('b' == 1) & ('B' == 0) |
>         ('c' == 1) & ('C' == 0))

Not sure what the use of this is, because 'a' will never be equal to
1. Are you trying to magically fetch names from your environment?
Because that's usually a bad idea.

ChrisA

[toc] | [prev] | [next] | [standalone]

#75471

In article <53db8bd8$0$2976$e4fe514c@news2.news.xs4all.nl>,
 Alex van der Spek <zdoor@xs4all.nl> wrote:

> With a dict like so:
> 
> cond = {'a': 1, 'b': 1, 'c': 1,
>         'A': 0, 'B', 0, 'C':0}
> 
> how would you make a boolean expression like this:
> 
> bool = (('a' == 1) & ('A' == 0) |
>         ('b' == 1) & ('B' == 0) |
>         ('c' == 1) & ('C' == 0))
> 
> The fact that lowercase and uppercase keys are stringed together with & is 
> intentional albeit the actual condition is a bit more tricky.
> 
> I've tried several approaches using eval() on a string built from the dict 
> but landed with just spelling it out literally.

I would certainly avoid eval(), especially if there's any chance 
(including ways you can't possibly imagine right now) of external data 
getting included.  Google for "little bobby tables".

Maybe something like (coding on the fly, not tested):

terms = []
for key in 'abc':
    term = cond[key] and not cond[key.upper()]
    terms.append(term)

bool_value = any(terms)

[toc] | [prev] | [next] | [standalone]

#75472

On Fri, 01 Aug 2014 12:45:12 +0000, Alex van der Spek wrote:

> With a dict like so:
> 
> cond = {'a': 1, 'b': 1, 'c': 1,
>         'A': 0, 'B', 0, 'C':0}
> 
> how would you make a boolean expression like this:
> 
> bool = (('a' == 1) & ('A' == 0) |
>         ('b' == 1) & ('B' == 0) |
>         ('c' == 1) & ('C' == 0))

That's False. It's always False, because 'a' does not equal 1, etc. Also, 
you're using bitwise operators & and | rather than boolean operators. 
Finally, you are shadowing the built-in bool() type, which is probably a 
bad idea.

In the first case, I think you mean cond['a'] == 1 rather than just 
'a'==1; in the second case, the bool operators are called "and" and "or"; 
and in the third case, there are many equally good names for a generic 
boolean flag, like "flag".

Putting those together, I think you probably mean something like this:

flag = (
    (cond['a'] == 1 and cond['A'] == 0) or
    (cond['b'] == 1 and cond['B'] == 0) or
    (cond['c'] == 1 and cond['C'] == 0)
    )

which can be simplified to:

flag = any( cond[c] == 1 and cond[c.upper()] for c in ['a', 'b', 'c'] )

If you *really* mean bitwise-and, you can change the "and"s to & and 
"or"s to | but honesty that's pointless and inefficient because all your 
conditions are bools, not arbitrary integers.


> The fact that lowercase and uppercase keys are stringed together with &
> is intentional albeit the actual condition is a bit more tricky.
> 
> I've tried several approaches using eval() on a string built from the
> dict but landed with just spelling it out literally.

Ayyyeee!!!! Don't use eval(). Really. Any time you think you need to use 
eval(), smash your hand with a hammer until the urge goes away.

*wink*

But seriously: if you need to ask "why not use eval?", you're not ready 
to use eval safely. But in a nutshell:

- using eval is a large performance hit (about 10 times slower);

- eval is dangerous and can introduce code injection vulnerabilities.

eval is almost never the right solution to any problem, and in the very 
few exceptions, it needs careful handling by an expert to ensure you're 
not introducing serious security bugs.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75475

In article <53db95e6$0$29986$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> eval is almost never the right solution to any problem, and in the very 
> few exceptions, it needs careful handling by an expert to ensure you're 
> not introducing serious security bugs.

Corollary to that rule: All the people who are smart enough to actually 
understand how to use eval() safety, are also smart enough to know not 
to use it.

[toc] | [prev] | [next] | [standalone]

#75483

On Fri, 01 Aug 2014 09:32:36 -0400, Roy Smith wrote:

> In article <53db95e6$0$29986$c3e8da3$5496439d@news.astraweb.com>,
>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> 
>> eval is almost never the right solution to any problem, and in the very
>> few exceptions, it needs careful handling by an expert to ensure you're
>> not introducing serious security bugs.
> 
> Corollary to that rule: All the people who are smart enough to actually
> understand how to use eval() safety, are also smart enough to know not
> to use it.

... smart enough to know WHEN to use it (which is *rarely*).

That's in production code, of course. There's nothing wrong with using 
eval in the interactive interpreter for quick and dirty exploration. But 
even then, I find that it's usually easier to write a line or two of 
Python code to process something than to try using eval.

There are uses for eval (or exec), even if production code. See 
collections.namedtuple, doctest, and timeit, for example. In the first 
place, namedtuple takes extra care to sanitise the data being used. In 
the case of doctest and timeit, the whole point of them is to run trusted 
code. If you can't trust your own code that you're timing, what can you 
trust?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75503

On 8/1/2014 11:02 AM, Steven D'Aprano wrote:
> On Fri, 01 Aug 2014 09:32:36 -0400, Roy Smith wrote:
>
>> In article <53db95e6$0$29986$c3e8da3$5496439d@news.astraweb.com>,
>>   Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>>
>>> eval is almost never the right solution to any problem, and in the very
>>> few exceptions, it needs careful handling by an expert to ensure you're
>>> not introducing serious security bugs.
>>
>> Corollary to that rule: All the people who are smart enough to actually
>> understand how to use eval() safety, are also smart enough to know not
>> to use it.

Eval is a specialized version of exec. Everything you do on a computer 
is evaluating and executing code. Here is a highly simplified version of 
a Python interpreter:

user_program = get_user_python_code()
__main__ = make_global_namespace()
exec(user_program, __main__, __main__)

CPython codes an expansion of the above in C. Idle uses an expansion of 
the above to do the same thing.

The threat model is executing code from someone who does not have 
physical access to a machine to just run code and who should not be trusted.

> ... smart enough to know WHEN to use it (which is *rarely*).
>
> That's in production code, of course. There's nothing wrong with using
> eval in the interactive interpreter for quick and dirty exploration. But
> even then, I find that it's usually easier to write a line or two of
> Python code to process something than to try using eval.
>
> There are uses for eval (or exec), even if production code. See
> collections.namedtuple, doctest, and timeit, for example. In the first
> place, namedtuple takes extra care to sanitise the data being used. In
> the case of doctest and timeit, the whole point of them is to run trusted
> code. If you can't trust your own code that you're timing, what can you
> trust?

Exactly. If someone can start Python or Idle on a machine, they can 
start Windows Explorer and Command Prompt. Idle exec's user code because 
emulating the interactive interpreter is part of its purpose. It 
sometimes evals expression within user code in response to user requests.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#75490

On 01/08/2014 14:28, Steven D'Aprano wrote:
> On Fri, 01 Aug 2014 12:45:12 +0000, Alex van der Spek wrote:
>
>> With a dict like so:
>>
>> cond = {'a': 1, 'b': 1, 'c': 1,
>>          'A': 0, 'B', 0, 'C':0}
>>
>> how would you make a boolean expression like this:
>>
>> bool = (('a' == 1) & ('A' == 0) |
>>          ('b' == 1) & ('B' == 0) |
>>          ('c' == 1) & ('C' == 0))
>
> That's False. It's always False, because 'a' does not equal 1, etc. Also,
> you're using bitwise operators & and | rather than boolean operators.
> Finally, you are shadowing the built-in bool() type, which is probably a
> bad idea.
>
> In the first case, I think you mean cond['a'] == 1 rather than just
> 'a'==1; in the second case, the bool operators are called "and" and "or";
> and in the third case, there are many equally good names for a generic
> boolean flag, like "flag".
>
> Putting those together, I think you probably mean something like this:
>
> flag = (
>      (cond['a'] == 1 and cond['A'] == 0) or
>      (cond['b'] == 1 and cond['B'] == 0) or
>      (cond['c'] == 1 and cond['C'] == 0)
>      )
>
> which can be simplified to:
>
> flag = any( cond[c] == 1 and cond[c.upper()] for c in ['a', 'b', 'c'] )
>

Shouldn't that be cond[c.upper()] == 0 ?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#75493

On Fri, 01 Aug 2014 16:50:51 +0100, Mark Lawrence wrote:

>> which can be simplified to:
>>
>> flag = any( cond[c] == 1 and cond[c.upper()] for c in ['a', 'b', 'c'] )
>>
>>
> Shouldn't that be cond[c.upper()] == 0 ?

Yes it should be, thank you!


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75478

On Fri, 01 Aug 2014 12:45:12 +0000, Alex van der Spek wrote:

> With a dict like so:
> 
> cond = {'a': 1, 'b': 1, 'c': 1,
>         'A': 0, 'B', 0, 'C':0}
> 
> how would you make a boolean expression like this:
> 
> bool = (('a' == 1) & ('A' == 0) |
>         ('b' == 1) & ('B' == 0) |
>         ('c' == 1) & ('C' == 0))
> 
> The fact that lowercase and uppercase keys are stringed together with &
> is intentional albeit the actual condition is a bit more tricky.
> 
> I've tried several approaches using eval() on a string built from the
> dict but landed with just spelling it out literally.
> 
> 
> Any pointers welcome.
> Alex

I am sorry, the problem is ill posed.

'a', 'A' and so forth are my failed attempt to shorthand.

In reality the dict's keys are column names in a pandas dataframe df.

The boolean expression would therefore look like:

bool = ((df['a'] == 1) & (df['A'] == 0) |
         (df['b'] == 1) & (df['B'] == 0) |
         (df['c'] == 1) & (df['C'] == 0))

I do know eval() lends itself to code injection but can't say I am
fully aware of its dangers. It seemed like a handy tool to me.

This newsgroup scares me, it appears to be for professional computer 
scientists only, the theoretical part is sometimes too much for this 
practical physicist with an old background in FORTRAN.

Is there a better place to ask questions of this nature?

Alex van der Spek

[toc] | [prev] | [next] | [standalone]

#75484

Hi,

Are you aware of the Python operator module?
It provides function equivalents of all (most?)
python operator. So instead of a==b, you can
state operator.eq(a,b). As a result, you can
loop over the key/value pairs in the dict and
built your logic with the operator.eq, 
operator.and_, and operator.or_ (notice the 
trailing underscore to avoid clashing with
normal "and" and "or" statements.

Marco
 

[toc] | [prev] | [next] | [standalone]

#75486

On Fri, 01 Aug 2014 14:26:38 +0000, Alex van der Spek wrote:

[...]
> bool = ((df['a'] == 1) & (df['A'] == 0) |
>          (df['b'] == 1) & (df['B'] == 0) |
>          (df['c'] == 1) & (df['C'] == 0))
> 
> I do know eval() lends itself to code injection but can't say I am fully
> aware of its dangers. It seemed like a handy tool to me.

And in expert hands, it can be. But I can't see how you're using eval in 
this case. Since you're not having any success, perhaps it's not as handy 
as you imagine?

Did you try the suggestion to use any( ... )? Did it work?

 
> This newsgroup scares me, it appears to be for professional computer
> scientists only, 

Not even close! Some of us aren't even professional programmers!


> the theoretical part is sometimes too much for this
> practical physicist with an old background in FORTRAN.

Oh, I sympathise. I know it can be hard to avoid feeling overwhelmed when 
you are dropped far outside of your comfort zone in a topic you know 
nothing about. But you're a physicist, and therefore you've already 
proven you have the logical skills to reason about quite complex and 
mathematical topics. If you're lost now, I wager it's because you're 
unfamiliar with the territory, not because it's beyond you.

So please, don't be shy about asking people to explain in simpler terms. 
Sometimes we forget that things which are second nature to us are not 
familiar to somebody new to the language. Or we don't want to patronise 
them by assuming that every person asking a question needs their hand-
held and every little detail spelled out in painfully small steps. It's 
quite hard to find the right balance, especially when we don't know your 
level of knowledge.

So please, ask concrete questions, the more specific the better. Don't be 
afraid to ask what unfamiliar terms means.


> Is there a better place to ask questions of this nature?

I don't know of anyone here who is an expert in pandas, so if you ask 
questions which are specific to pandas, we may run into the limits of our 
knowledge. If you can find a dedicated pandas mailing list or other 
forum, they may help too, but I don't know if they will be more or less 
willing to explain the basics about Python.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75491

On 01/08/2014 16:24, Steven D'Aprano wrote:
> I don't know of anyone here who is an expert in pandas, so if you ask
> questions which are specific to pandas, we may run into the limits of our
> knowledge. If you can find a dedicated pandas mailing list or other
> forum, they may help too, but I don't know if they will be more or less
> willing to explain the basics about Python.
>

https://mail.python.org/mailman/listinfo/pandas-dev or 
gmane.comp.python.pydata

As it's so handy see here 
http://news.gmane.org/index.php?prefix=gmane.comp.python for the all 
python lists on gmane.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#75489

[Multipart message — attachments visible in raw view] — view raw

Alex van der Spek wrote:


> I do know eval() lends itself to code injection but can't say I am
> fully aware of its dangers. It seemed like a handy tool to me.

In a lab if you don't need to protect your script against attacks from 
outside eval() (and exec()) is fine. If the data fed to eval() is completely 
under your control (think collections.namedtuple) eval() is also fine.

Adding a public web interface on such a lab application means trouble.

> This newsgroup scares me, 

If you dare say that you are not scared enough ;)

> it appears to be for professional computer
> scientists only, the theoretical part is sometimes too much for this
> practical physicist with an old background in FORTRAN.

That sounds like you are experienced enough to say "There may be problems 
with this code, but I choose not to care about them this time -- at my own 
risk"
 
> Is there a better place to ask questions of this nature?

There is the tutor mailing list which is mostly geared at absolute 
beginners, does more hand-holding, and the threads are more likely to stay 
on topic. You might take a look, but with your background you are probably 
better off here.

> I am sorry, the problem is ill posed.
> 
> 'a', 'A' and so forth are my failed attempt to shorthand.
> 
> In reality the dict's keys are column names in a pandas dataframe df.
> 
> The boolean expression would therefore look like:
> 
> bool = ((df['a'] == 1) & (df['A'] == 0) |
>          (df['b'] == 1) & (df['B'] == 0) |
>          (df['c'] == 1) & (df['C'] == 0))
 
This is how it might look without eval():

#untested
result = functools.reduce(operator.or_, ((v == 1) & (df[k.upper()] == 0) for 
k, v in df.items() if k.islower()))

And here is an eval-based solution:

# untested
expr = "|".join(
    "((df[{}] == 1) | (df[{}] == 0))".format(c, c.upper()) 
    for c in df is c.islower())
result = eval(expr)

[toc] | [prev] | [next] | [standalone]

#75514 — eval [was Re: dict to boolean expression, how to?]

On Fri, 01 Aug 2014 17:44:27 +0200, Peter Otten wrote:

> Alex van der Spek wrote:
> 
> 
>> I do know eval() lends itself to code injection but can't say I am
>> fully aware of its dangers. It seemed like a handy tool to me.
> 
> In a lab if you don't need to protect your script against attacks from
> outside eval() (and exec()) is fine. If the data fed to eval() is
> completely under your control (think collections.namedtuple) eval() is
> also fine.

I'm not entirely happy with describing eval as "fine", even when you're 
not concerned with security. There are at least two problems with eval 
that should make it a tool of last resort:

* its less direct, typically more convoluted, which makes it 
  hard to read, harder to write, and harder to reason about;

* it's slower than running code directly (in my experience, 
  about ten times slower).


If you have the choice between running code directly, or running eval() 
or exec() that runs the same code, you should nearly always prefer to run 
the code directly.

y = x + 1  # Yes.
y = eval("x + 1")  # No.
y = eval("eval('x + 1')")  # Good grief what are you thinking???


Now obviously nobody sensible is going to use eval in production code for 
such simple expressions as "x+1", but the same principle applies even for 
more complex examples. If you can write the code once, as source code, 
it's usually better than generating the source code at runtime and 
running it with eval or exec.

Consider the namedtuple implementation in the standard library. There's a 
lot of criticism of it, some of it justified. It uses exec extensively, 
which means the code is dominated by a giant string template. This 
defeats your editor's syntax colouring, makes refactoring harder, and 
makes how the namedtuple works rather less understandable. It seems to me 
that it's only generating the __new__ method which genuinely needs to use 
exec, the rest of the namedtuple could and should use just an ordinary 
class object (although I concede that some of this is just a matter of 
personal taste).

Raymond Hettinger's original, using exec for the entire inner class:

http://code.activestate.com/recipes/500261-named-tuples/


My refactoring, with the bare minimum use of exec necessary:

https://code.activestate.com/recipes/578918-yet-another-namedtuple/


[...]
>> bool = ((df['a'] == 1) & (df['A'] == 0) |
>>          (df['b'] == 1) & (df['B'] == 0) |
>>          (df['c'] == 1) & (df['C'] == 0))
>  
> This is how it might look without eval():
> 
> #untested
> result = functools.reduce(operator.or_, ((v == 1) & (df[k.upper()] == 0)
> for k, v in df.items() if k.islower()))

For those who agree with Guido that reduce makes code unreadable:

result = True
for key in df:
    if key.islower():
        result = result or (df[key] == 1 and df[key.upper()] == 0)


Or if you insist on a single expression:

result = any(df[k] == 1 and df[k.upper()] == 0 for k in df if k.islower())


> And here is an eval-based solution:
> 
> # untested
> expr = "|".join(
>     "((df[{}] == 1) | (df[{}] == 0))".format(c, c.upper()) for c in df
>     is c.islower())
> result = eval(expr)

I really don't believe that there is any benefit to that in readability, 
power, flexibility, or performance. Also, you're using bitwise operators 
instead of shortcut bool operators. Any reason why?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75522 — Re: eval [was Re: dict to boolean expression, how to?]

Steven D'Aprano wrote:

> On Fri, 01 Aug 2014 17:44:27 +0200, Peter Otten wrote:
> [...]
>>> bool = ((df['a'] == 1) & (df['A'] == 0) |
>>>          (df['b'] == 1) & (df['B'] == 0) |
>>>          (df['c'] == 1) & (df['C'] == 0))
>>  
>> This is how it might look without eval():
>> 
>> #untested
>> result = functools.reduce(operator.or_, ((v == 1) & (df[k.upper()] == 0)
>> for k, v in df.items() if k.islower()))
> 
> For those who agree with Guido that reduce makes code unreadable:
> 
> result = True
> for key in df:
>     if key.islower():
>         result = result or (df[key] == 1 and df[key.upper()] == 0)

I cheated a bit and gave the solution that the OP was unlikely to come up 
with ;)


> Or if you insist on a single expression:
> 
> result = any(df[k] == 1 and df[k.upper()] == 0 for k in df if k.islower())
> 
> 
>> And here is an eval-based solution:
>> 
>> # untested
>> expr = "|".join(
>>     "((df[{}] == 1) | (df[{}] == 0))".format(c, c.upper()) for c in df
>>     is c.islower())
>> result = eval(expr)
> 
> I really don't believe that there is any benefit to that in readability,
> power, flexibility, or performance. 

You can put in a print(expr) to verify that it's identical to the original 
spelt out expression.

> Also, you're using bitwise operators
> instead of shortcut bool operators. Any reason why?

Since he started the thread Alex has moved the goal posts a bit and added 
that he is using a pandas DataFrame. That works much like a numpy array. So:

>>> df
   a  A    b     B   c   C
0  1  4  100  1000   7  14
1  2  1  200  2000  21  28
2  1  0  300  3000  35  42
>>> df["a"] == 1
0     True
1    False
2     True
Name: a, dtype: bool
>>> (df["a"] == 1) and (df["A"] == 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is 
ambiguous. Use a.any() or a.all()
>>> (df["a"] == 1) & (df["A"] == 0)
0    False
1    False
2     True
dtype: bool

[toc] | [prev] | [next] | [standalone]

#75524 — Re: eval [was Re: dict to boolean expression, how to?]

On 02/08/2014 03:59, Steven D'Aprano wrote:
>
> My refactoring, with the bare minimum use of exec necessary:
>
> https://code.activestate.com/recipes/578918-yet-another-namedtuple/
>

FTR I get the feed of new recipes from 
gwene.com.activestate.code.feeds.recipes.langs.python from news.gmane.org.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#75727 — Re: eval [was Re: dict to boolean expression, how to?]

Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> Consider the namedtuple implementation in the standard library.
> There's a lot of criticism of it, some of it justified. It uses exec
> extensively, which means the code is dominated by a giant string
> template. This defeats your editor's syntax colouring, makes
> refactoring harder, and makes how the namedtuple works rather less
> understandable. It seems to me that it's only generating the __new__
> method which genuinely needs to use exec, the rest of the namedtuple
> could and should use just an ordinary class object (although I concede
> that some of this is just a matter of personal taste).
> 
> Raymond Hettinger's original, using exec for the entire inner class:
> 
> http://code.activestate.com/recipes/500261-named-tuples/
> 
> 
> My refactoring, with the bare minimum use of exec necessary:
> 
> https://code.activestate.com/recipes/578918-yet-another-namedtuple/


This may be a silly question, but what would stop you moving the exec inside the class?

So:
    ns = {'_new': tuple.__new__}

    class Inner(tuple):
        # Work around for annoyance: type __doc__ is read-only :-(
        __doc__ = ("%(typename)s(%(argtxt)s)"
                   % {'typename': typename, 'argtxt': argtxt})

        __slots__ = ()
        _fields = field_names

        exec """def __new__(_cls, %(argtxt)s):
        return _new(_cls, (%(argtxt)s))""" % { 'argtxt': argtxt } in ns, locals()

... and so on ...

and remove lines from 'ns = ...' to 'Inner.__new__ = ...'

The tests at the end of the file still pass so I'm not sure whether there is any situation
that wouldn't work.

For that matter I don't understand why tuple.__new__ needs to be pre-bound. Just referring 
to tuple.__new__ directly in the exec simplifies things even more as there is no need to 
specify any namespaces.

        exec """def __new__(_cls, %(argtxt)s):
        return tuple.__new__(_cls, (%(argtxt)s))""" % { 'argtxt': argtxt }

also passes the tests.

-- 
Duncan Booth 

[toc] | [prev] | [next] | [standalone]

#75745 — Re: eval [was Re: dict to boolean expression, how to?]

Duncan Booth wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
[...]
>> My refactoring, with the bare minimum use of exec necessary:
>> 
>> https://code.activestate.com/recipes/578918-yet-another-namedtuple/
> 
> 
> This may be a silly question, but what would stop you moving the exec
> inside the class?

I don't know. I haven't tried it. I didn't think of it at the time.

I don't have any specific arguments either against or in favour of your
suggestions, except to say that exec is sometimes tricky to get working
unless you explicitly specify the namespace to operate in.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#75492

On 01/08/2014 15:26, Alex van der Spek wrote:
> On Fri, 01 Aug 2014 12:45:12 +0000, Alex van der Spek wrote:
>
>> With a dict like so:
>>
>> cond = {'a': 1, 'b': 1, 'c': 1,
>>          'A': 0, 'B', 0, 'C':0}
>>
>> how would you make a boolean expression like this:
>>
>> bool = (('a' == 1) & ('A' == 0) |
>>          ('b' == 1) & ('B' == 0) |
>>          ('c' == 1) & ('C' == 0))
>>
>> The fact that lowercase and uppercase keys are stringed together with &
>> is intentional albeit the actual condition is a bit more tricky.
>>
>> I've tried several approaches using eval() on a string built from the
>> dict but landed with just spelling it out literally.
>>
>>
>> Any pointers welcome.
>> Alex
>
> I am sorry, the problem is ill posed.
>
> 'a', 'A' and so forth are my failed attempt to shorthand.
>
> In reality the dict's keys are column names in a pandas dataframe df.
>
> The boolean expression would therefore look like:
>
> bool = ((df['a'] == 1) & (df['A'] == 0) |
>           (df['b'] == 1) & (df['B'] == 0) |
>           (df['c'] == 1) & (df['C'] == 0))

See this 
http://stackoverflow.com/questions/19482970/python-get-list-from-pandas-dataframe-column-headers 
to get the column names.  Combine this with Steven D'Aprano's earlier 
answer using any() and I suspect you're there.

>
> I do know eval() lends itself to code injection but can't say I am
> fully aware of its dangers. It seemed like a handy tool to me.

It strikes me as being like looking down the gun barrel to see if 
there's a bullet loaded whilst holding the trigger.  Best avoided but 
there's always the "consenting adults" approach here :)

>
> This newsgroup scares me, it appears to be for professional computer
> scientists only, the theoretical part is sometimes too much for this
> practical physicist with an old background in FORTRAN.

If you're smart enough to state that your first question was ill posed, 
you're smart enough to ask questions when you don't understand.

>
> Is there a better place to ask questions of this nature?

No :)

>
> Alex van der Spek
>

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web