Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #22443 > unrolled thread

Re: string interpolation for python

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 00:39 -0700
    Re: string interpolation for python Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2012-04-02 11:01 +0300
    Re: string interpolation for python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-04-02 08:26 +0000
      Re: string interpolation for python Chris Angelico <rosuav@gmail.com> - 2012-04-02 18:47 +1000
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 02:11 -0700
        Re: string interpolation for python Duncan Booth <duncan.booth@invalid.invalid> - 2012-04-02 10:19 +0000
        Re: string interpolation for python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-04-02 11:54 +0000
          Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 05:40 -0700
            Re: string interpolation for python Laurent Claessens <moky.math@gmail.com> - 2012-04-02 15:02 +0200
              Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 07:25 -0700
              Re: Re: string interpolation for python Evan Driscoll <driscoll@cs.wisc.edu> - 2012-04-02 15:36 -0500
            Re: string interpolation for python mwilson@the-wire.com - 2012-04-02 10:46 -0400
              Re: string interpolation for python mwilson@the-wire.com - 2012-04-02 11:34 -0400
                Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 09:02 -0700
        Re: string interpolation for python rusi <rustompmody@gmail.com> - 2012-04-02 06:04 -0700
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 02:17 -0700
        Re: string interpolation for python alex23 <wuwei23@gmail.com> - 2012-04-02 22:47 -0700
      Re: string interpolation for python Chris Angelico <rosuav@gmail.com> - 2012-04-02 19:56 +1000
      Re: string interpolation for python Chris Rebert <clp2@rebertia.com> - 2012-04-02 03:23 -0700
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 04:46 -0700
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 05:00 -0700
      Re: string interpolation for python Chris Angelico <rosuav@gmail.com> - 2012-04-03 00:58 +1000
        Re: string interpolation for python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-04-02 18:56 +0000
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 08:49 -0700
      Re: string interpolation for python Chris Angelico <rosuav@gmail.com> - 2012-04-03 08:38 +1000
        Re: string interpolation for python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-04-02 23:18 +0000
      Re: string interpolation for python Yingjie Lan <lanyjie@yahoo.com> - 2012-04-02 18:57 -0700

Page 2 of 2 — ← Prev page 1 [2]

#22461

> "Are you %(name)s" % locals() # or vars()



This partly solves the problem, however, you 
can't work with expressions inside, like:

> d"sin($x$) = $sin(x)$"


Also, what if locals() or vars() does not contain
the variable "x"? (x could be nonlocal or global).


> It's more conservative than hostile. Here's some insight:
> http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html
>

You are probably right...Harmless enhancement is still probable?


> 
> Personally, in isolation, the only part of your proposal I find
> /truly/ objectionable is the support for arbitrary expressions, since
> it would tend towards encouraging suboptimal factoring. But we also

I don't quite see that as a problem. The compiler (or translator, as you 
mentioned earlier) could easily make d"sin($x$) = $sin(x)$" into
something like: ''.join(["sin(", str(x), ") = ", str(sin(x))]
which would far more efficient than calling the format() method.

Cheers,
Yingjie

[toc] | [prev] | [next] | [standalone]

#22473

On Mon, Apr 2, 2012 at 9:46 PM, Yingjie Lan <lanyjie@yahoo.com> wrote:
>>>>>  "Are you "+name+"?"
>>
>> That allows arbitrary expressions and everything.
>>
>
> To make that work for any type, you need:
>
>>>> "Are you "+ str(name) + "?"
>
> Another concern is performance.
>
> You are absolutely right, they are
> equivalent in that both are expressions.

Right, meaning that both have the same issues of performance, need for
str(), etc. There's absolutely no difference.

> As long as people start to realize that
> dynamic strings are expressions,
> there is no magic in it any more.

And no benefit. You lose out on syntax highlighting in your editor and
gain nothing.

> And allowing expressions in those
> dynamic strings would make sense
> since they are of the same sort.
>
>>>> d"sin($x$) = $ sin(x):0.3f $"
>
> is equivalent to the expression of
>
>>>> "sin(%s"%x + ")= %0.3f"%sin(x)
>
> Comparing th e two, I would say the latter
> is more computer friendly while
> the former, more human friendly.

The former is more magical, the second is more explicit. Computers do
tend to like the explicit, but I wouldn't assume that humans like the
magical *necessarily*. There are times when we prefer the explicit,
too.

> Sure, once you get used to it, it would be harder to stop it
>  the harder it is :). That's part of human nature, anyway.

Maybe. But it (percent-notation) is expressive and insanely powerful.
Moreover, it obeys the rule that you pay for the complexity you use,
no more and no less. (Although I think there's one lack in Python's
implementation - I can't find a way to use an argument more than once,
without switching to "dictionary mode" and using keys for everything.
I can't, for instance, use "Hello hello, %s %[0]s!"%name to use the
name twice. But since that isn't in the original C implementation,
it's hardly mandatory.)

Some implementations go even further than Python's does and allow an
array notation.

sprintf("UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
key=%d",array_of_field_names,primary_key_value)
--> "UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
WHERE key=1234"

You're still paying for no complexity you aren't actually using. It's
clear and readable.

> I would simply say: this new way is much more simple
> and much more powerful. And there is no security issues
> as long as you don't use the evil eval to evaluate expressions,
> which is always a security issue.

It's powerful only if you use eval to allow full expression syntax.
Otherwise, what does it have above str.format()?

> It is new, and has no compatibility issues with old ways at all.
> In syntax, all you need is to allow d"...", which clearly won't
> affect any old ways of business.

You may well be able to get past the compatibility issues. I'm not yet
convinced that the new syntax is worth it, but it may be possible.

Here's a recommendation: Write a parser for your notation that turns
it into executable Python code (that is, executable in Python 3.3
without any d"..." support). Since a dynamic string is really an
expression in disguise, it doesn't need to be translated at run time,
and in fact is best translated at compile time. It wouldn't be hard to
make the precompiler as per your equivalency above; and since you can
use "%s"%123 without raising TypeError, you don't even have to figure
out data types. With that written, the Python community can more
adequately evaluate your proposal, and even start making use of it and
getting a "feel" for the new syntax. If it catches on, someone'll
probably invite you to write a PEP or something; if it doesn't, well,
you still have it on your own version, and you don't have to worry
about upgrades (your precompiler/"patch" should be able to slide up to
a new version easily).

I'm still against the idea personally, mainly because it's nothing but
a more magical way of doing what we already can do, but there may well
be many who disagree.

Chris Angelico

[toc] | [prev] | [next] | [standalone]

#22478

On Tue, 03 Apr 2012 00:58:38 +1000, Chris Angelico wrote:

> Maybe. But it (percent-notation) is expressive and insanely powerful.
> Moreover, it obeys the rule that you pay for the complexity you use, no
> more and no less. (Although I think there's one lack in Python's
> implementation - I can't find a way to use an argument more than once,
> without switching to "dictionary mode" and using keys for everything. I
> can't, for instance, use "Hello hello, %s %[0]s!"%name to use the name
> twice. But since that isn't in the original C implementation, it's
> hardly mandatory.)

Ack. 

In this case, you can use format:

>>> "Hello {0}, what's up with {1}? Hey, {0} I'm speaking to you!".format
('George', 'Melissa')
"Hello George, what's up with Melissa? Hey, George I'm speaking to you!"



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#22475

> Right, meaning that both have the same issues 
> of performance, need for

> str(), etc. There's absolutely no difference.


OK, performance. Here is a new solution:
 
Suppose we have a new string method
    str.format_join([...])
taking a list of strings and objects,
with even-indexed ones being strings,
odd-indexed ones being objects.
Each even-indexed string *ends* with a formatting
specification for the next object in the list.
Then we can have:
>>> d"sin($x$) = $ sin(x):0.3f $"
get translated to:
>>> ''.format_join(["sin(%s",x,") = %0.3f", sin(x)])
This seems to be at least as good in performance.


> And no benefit. You lose out on syntax highlighting 
> in your editor and gain nothing.

Gain: readability, terseness, hassle-free, and possibly
better performance if done right.

Syntax highlighting: can be done more creatively.
For dynamic strings, string parts are like normal 
strings, but the embedded expressions are like
normal expressions :)

> 
> sprintf("UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
> key=%d",array_of_field_names,primary_key_value)
> --> "UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
> WHERE key=1234"
> 
> You're still paying for no complexity you aren't actually using. 
> It's clear and readable.

You are really good at that. Maybe not everybody is as
experience as you, and I suppose the learning curve is 
kind of hard to climb.

> It's powerful only if you use eval to allow full expression syntax.
> Otherwise, what does it have above str.format()?


Those expressions are embedded, you don't need eval()
to have the result though. Are we on the same page?

> You may well be able to get past the compatibility issues. I'm not yet
> convinced that the new syntax is worth it, but it may be possible.
> 
> Here's a recommendation: Write a parser for your notation that turns
> it into executable Python code (that is, executable in Python 3.3
> without any d"..." support). 

You mean a translator?

The syntax is essential for compatibility.
We must distinguish dynamic strings from common strings.
They will live peacefully together. 
(escaping the '$' in normal strings breaks compatibility, 
and the consequence of forgetting to escape could be
disastrous, so definitely not an option). 

May be d" is too tiny, $"..." is easier to pick out.



Cheers,
Yingjie

[toc] | [prev] | [next] | [standalone]

#22552

On Tue, Apr 3, 2012 at 4:56 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Tue, 03 Apr 2012 00:58:38 +1000, Chris Angelico wrote:
>
>> I can't find a way to use an argument more than once,
>> without switching to "dictionary mode" and using keys for everything.
>
> Ack.
>
> In this case, you can use format:
>
>>>> "Hello {0}, what's up with {1}? Hey, {0} I'm speaking to you!".format
> ('George', 'Melissa')
> "Hello George, what's up with Melissa? Hey, George I'm speaking to you!"

Yes, it's possible with that version but not with % formatting. (Is
your "Ack" short for "Acknowledgement", or is it an ejaculation of
disgust?)

On Tue, Apr 3, 2012 at 1:49 AM, Yingjie Lan <lanyjie@yahoo.com> wrote:
> Then we can have:
>>>> d"sin($x$) = $ sin(x):0.3f $"
> get translated to:
>>>> ''.format_join(["sin(%s",x,") = %0.3f", sin(x)])
> This seems to be at least as good in performance.

Or don't bother with the initial string, and simply pass everything as
arguments:

def yingjie_format(*args):
    it=iter(args)
    return ''.join(s%next(it,None) for s in it)
yingjie_format("sin(%s",x,") = %0.3f", sin(x))

Tested and working (Python 3.2 on Windows). Okay, the generator's a
bit cryptic, but you could do the same with a more explicit loop if
you prefer.

> Syntax highlighting: can be done more creatively.
> For dynamic strings, string parts are like normal
> strings, but the embedded expressions are like
> normal expressions :)

So if they're exactly like normal expressions, why not simply use
normal expressions?

>> sprintf("UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
>> key=%d",array_of_field_names,primary_key_value)
>> --> "UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
>> WHERE key=1234"
>>
>> You're still paying for no complexity you aren't actually using.
>> It's clear and readable.
>
> You are really good at that. Maybe not everybody is as
> experience as you, and I suppose the learning curve is
> kind of hard to climb.

Yes, it takes some learning to use it. But that's true of everything,
no less of your magical string interpolation. My point is that simple
examples should be (and are, with printf formatting) simple, such that
you only get those more complicated format strings when you're
actually doing a complicated job (in that case, taking each element of
an array and using it twice - actually, it was taking the indices of a
mapping that would end up being passed to the DB query function, thus
providing values to the :foo :bar variables).

> Those expressions are embedded, you don't need eval()
> to have the result though. Are we on the same page?

I can see three implementation paths:

1) Language feature. It really *is* just an expression. There's no way
that a user can provide them, so there's actually no similarity to
eval. But this requires that Python itself handle things.

2) Precompiler. It *becomes* an expression. Again, perfectly safe,
although I don't know how useful this really is.

3) Functoin. As several have suggested, you could do some magic and
use d("this is a $dollar$ $interpolated$ string") to implement. For
this, you *will* need eval (or something like it).

>> Here's a recommendation: Write a parser for your notation that turns
>> it into executable Python code (that is, executable in Python 3.3
>> without any d"..." support).
>
> You mean a translator?

Yes. It translates your dollar-strings into something that's legal
Python 3.3 syntax - either calls to a function like I provided above,
or actual embedded expressions.

> The syntax is essential for compatibility.
> We must distinguish dynamic strings from common strings.
> They will live peacefully together.
> (escaping the '$' in normal strings breaks compatibility,
> and the consequence of forgetting to escape could be
> disastrous, so definitely not an option).
>
> May be d" is too tiny, $"..." is easier to pick out.

I don't like the use of symbols like that; can someone, glancing at
your code, tell whether $ is an operator, a name, or something else?
The original d is probably better for that.

ChrisA

[toc] | [prev] | [next] | [standalone]

#22555

On Tue, 03 Apr 2012 08:38:13 +1000, Chris Angelico wrote:

> (Is your
> "Ack" short for "Acknowledgement", or is it an ejaculation of disgust?)

Acknowledgement.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#22565

>>>  I can't find a way to use an argument more than once,

>>>  without switching to "dictionary mode" and using keys for 
>>> everything.


Even in "dictionary mode", the key is spelled more than

once. The "tuple mode" below seems to save some typing. 

However, when there are more and more items to format, 
the readability deteriorates quickly in the "tuple" mode.

Use an argument more than once is, after all, probably a
not-so-often use case.


The "dictionary mode" greatly enhances readability, but 
you have to provide a dict object and make sure the name in 
your formatting string matches the keys in the dictionary.
And this matching requirement is because of information
redundancy in the arrangement. And information redundancy
is often the root of evil for various kinds of trouble.

Dynamic string seems to have the good without redundancy.
And there is no need to build a dict for it. 
However, when there is already the dict for use, 
clearly the dict format is the winner.
>>  Ack.
>> 
>>  In this case, you can use format:
>> 
>>>>>  "Hello {0}, what's up with {1}? Hey, {0} I'm 
> speaking to you!".format
>>  ('George', 'Melissa')
>>  "Hello George, what's up with Melissa? Hey, George I'm 
> speaking to you!"
> 
> Or don't bother with the initial string, and simply pass everything as
> arguments:
> 
> def yingjie_format(*args):
>     it=iter(args)
>     return ''.join(s%next(it,None) for s in it)
> yingjie_format("sin(%s",x,") = %0.3f", sin(x))
> 

That's very nice, thanks!

> So if they're exactly like normal expressions, why not simply use

> normal expressions?


I think use dynamic strings can have these benefits:
1) you less keys (punch less keys).
2) better readability (less clutters)
3) you don't have to explicit convert/format expressions into strings
4) better performance too (adding strings together is generally slow).
 

>>>  sprintf("UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
>>>  key=%d",array_of_field_names,primary_key_value)
>>>  --> "UPDATE tablename SET 
> modified=now(),foo=:foo,bar=:bar,quux=:quux
>>>  WHERE key=1234"
>>> 
>>>  You're still paying for no complexity you aren't actually 
> using.
>>>  It's clear and readable.
>> 
>>  You are really good at that. Maybe not everybody is as
>>  experience as you, and I suppose the learning curve is
>>  kind of hard to climb.
> 
> Yes, it takes some learning to use it. But that's true of everything,
> no less of your magical string interpolation. My point is that simple
> examples should be (and are, with printf formatting) simple, such that
> you only get those more complicated format strings when you're
> actually doing a complicated job (in that case, taking each element of
> an array and using it twice - actually, it was taking the indices of a
> mapping that would end up being passed to the DB query function, thus
> providing values to the :foo :bar variables).
> 

Sure. But if one thing does well on both simple and complex 
situations, why not that thing?

>>  Those expressions are embedded, you don't need eval()
>>  to have the result though. Are we on the same page?
> 
> I can see three implementation paths:
> 
> 1) Language feature. It really *is* just an expression. There's no way
> that a user can provide them, so there's actually no similarity to
> eval. But this requires that Python itself handle things.
> 
> 2) Precompiler. It *becomes* an expression. Again, perfectly safe,
> although I don't know how useful this really is.
> 
> 3) Functoin. As several have suggested, you could do some magic and
> use d("this is a $dollar$ $interpolated$ string") to implement. For
> this, you *will* need eval (or something like it).
> 

Sure. for 1), things can be done most conveniently and efficiently.
for 2), yeah, not sure how useful it is.
for 3), maybe can let str class have a property like: dy.
which can do all the dirty processing. Then we may do:
>>> x = 0
>>> "sin($x$) = $sin(x)$".dy
'sin(0) = 0.0'

Not much burden to use except for the CPU, I suppose.

>>  You mean a translator?
> 
> Yes. It translates your dollar-strings into something that's legal
> Python 3.3 syntax - either calls to a function like I provided above,
> or actual embedded expressions.
> 
>>  The syntax is essential for compatibility.
>>  We must distinguish dynamic strings from common strings.
>>  They will live peacefully together.
>>  (escaping the '$' in normal strings breaks compatibility,
>>  and the consequence of forgetting to escape could be
>>  disastrous, so definitely not an option).
>> 
>>  May be d" is too tiny, $"..." is easier to pick out.
> 
> I don't like the use of symbols like that; can someone, glancing at
> your code, tell whether $ is an operator, a name, or something else?
> The original d is probably better for that.
> 


Yeah, the d is probably better. 

Cheers,
Yingjie

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web