Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #66741 > unrolled thread

Commonly-used names in the Python standard library

Back to article view | Back to comp.lang.python

Contents

  Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-20 20:39 +1100
    Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 12:22 +0200
      Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-20 21:43 +1100
        Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 13:28 +0200
          Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-20 22:37 +1100
          Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-20 22:47 +1100
            Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 14:09 +0200
              Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-20 23:19 +1100
                Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 14:46 +0200
                  Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 00:34 +1100
                    Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 17:14 +0200
                      Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 02:48 +1100
                        Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 18:26 +0200
                          Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 03:36 +1100
                          Re: Commonly-used names in the Python standard library 88888 Dihedral <dihedral88888@gmail.com> - 2014-02-21 12:57 -0800
                  Re: Commonly-used names in the Python standard library Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-02-21 06:49 +0000
              Re: Commonly-used names in the Python standard library Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-02-21 06:43 +0000
      Re: Commonly-used names in the Python standard library Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-02-21 06:59 +0000
        Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 18:10 +1100
        Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-21 09:21 +0200
          Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 19:21 +1100
            Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-21 12:26 +0200
              Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 21:49 +1100
                Re: Commonly-used names in the Python standard library Marko Rauhamaa <marko@pacujo.net> - 2014-02-21 14:03 +0200
                  Re: Commonly-used names in the Python standard library Ethan Furman <ethan@stoneleaf.us> - 2014-02-21 11:16 -0800
          Re: Commonly-used names in the Python standard library Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-02-21 09:12 +0000
    Re: Commonly-used names in the Python standard library Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-02-21 06:51 +0000
      Re: Commonly-used names in the Python standard library Chris Angelico <rosuav@gmail.com> - 2014-02-21 18:02 +1100

Page 1 of 2  [1] 2  Next page →

#66741 — Commonly-used names in the Python standard library

In working on a proposal that might result in the creation of a new
keyword, I needed to ascertain what names were used extensively in
existing Python code. Out of random curiosity, I asked my script what
names were the most commonly used. The script responded with 21854
names and a total of 297164 references, averaging 13-14 refs per name.

A good number of names are referenced just once - set and never
referenced. They're there for applications to use. That takes out 6217
names. But I'm more interested in the ones that see a lot of use.

By far the most common name is 'self', for obvious reasons; after
that, it's a fairly steady drop-off. The most popular names in the
standard library are... *drumroll*

45298 - self
3750 - os
3744 - name
3166 - i
3140 - s
2685 - value
2648 - a
2451 - len
2348 - c
2331 - sys
2255 - b
2238 - line
2132 - print
2131 - x

Few surprises there. The os and sys modules are used extensively, and
short variable names are reused frequently. To the print-detractors:
That's two thousand uses in the standard library, more than any other
single function bar 'len'! (And by the way, this is ignoring any file
with /test/ in the name.)

I find the pairing of 'name' and 'value' interesting. There are 40%
more names than values in Python, apparently. And on that bombshell,
as they say on Top Gear, it's time to end!

ChrisA

[toc] | [next] | [standalone]

#66744

Chris Angelico <rosuav@gmail.com>:

> In working on a proposal that might result in the creation of a new
> keyword,

I'm looking forward to the day when every application can add its own
keywords as is customary in Lisp.

> I needed to ascertain what names were used extensively in existing
> Python code

One advantage of Perl is that names and keywords are in separate
namespaces so introducing new keywords is easy.

Should Python have something like:

  from py35 import *

That way, you could choose between:

  unless x > 7:
      return

and:

  py35.unless x > 7:
      return

in case you have already made use of the name "unless" in your program.


Marko

[toc] | [prev] | [next] | [standalone]

#66745

On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>   from py35 import *
>
> That way, you could choose between:
>
>   unless x > 7:
>       return
>
> and:
>
>   py35.unless x > 7:
>       return
>
> in case you have already made use of the name "unless" in your program.

What about return? Are you allowed to namespace that? And 'from' and
'import' and '*'?

In languages with keywords, they're there to signal things to the
parser. There are languages that have no keywords at all, like REXX,
and their grammars are usually restricted to non-alphabetic tokens
(for instance, REXX has & and | instead of "and" and "or"). Python
already has most of its important names in either builtins (which can
be shadowed) or actual modules, so it's only actual language keywords
that can't be reused; and there aren't all that many of those. But
more can be created, and it's worth being careful.

In this instance, various proposals included "then", "when", "use",
and "raises". My script reported the following:

1 instances of the name 'use'
12 instances of the name 'when'

and none of either of the others. Granted, the stdlib isn't
everything, and isn't even reliably representative, but that supported
my gut feeling that keywording 'when' would be likely to trip code up.

If you're curious about the full proposal, it's PEP 463, an expression
form of the 'except' statement. The latest draft PEP can be found
here:

https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt

and the official repo (currently out of date, but later on will be the
official and maintained version) has it here:

http://www.python.org/dev/peps/pep-0463/

ChrisA

[toc] | [prev] | [next] | [standalone]

#66747

Chris Angelico <rosuav@gmail.com>:

> On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>   py35.unless x > 7:
>>       return
>
> What about return? Are you allowed to namespace that? And 'from' and
> 'import' and '*'?

Old keywords are guaranteed not to clash with programs. Introducing new
keywords runs that risk. Hence, C had to introduce the ugly _Bool
keyword.

> If you're curious about the full proposal, it's PEP 463, an expression
> form of the 'except' statement. The latest draft PEP can be found
> here:
>
> https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt

A coworker pointed out that the gist of the PEP has already been
implemented by <URL: https://pypi.python.org/pypi/fuckit>.


Marko

[toc] | [prev] | [next] | [standalone]

#66748

On Thu, Feb 20, 2014 at 10:28 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> A coworker pointed out that the gist of the PEP has already been
> implemented by <URL: https://pypi.python.org/pypi/fuckit>.

I love how that's categorized "Topic :: Software Development ::
Quality Assurance". It certainly assures _something_ about quality...

ChrisA

[toc] | [prev] | [next] | [standalone]

#66749

On Thu, Feb 20, 2014 at 10:28 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>>   py35.unless x > 7:
>>>       return
>>
>> What about return? Are you allowed to namespace that? And 'from' and
>> 'import' and '*'?
>
> Old keywords are guaranteed not to clash with programs. Introducing new
> keywords runs that risk. Hence, C had to introduce the ugly _Bool
> keyword.

Okay, so what you're saying is that there are three states:

Before Python X.Y, the unless keyword simply doesn't exist. (It can't
be coded in as a module, so it can't exist until someone implements
the code.)

>From X.Y, it can be called up by importing it from "pyAB" and used in
its namespace.

>From A.B onward, it always exists.

Python has a facility like this. It doesn't namespace the keywords,
but it does let you choose whether to have them or not. In Python 2.5,
you could type "from __future__ import with_statement" to turn 'with'
into a keyword. After Python 2.6, it's always a keyword.

ChrisA

[toc] | [prev] | [next] | [standalone]

#66750

Chris Angelico <rosuav@gmail.com>:

> Python has a facility like this. It doesn't namespace the keywords,
> but it does let you choose whether to have them or not. In Python 2.5,
> you could type "from __future__ import with_statement" to turn 'with'
> into a keyword. After Python 2.6, it's always a keyword.

That certainly softens the blow but might still cause unnecessary
suffering when maintaining/resurrecting legacy Python code.

How about blocking the introduction of new keywords for ever except if
you specify:

   from __py35__ import syntax

Eventually, every Python module would likely begin with a statement like
that, and it would document the assumption more clearly than __future__.


Marko

[toc] | [prev] | [next] | [standalone]

#66751

On Thu, Feb 20, 2014 at 11:09 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> How about blocking the introduction of new keywords for ever except if
> you specify:
>
>    from __py35__ import syntax
>
> Eventually, every Python module would likely begin with a statement like
> that, and it would document the assumption more clearly than __future__.

It's more self-documenting with the __future__ directive, because it
says *what* syntax you're importing from the future. And at some
point, the new keywords must just become standard. There's no point
polluting every Python script forever with these directives, and no
point maintaining two branches of code in the interpreter.

ChrisA

[toc] | [prev] | [next] | [standalone]

#66753

Chris Angelico <rosuav@gmail.com>:

> On Thu, Feb 20, 2014 at 11:09 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>    from __py35__ import syntax
>
> It's more self-documenting with the __future__ directive, because it
> says *what* syntax you're importing from the future.

As a developer, I will probably want to state the Python dialect that
was used to write the module. Each dialect comes with hundreds of
features. I don't want to list them individually (even if I could).

> And at some point, the new keywords must just become standard.

That's an explicit program of destroying backwards-compatibility: a war
on legacy code. That may be the Python way, but it's not a necessary
strategy.

> There's no point polluting every Python script forever with these
> directives, and no point maintaining two branches of code in the
> interpreter.

Two branches? I would imagine there would be dozens of "branches" in the
interpreter if the latest interpreter were to support all past Python
dialects (as it should, IMO).


Marko

[toc] | [prev] | [next] | [standalone]

#66754

On Thu, Feb 20, 2014 at 11:46 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>> And at some point, the new keywords must just become standard.
>
> That's an explicit program of destroying backwards-compatibility: a war
> on legacy code. That may be the Python way, but it's not a necessary
> strategy.
>
>> There's no point polluting every Python script forever with these
>> directives, and no point maintaining two branches of code in the
>> interpreter.
>
> Two branches? I would imagine there would be dozens of "branches" in the
> interpreter if the latest interpreter were to support all past Python
> dialects (as it should, IMO).

Indeed. If the interpreter were to include every dialect of "old
Python", then it would have a lot more than two branches. They would,
in fact, increase exponentially with every Python version.
Fortunately, there is an alternative. You can specify the version of
Python like this:

#!/usr/local/bin/python3.4

or any of several other ways. You then choose exactly which versions
of Python to have installed, and continue to use them for as long as
you wish. There's no reason for the 3.4 interpreter to be able to run
code "as if it were" the 3.1 interpreter, when you can just have the
3.1 interpreter itself right there.

ChrisA

[toc] | [prev] | [next] | [standalone]

#66759

Chris Angelico <rosuav@gmail.com>:

> If the interpreter were to include every dialect of "old Python", then
> it would have a lot more than two branches. They would, in fact,
> increase exponentially with every Python version.

It shouldn't be *that bad*; the linux kernel is grappling with the glut
of system calls, but they are managing it reasonably well. I don't see
why Python, especially at this mature stage, couldn't adopt a similar
stance *going forward*.

In fact, not every syntax change requires special
backwards-compatibility treatment in the compiler. Constructs that used
to be illegal might become legal (say, try-except-finally). They don't
require any attention. Even new keywords have a very small impact on the
parser; it should be a simple matter of enabling dictionary entries.

> Fortunately, there is an alternative. You can specify the version of
> Python like this:
>
> #!/usr/local/bin/python3.4

Well,

 * you won't be finding old Python versions on newer operating system
   distributions,

 * even <URL: http://www.python.org/downloads/> isn't all that extensive
   and

 * the program may import modules that were written in different Python
   dialects.


Marko

[toc] | [prev] | [next] | [standalone]

#66765

On Fri, Feb 21, 2014 at 2:14 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>  * you won't be finding old Python versions on newer operating system
>    distributions,
>
>  * even <URL: http://www.python.org/downloads/> isn't all that extensive
>    and
>
>  * the program may import modules that were written in different Python
>    dialects.

You can always build your own Python, if it really matters... but more
likely, if you care about old versions, you actually care about *one
specific old version* which your program uses. That's why Red Hat
still supports Python 2.4 and, I think, 2.3. You can't randomly pick
up 2.2 or 1.5, but if you want 2.4, you can keep on using that for as
long as this RHEL is supported.

As to importing modules written for other versions... that can be a
major problem. Often the new keywords come with new functionality.
Take string exceptions, for instance. Say you import a module that was
written for a version that still supported them - if it raises a
string, you can't catch it. There is a limit to how far the
compatibility can be taken. Also, what happens if two modules (one of
which might be your script) written for different versions both import
some third module? Should they get different versions, based on what
version tags they use themselves? Compatibility can't be changed that
easily. You either run on the new version, or run on the old. Not
both.

ChrisA

[toc] | [prev] | [next] | [standalone]

#66770

Chris Angelico <rosuav@gmail.com>:

> Also, what happens if two modules (one of which might be your script)
> written for different versions both import some third module? Should
> they get different versions, based on what version tags they use
> themselves? Compatibility can't be changed that easily. You either run
> on the new version, or run on the old. Not both.

Shared C libraries face the exact same issue. Java seems pretty good on
this front as well. When there is a will, there is a way.


Marko

[toc] | [prev] | [next] | [standalone]

#66772

On Fri, Feb 21, 2014 at 3:26 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> Also, what happens if two modules (one of which might be your script)
>> written for different versions both import some third module? Should
>> they get different versions, based on what version tags they use
>> themselves? Compatibility can't be changed that easily. You either run
>> on the new version, or run on the old. Not both.
>
> Shared C libraries face the exact same issue. Java seems pretty good on
> this front as well. When there is a will, there is a way.

Shared C libraries usually do it by linking against a particular
version. That's why you often need to keep multiple versions around.
Once it's all binary code, there's no more compatibility question - it
all runs on the same CPU. With Python code, the module's written to
run on a particular interpreter, and that can't just switch around -
it's like the weird and wonderful life I enjoyed as 32-bit computing
started coming along, and I wanted to call on code that used the other
word length...

ChrisA

[toc] | [prev] | [next] | [standalone]

#66847

On Friday, February 21, 2014 12:26:00 AM UTC+8, Marko Rauhamaa wrote:
> Chris Angelico <rosuav@gmail.com>:
> 
> 
> 
> > Also, what happens if two modules (one of which might be your script)
> 
> > written for different versions both import some third module? Should
> 
> > they get different versions, based on what version tags they use
> 
> > themselves? Compatibility can't be changed that easily. You either run
> 
> > on the new version, or run on the old. Not both.
> 
> 
> 
> Shared C libraries face the exact same issue. Java seems pretty good on
> 
> this front as well. When there is a will, there is a way.
> 
> 
> 
> 
> 
> Marko

Well, please check jython first 
under OS Andrioids( LINUX+JAVA-XWIN).
The mobile phone SW applications are getting hoter and hoter these days.

[toc] | [prev] | [next] | [standalone]

#66797

On Thu, 20 Feb 2014 14:46:35 +0200, Marko Rauhamaa wrote:

> I would imagine there would be dozens of "branches" in the interpreter
> if the latest interpreter were to support all past Python dialects (as
> it should, IMO).

Well thank goodness you're not in charge of Python's future development. 
That way leads to madness: madness for the core developers (if you think 
maintaining Python 2 and 3 branches is hard imagine maintaining *dozens* 
of them, *forever*), madness of the programmers using the language, and 
madness for anyone trying to learn the language. It's hard enough for 
newbies to deal with *two* dialects, 2 and 3. And you want to introduce 
dozens. Thanks, but no thanks.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#66794

On Thu, 20 Feb 2014 14:09:19 +0200, Marko Rauhamaa wrote:

> Chris Angelico <rosuav@gmail.com>:
> 
>> Python has a facility like this. It doesn't namespace the keywords, but
>> it does let you choose whether to have them or not. In Python 2.5, you
>> could type "from __future__ import with_statement" to turn 'with' into
>> a keyword. After Python 2.6, it's always a keyword.
> 
> That certainly softens the blow but might still cause unnecessary
> suffering when maintaining/resurrecting legacy Python code.
> 
> How about blocking the introduction of new keywords for ever except if
> you specify:
> 
>    from __py35__ import syntax
> 
> Eventually, every Python module would likely begin with a statement like
> that, and it would document the assumption more clearly than __future__.

What *actual* problem is this supposed to solve? Do you often find that 
Python has introduced new keywords, breaking your code?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#66799

On Thu, 20 Feb 2014 12:22:29 +0200, Marko Rauhamaa wrote:

> Chris Angelico <rosuav@gmail.com>:
> 
>> In working on a proposal that might result in the creation of a new
>> keyword,
> 
> I'm looking forward to the day when every application can add its own
> keywords as is customary in Lisp.

And what a wonderful day that will be! Reading any piece of code you 
didn't write yourself -- or wrote a long time ago -- will be an 
adventure! Every script will have it's own exciting new set of keywords 
doing who knows what, which makes every script nearly it's own language! 
Oh joy, I cannot wait!

That's sarcasm, by the way.


>> I needed to ascertain what names were used extensively in existing
>> Python code
> 
> One advantage of Perl is that names and keywords are in separate
> namespaces so introducing new keywords is easy.

Then I can write code like:

for for in in:
    while while:
        if if:
            raise raise

which will go a long way to ensuring that my code is an hostile and 
unreadable as possible.


(Sometimes, less can be more. That's especially true of programming 
languages.)


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#66803

On Fri, Feb 21, 2014 at 5:59 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Then I can write code like:
>
> for for in in:
>     while while:
>         if if:
>             raise raise
>
> which will go a long way to ensuring that my code is an hostile and
> unreadable as possible.

REXX allows that. Most people wouldn't use classic keywords like 'if',
as that'll only cause confusion (although "if if then then; else else"
is legal), but some of the other keywords are useful in other
contexts. The main advantage is that, for instance, the PARSE command
can freely use keywords:

PARSE VAR x blah blah
PARSE VALUE linein(blah) WITH blah blah

All those words (parse, var, value, with) are keywords - in that
context. But I can happily use "var" and "value" elsewhere, and will
do so. Python, on the other hand, has to be more careful; so you see
things like "cls" instead of "class", or "import_" and so on, with the
trailing underscore.

Trade-offs.

ChrisA

[toc] | [prev] | [next] | [standalone]

#66806

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> On Thu, 20 Feb 2014 12:22:29 +0200, Marko Rauhamaa wrote:
>> I'm looking forward to the day when every application can add its own
>> keywords as is customary in Lisp.
>
> And what a wonderful day that will be! Reading any piece of code you
> didn't write yourself -- or wrote a long time ago -- will be an
> adventure! Every script will have it's own exciting new set of
> keywords doing who knows what, which makes every script nearly it's
> own language! Oh joy, I cannot wait!
>
> That's sarcasm, by the way.

I don't hear Lispers or C programmers complaining. Yes, you can shoot
yourself in the foot with macro trickery, but macros can greatly enhance
code readability and remove the need for code generators. That's why
they are used extensively in Linux kernel code and GOOPS (Guile's object
system), for example.

>> One advantage of Perl is that names and keywords are in separate
>> namespaces so introducing new keywords is easy.
>
> Then I can write code like:
>
> for for in in:
>     while while:
>         if if:
>             raise raise
>
> which will go a long way to ensuring that my code is an hostile and
> unreadable as possible.

Perl does that by forcing you to prefix you variables with $ et al. The
analogy would be:

for $for in @in:
    while $while:
        if $if:
            raise $raise

I'm not saying I like the look of that, but it does have a distinct
advantage in introducing new keywords.


Marko

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web