Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #87503 > unrolled thread

Re: Python 2 to 3 conversion - embrace the pain

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-16 14:22 +1100
    Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-15 22:07 -0700
      Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-16 16:39 +1100
        Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-15 23:17 -0700
          Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 19:00 +1100
            Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-16 01:31 -0700
              Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 00:05 +1100
                Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-17 00:29 +1100
              Re: Python 2 to 3 conversion - embrace the pain Terry Reedy <tjreedy@udel.edu> - 2015-03-16 13:59 -0400
        Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 18:56 +1100
          Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-16 01:25 -0700
            Re: Python 2 to 3 conversion - embrace the pain INADA Naoki <songofacandy@gmail.com> - 2015-03-16 18:13 +0900
              Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 22:55 +1100
            Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 01:25 +1100
              Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-16 15:36 -0700
                Re: Python 2 to 3 conversion - embrace the pain Terry Reedy <tjreedy@udel.edu> - 2015-03-16 22:28 -0400
                  Re: Python 2 to 3 conversion - embrace the pain wxjmfauth@gmail.com - 2015-03-17 01:31 -0700
                Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 22:26 +1100
                  Re: Python 2 to 3 conversion - embrace the pain wxjmfauth@gmail.com - 2015-03-17 07:03 -0700
                  Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-17 15:35 +0100
                  Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-19 16:23 -0700
                    Re: Python 2 to 3 conversion - embrace the pain Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-03-19 22:03 -0400
                      Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-20 03:54 +0100
            Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 11:20 -0600
              Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 07:36 +1100
                Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 20:14 -0600
                Re: Python 2 to 3 conversion - embrace the pain Cameron Simpson <cs@zip.com.au> - 2015-03-17 17:47 +1100
            Re: Python 2 to 3 conversion - embrace the pain Terry Reedy <tjreedy@udel.edu> - 2015-03-16 13:47 -0400
            Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-16 18:14 +0000
            Re: Python 2 to 3 conversion - embrace the pain INADA Naoki <songofacandy@gmail.com> - 2015-03-17 04:41 +0900
            Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-17 09:02 +1100
              Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-17 04:04 +0100
                Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 21:33 -0600
                  Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 19:36 +1100
                Re: Python 2 to 3 conversion - embrace the pain Ben Finney <ben+python@benfinney.id.au> - 2015-03-17 14:42 +1100
                  Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-17 05:30 +0100
                    Re: Python 2 to 3 conversion - embrace the pain Cameron Simpson <cs@zip.com.au> - 2015-03-18 08:53 +1100
                Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-17 14:49 +1100
                  Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-17 05:26 +0100
                    Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 04:36 +0000
                      Re: Python 2 to 3 conversion - embrace the pain Grant Edwards <invalid@invalid.invalid> - 2015-03-17 14:50 +0000
                        Re: Python 2 to 3 conversion - embrace the pain wxjmfauth@gmail.com - 2015-03-17 08:15 -0700
                        Re: Python 2 to 3 conversion - embrace the pain Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-03-17 18:36 -0400
                        Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 22:41 +0000
                        Re: Python 2 to 3 conversion - embrace the pain Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-03-18 08:04 -0400
                      Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-18 02:02 +0100
            [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 19:46 -0600
              Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 19:39 +1100
                Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-17 09:02 -0600
              Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-19 16:37 -0600
                Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain wxjmfauth@gmail.com - 2015-03-20 02:40 -0700
                  Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-20 15:59 +0100
                    Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-20 16:03 +0100
                    Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-20 15:42 +0000
                    Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-21 04:13 +1100
                    Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain wxjmfauth@gmail.com - 2015-03-20 12:21 -0700
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-17 12:57 +1100
              Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-16 19:45 -0700
                Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 21:05 -0600
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 20:07 -0600
            Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 02:21 +0000
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Ben Finney <ben+python@benfinney.id.au> - 2015-03-17 13:40 +1100
              Package manager cooperation? (was Weaknesses of distro package managers) Rustom Mody <rustompmody@gmail.com> - 2015-03-16 20:09 -0700
                Re: Package manager cooperation? (was Weaknesses of distro package managers) Michael Torrie <torriem@gmail.com> - 2015-03-16 21:32 -0600
                Re: Package manager cooperation? (was Weaknesses of distro package managers) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 04:03 +0000
                Re: Package manager cooperation? (was Weaknesses of distro package managers) Chris Angelico <rosuav@gmail.com> - 2015-03-17 15:18 +1100
                  Re: Package manager cooperation? (was Weaknesses of distro package managers) wxjmfauth@gmail.com - 2015-03-17 01:44 -0700
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 20:51 -0600
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 03:01 +0000
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Michael Torrie <torriem@gmail.com> - 2015-03-16 21:05 -0600
              Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mario Figueiredo <marfig@gmail.com> - 2015-03-17 04:07 +0100
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 03:17 +0000
            Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain Alexander <xr.lists@gmail.com> - 2015-03-17 16:44 +1300
          Re: Python 2 to 3 conversion - embrace the pain Chris Angelico <rosuav@gmail.com> - 2015-03-16 19:51 +1100
      Re: Python 2 to 3 conversion - embrace the pain Terry Reedy <tjreedy@udel.edu> - 2015-03-16 04:53 -0400
        Re: Python 2 to 3 conversion - embrace the pain Paul Rubin <no.email@nospam.invalid> - 2015-03-16 02:27 -0700

Page 1 of 4  [1] 2 3 4  Next page →

#87503 — Re: Python 2 to 3 conversion - embrace the pain

On Mon, Mar 16, 2015 at 1:53 PM, Cameron Simpson <cs@zip.com.au> wrote:
>> I would say that time clearly isn't the issue. Nine years IS enough...
>> if it's a matter of time. But since the bugs are still there, it means
>> that the problem is a lack of usage. Solution: Use it! Do the port to
>> Python 3, and file those upstream bug reports.
>
>
> One should mention that John did all of that.

Yep. I'm not saying that John did the wrong thing; what I'm saying is
that, sometimes, this kind of pain is the exact thing that makes life
better for everyone else. Someone has to be first, and if everyone
shies away from something for fear of being first, everyone misses out
on the benefit.

ChrisA

[toc] | [next] | [standalone]

#87504

Chris Angelico <rosuav@gmail.com> writes:
>>> Solution: Use it! Do the port to Python 3, and file those upstream
>>> bug reports.
>> One should mention that John did all of that.
> Yep. I'm not saying that John did the wrong thing; what I'm saying is
> that, sometimes, this kind of pain is the exact thing that makes life
> better for everyone else. Someone has to be first, and if everyone
> shies away from something for fear of being first, everyone misses out
> on the benefit.

It's great to decide to be first.  That's called signing up for a beta
test program, and lots of people do that when there's something new and
interesting that they want to use for something non-critical.  The issue
is when something is advertised as mature and working properly, when
it's really better described as still being in beta test.  I doubt John
thought he was signing up for that.

Python 2 is by now pretty solid and its users don't feel like beta
testers any more.  If you're saying using Python 3 by contrast means
"being first" and "reporting bugs", that basically translates to "stay
away from it except for experimentation".

I saved a quote from Hacker News a while back (I don't know who the
author is):

    "You know why I'm not running python 3? Because it doesn't solve a
    single problem I have. It doesn't solve anyone's problems. It solves
    imaginary problems, while creating real problems."

      https://news.ycombinator.com/item?id=7802575

I think the person went a bit overboard, but other than the Unicode
revamp I don't know what Python 3 improvements couldn't have been done
in Python 2 without breaking anything.

[toc] | [prev] | [next] | [standalone]

#87506

On Mon, Mar 16, 2015 at 4:07 PM, Paul Rubin <no.email@nospam.invalid> wrote:
> Python 2 is by now pretty solid and its users don't feel like beta
> testers any more.  If you're saying using Python 3 by contrast means
> "being first" and "reporting bugs", that basically translates to "stay
> away from it except for experimentation".

Ah but it isn't Py3 that's all about being first - it's the latest
version of some third-party module. This entire discussion came about
because of non-stdlib modules - Python itself is quite mature. You
might be the first to use XYZ with Py3, but you're not the first to
use XYZ, nor the first to use Py3.

ChrisA

[toc] | [prev] | [next] | [standalone]

#87507

Chris Angelico <rosuav@gmail.com> writes:
> Ah but it isn't Py3 that's all about being first - it's the latest
> version of some third-party module.

You know, one of the attractions of Python used to be that it came with
a powerful enough standard library that you didn't really need third
party modules very often.  I realize there's so much useful code out
there now that it's unrealistic to get by with just the stdlib as much
as before, but maybe that should have been taken as a sign of maturity,
calling for higher unwillingness to break stuff.

[toc] | [prev] | [next] | [standalone]

#87525

On Monday 16 March 2015 17:17, Paul Rubin wrote:

> Chris Angelico <rosuav@gmail.com> writes:
>> Ah but it isn't Py3 that's all about being first - it's the latest
>> version of some third-party module.
> 
> You know, one of the attractions of Python used to be that it came with
> a powerful enough standard library that you didn't really need third
> party modules very often.

The std lib is *batteries* included. If you need a nuclear reactor, you turn 
to third-party frameworks and libraries like Twisted, Zope, numpy, PLY, etc. 
They would be a bad fit in the standard library. It has been said that the 
stdlib is the place good code goes to die.

It's precisely because the stdlib has much stronger backward compatibility 
requirements and a higher reluctance to break things that fast-changing 
projects (including just bug fixes, not just new features) cannot go into 
the stdlib.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#87532

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
> The std lib is *batteries* included. If you need a nuclear reactor, you turn 
> to third-party frameworks and libraries like Twisted, Zope, numpy, PLY, etc. 

I always thought twisted and zope were monstrosities.  I've used threads
instead of twisted and various other possibilities instead of zope.  Not
sure why numpy couldn't go in the stdlib: does it change all that fast?
PLY is an application not a library.  

> It's precisely because the stdlib has much stronger backward compatibility 
> requirements and a higher reluctance to break things that fast-changing 
> projects (including just bug fixes, not just new features) cannot go into 
> the stdlib.

Has that been a problem for PHP?

[toc] | [prev] | [next] | [standalone]

#87551

On Mon, 16 Mar 2015 07:31 pm, Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>> The std lib is *batteries* included. If you need a nuclear reactor, you
>> turn to third-party frameworks and libraries like Twisted, Zope, numpy,
>> PLY, etc.
> 
> I always thought twisted and zope were monstrosities.  I've used threads
> instead of twisted and various other possibilities instead of zope.  Not
> sure why numpy couldn't go in the stdlib: does it change all that fast?
> PLY is an application not a library.

I don't know whether numpy changes *fast* or not, but its release cycle is
nothing like CPython's release cycle and it would be impractical to
syncronise the two. numpy is also awfully specialised, with a bunch of
dependencies like Fortran compilers and BLAS.

As for PLY, I'm sorry that was a typo I actually was thinking of PIL.


>> It's precisely because the stdlib has much stronger backward
>> compatibility requirements and a higher reluctance to break things that
>> fast-changing projects (including just bug fixes, not just new features)
>> cannot go into the stdlib.
> 
> Has that been a problem for PHP?

You surely aren't looking at PHP for best practices are you? PHP doesn't
just break backwards compatibility on major updates like 4 to 5, but on
minor updates like 5.2 to 5.3:

http://php.net/manual/en/migration53.incompatible.php

Of course, "problem" seems to be relative. PHP culture seems to accept that
code will break when you do a minor upgrade, and it's no big deal to do a
search-and-replace and rename your functions. The same thing would be
considered unacceptable in Python and unthinkable in C.

http://stackoverflow.com/questions/4693575/is-php-5-3-backwards-compatible-with-php-5-2



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87556

On Tue, Mar 17, 2015 at 12:05 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Of course, "problem" seems to be relative. PHP culture seems to accept that
> code will break when you do a minor upgrade, and it's no big deal to do a
> search-and-replace and rename your functions. The same thing would be
> considered unacceptable in Python and unthinkable in C.

Because, after all, it's no problem to keep a dozen PHP versions
around, and to select which one you want via a shebang at the top of
the file... oh wait. So how do you cope with breakage? Massive code
churn? What about those big PHP applications/engines/frameworks like
MediaWiki, WordPress, etc - do they maintain multiple versions that
are compatible with different PHP versions??

No, don't answer any of those questions. We don't need to discuss this
in detail. *sigh*

ChrisA

[toc] | [prev] | [next] | [standalone]

#87579

On 3/16/2015 4:31 AM, Paul Rubin wrote:

> sure why numpy couldn't go in the stdlib: does it change all that fast?

First there was Numerical Python, the first killer app (though a 
library) for Python.  Then there was was NumArray by a competing group, 
with some not-quite forward compatible changes.  This somewhat split 
scientific Python computing.  This was not good.  Then some some people 
(successfully) reunified things with numpy, which drew on both 
numberical_python and numarray.  I guess things are pretty stable now, 
but not static.

In any case, the developer groups are separate, and both codebases are 
pretty large.  Having to compile both to work on either would be a nuisance.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#87524

On Monday 16 March 2015 16:39, Chris Angelico wrote:

> On Mon, Mar 16, 2015 at 4:07 PM, Paul Rubin <no.email@nospam.invalid>
> wrote:
>> Python 2 is by now pretty solid and its users don't feel like beta
>> testers any more.  If you're saying using Python 3 by contrast means
>> "being first" and "reporting bugs", that basically translates to "stay
>> away from it except for experimentation".
> 
> Ah but it isn't Py3 that's all about being first - it's the latest
> version of some third-party module. This entire discussion came about
> because of non-stdlib modules - Python itself is quite mature. You
> might be the first to use XYZ with Py3, but you're not the first to
> use XYZ, nor the first to use Py3.

I think that's a distinction that makes no difference. Somebody has to be 
the first to migrate a large project to Python3+XYZ. The fact that others 
have successfully used Python2+XYZ or Python3+UVW or PHP+ABC is no comfort.

Let's not look at this through rose-tinted glasses. Big software projects 
are complicated, complex and even convoluted and major changes to libraries, 
languages or operating systems can break them. John ran into that, and bless 
him for reporting the bugs he found instead of keeping them secret so the 
next guy will run into them too.

It may, or may not, turn out that in hindsight there might have been better 
ways to manage the Python2-3 transaction. Let's be honest, a lot of the 
changes could have been introduced incrementally: with the possibly 
exception of the Unicode string business, I don't think anything *needed* a 
"flag day" cutover, they could have been handled by an incremental 
transition, e.g.:

2.5 add dict.viewkeys
2.6 noisily deprecate dict.keys and dict.iterkeys
2.7 remove dict.keys and dict.iterkeys and alias viewkeys to keys
2.8 deprecate viewkeys
2.9 remove viewkeys alias

Module renames could be handled via stub modules. Even Unicode strings could 
hypothetically have been added via a __future__ import.

BUT... would that actually have improved matters? I doubt it. It would have 
drawn the pain out much longer: every single new release will break 
*something*, and library authors will likely have a much harder job of 
supporting multiple versions.

I would have meant that users would be stuck with long periods of annoying 
deprecation warnings that they can't do anything about, and exponentially 
increased the pressure on the core devs to hold off removing deprecated 
features. What would have happened in practice probably would have been:

2.5 add dict.viewkeys
2.6 noisily deprecate dict.keys and dict.iterkeys
2.7 change deprecation to silent by default

and there it would stay, forever.

I suspect that without the discipline and bloody-mindedness to just say NO 
to requests to delay removal, Python would follow all those other languages 
that have an ever-increasing pile of cruft that can never be removed because 
it will break somebody's code. Design mistakes, obsolete features, 
misspellings, "it's like that for historical reasons", and other nasties.

Let's be frank: the average programmer has the aesthetic sense of a 
hyperactive weasel on LSD and wouldn't know a cleanly designed language if 
it fell from the sky and hit them on the head. Hence the popularity of PHP, 
Perl and C++. And the pointy-haired bosses holding the budget have even 
less, and zero care factor. "What do you mean the Python update broke our 
application *again*?"

With the Python3000 strategy, most user-space application developers can put 
off the pain until they are ready to deal with it, or put it off forever if 
they don't mind lacking an upgrade path beyond a certain point. Sometimes 
applications are just *finished*, there are no more features that need 
adding, no bugs that need fixing, and the only security feature you need is 
to lock the doors to the office when you leave at night. Those people have 
it easy: they can just get off the treadmill and ignore Python 3.x 
(2.7/2.6/2.5/2.4 ...) forever.



-- 
Steve

[toc] | [prev] | [next] | [standalone]

#87526

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
> It may, or may not, turn out that in hindsight there might have been better 
> ways to manage the Python2-3 transaction. Let's be honest, a lot of the 
> changes could have been introduced incrementally...

Why did the changes have to be introduced at all?

> I suspect that without the discipline and bloody-mindedness to just say NO 
> to requests to delay removal, Python would follow all those other languages 
> that have an ever-increasing pile of cruft that can never be removed because 
> it will break somebody's code. Design mistakes, obsolete features, 
> misspellings, "it's like that for historical reasons", and other nasties.

The idea of "maturity" (an advertised feature of Python 2 back in the
day) is that the pile of features is no longer growing so fast.  Right
now I'm messing with some C code written in the 1980s and it still works
fine.  I have a hard time imagining using Python 3 for anything.  Python
2 has its cruft but it's familiar and comfortable, and Python 3 gets
minimal benefit for most of the breakage that it did (possible exception
of the Unicode changes).  It still has most of the same mistakes and
warts.  If I wanted to get away from those, I'd use Go or Ocaml or
Erlang or Haskell or Javascript (possibly through a compiler like
Purescript) or whatever.

I'll see if I can remember what it was but I found some annoying
incompatibility just a week or so ago that I don't think 2to3 would have
caught, making me scared of Python 3 all over again.  I still think
CPython should have stayed with Python 2 forever, and Python 3 should
have been PyPy-based, letting PyPy drive its design, so that it could
have introduced bigger changes with bigger benefits.  As we're seeing,
even small incompatibilities and breakage cause big resistance, so if
you're going to break stuff at all, you might as well go big.  Most old
programs will never be ported either way.

[toc] | [prev] | [next] | [standalone]

#87540

google-api-client has ported from PY2 to PY2/3 recently.
https://github.com/google/google-api-python-client

It has 4000 lines of code and 3200 lines of tests.  It is legacy
library. It is indented by 2 spaces
since it was created before Google changes their coding style.

There are two major pull request: automatic change by modernize and
heuristic changes.
https://github.com/google/google-api-python-client/pull/62
https://github.com/google/google-api-python-client/pull/64

I feel this is reasonable effort, not bad.


Another experience is porting Flask application in my company from
Python 2 to Python 3.
It has 26k lines of code and 7.6k lines of tests.

Since we don't need to support both of PY2 and PY3, we used 2to3.
2to3 changes 740 lines. I had to replace google-api-client with
requests+oauthlib since
it had not supported PY3 yet.

After that, we encountered few trouble with untested code. But Porting
effort is surprisingly small.
We're happy now with Python 3.  We can write non-ascii string to log
without fear of UnicodeError.
We can use csv with unicode without hack.


Porting *modern* *application* code to *PY3 only* is easy, while
porting libraries on the edge of
bytes/unicode like google-api-client to PY2/3 is not easy.

I think application developers should use *only* Python 3 from this year.
If we start moving, more library developers will be able to start
writing Python 3 only code from next year.

On Mon, Mar 16, 2015 at 5:25 PM, Paul Rubin <no.email@nospam.invalid> wrote:
> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>> It may, or may not, turn out that in hindsight there might have been better
>> ways to manage the Python2-3 transaction. Let's be honest, a lot of the
>> changes could have been introduced incrementally...
>
> Why did the changes have to be introduced at all?
>
>> I suspect that without the discipline and bloody-mindedness to just say NO
>> to requests to delay removal, Python would follow all those other languages
>> that have an ever-increasing pile of cruft that can never be removed because
>> it will break somebody's code. Design mistakes, obsolete features,
>> misspellings, "it's like that for historical reasons", and other nasties.
>
> The idea of "maturity" (an advertised feature of Python 2 back in the
> day) is that the pile of features is no longer growing so fast.  Right
> now I'm messing with some C code written in the 1980s and it still works
> fine.  I have a hard time imagining using Python 3 for anything.  Python
> 2 has its cruft but it's familiar and comfortable, and Python 3 gets
> minimal benefit for most of the breakage that it did (possible exception
> of the Unicode changes).  It still has most of the same mistakes and
> warts.  If I wanted to get away from those, I'd use Go or Ocaml or
> Erlang or Haskell or Javascript (possibly through a compiler like
> Purescript) or whatever.
>
> I'll see if I can remember what it was but I found some annoying
> incompatibility just a week or so ago that I don't think 2to3 would have
> caught, making me scared of Python 3 all over again.  I still think
> CPython should have stayed with Python 2 forever, and Python 3 should
> have been PyPy-based, letting PyPy drive its design, so that it could
> have introduced bigger changes with bigger benefits.  As we're seeing,
> even small incompatibilities and breakage cause big resistance, so if
> you're going to break stuff at all, you might as well go big.  Most old
> programs will never be ported either way.
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
INADA Naoki  <songofacandy@gmail.com>

[toc] | [prev] | [next] | [standalone]

#87544

INADA Naoki wrote:

> I think application developers should use only Python 3 from this year.
> If we start moving, more library developers will be able to start
> writing Python 3 only code from next year.

Where this is possible, that is an excellent idea.

Alas, too many (Linux) developers insist on using the system Python as their
application language. In most cases they could trivially run "aptitude
install python3" and be done with it, but that's an extra dependency and
therefore Bad.

Red Hat, Debian and Ubuntu are all planning to move to Python 3 Real Soon
Now, and I expect that will give Py3 some more impetus.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87559

On Mon, 16 Mar 2015 07:25 pm, Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>> It may, or may not, turn out that in hindsight there might have been
>> better ways to manage the Python2-3 transaction. Let's be honest, a lot
>> of the changes could have been introduced incrementally...
> 
> Why did the changes have to be introduced at all?

For the same reason any improvement and functional update is introduced. To
make a better language.

If we were designing Python from scratch today, here are some of the changes
we would certainly make:


- only a single class system, not two independent ones (one of which 
  is buggy);
- Unicode strings by default;
- views and iterators in preference to lists;
- True and False as keywords, like None, not names that can be 
  redefined and shadowed;
- cleaner, more consistent, and more sensible naming conventions 
  for the standard library;
- Python is not C and / should perform true division, not integer
  division;
- some builtins of lesser use should be relegated to modules
- print as a statement was a mistake, it should be a function


etc. Python is a lesser language because of the accumulated cruft and design
mistakes from its 20-year history. If only we could turn back the clock and
redesign it with the benefit of hindsight, without worrying about backwards
compatibility!

Backwards compatibility limits your ability to improve the language and fix
mistakes and obsolete features. Whether you introduce those design fixes
all in one lump or spread out of twenty versions, you still have to break
backwards compatibility, or keep carrying the obsolete code forever.


>> I suspect that without the discipline and bloody-mindedness to just say
>> NO to requests to delay removal, Python would follow all those other
>> languages that have an ever-increasing pile of cruft that can never be
>> removed because it will break somebody's code. Design mistakes, obsolete
>> features, misspellings, "it's like that for historical reasons", and
>> other nasties.
> 
> The idea of "maturity" (an advertised feature of Python 2 back in the
> day) is that the pile of features is no longer growing so fast.  Right
> now I'm messing with some C code written in the 1980s and it still works
> fine.

Of course, not *all* C code written in the 1980s still works fine, but your
point is taken. C, like Fortran and Java, have an even stricter view of
backward compatibility than Python.

The downside of that is that learning and using C means you have to know and
learn a whole lot of ugly, dirty cruft, if only so you know what it means
when you see it in somebody else's code. Long after feature X has been made
obsolete, people still need to deal with it, *even if nobody uses it*.

And they will. We both know that no matter how bad and foolish a feature is,
no matter how loudly you tell people not to use this deprecated, insecure,
unsafe feature, until it is actually gone people will continue to write new
code using it.

You can't please everybody. Some people want Python to change even faster,
some want Python to stop changing so fast...


> I have a hard time imagining using Python 3 for anything.  Python 
> 2 has its cruft but it's familiar and comfortable, 

And that, I think, is the crux of the matter. Programmers, for all their
attraction to shiny new tech, are like cats in that they have a deep
distrust for anything too different. "Give me something completely new,
only exactly the same as the old one."


> and Python 3 gets 
> minimal benefit for most of the breakage that it did (possible exception
> of the Unicode changes).  It still has most of the same mistakes and
> warts.  If I wanted to get away from those, I'd use Go or Ocaml or
> Erlang or Haskell or Javascript (possibly through a compiler like
> Purescript) or whatever.

I find it hard to imagine anyone seriously preferring Javascript to Python
because Python has too many language warts. That's like somebody sitting
down to an entire Death By Chocolate cake for desert instead of a slice of
apple pie because apple pie has too many calories.

Nobody is stopping you from migrating all your code to Ocaml if you like.
(If you think migrating it from Python 2 to 3 is too hard, just try
migrating your code to another language.) And of course people *should*
learn new languages, that is a good thing.


> I'll see if I can remember what it was but I found some annoying
> incompatibility just a week or so ago that I don't think 2to3 would have
> caught, making me scared of Python 3 all over again.

If you remember, do tell.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87585

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>> Why did the changes have to be introduced at all?
> For the same reason any improvement and functional update is
> introduced. To make a better language.

There comes a point when you decide that maintaining existing code is
important enough that you have to stop breaking things.  That point is
called "maturity", and it's fine if you decide that your language
doesn't have it.  Haskell for a long time had a motto "avoid success at
all costs", i.e. it had an explicit goal of being a research testbed for
language geeks and its designers wanted to be able to change stuff
without alienating a big user community.  Python claimed

> If we were designing Python from scratch today, here are some of the
> changes we would certainly make: [mostly good changes]

I agree with most of those changes and I'd add some of my own that are
even more radical.  But, they shouldn't be made in dribs and drabs in
the production releases: it's better to make a fork of the language that
does all the changes at the same time.  Think of C++ as a fork of C in
that manner, although it suffered a lot from trying to be a superset.

> If only we could turn back the clock and redesign it with the benefit
> of hindsight, without worrying about backwards compatibility!

We are currently getting the disadvantages of both approaches.  We break
stuff enough to inflict pain on users, without bringing enough benefits
to make them want to accept the pain.

> design fixes all in one lump or spread out of twenty versions, you
> still have to break backwards compatibility, or keep carrying the
> obsolete code forever.

If people are still using it then it's not obsolete.  Code is only
obsolete when the last user is dead.

> Of course, not *all* C code written in the 1980s still works fine,

Can you give an example of some way C changed that broke old code that
was written in accordance with the language manuals?  Of course tons of
C code then and now used implementation-dependent hacks that went
outside the documented behavior, but that doesn't count.  

> but your point is taken. C, like Fortran and Java, have an even
> stricter view of backward compatibility than Python.

Right, they take their deployed base seriously, while Python has been
more like an experimental language by comparison.

> The downside of that is that learning and using C means you have to
> know and learn a whole lot of ugly, dirty cruft, if only so you know
> what it means when you see it in somebody else's code. Long after
> feature X has been made obsolete, people still need to deal with it,
> *even if nobody uses it*.

Can you give an example?  I wouldn't count things like gets, which
aren't as much changes in the language, as recognition that using it was
buggy from the start.

> And they will. We both know that no matter how bad and foolish a
> feature is, no matter how loudly you tell people not to use this
> deprecated, insecure, unsafe feature, until it is actually gone people
> will continue to write new code using it.

I haven't noticed that much in C.  Maybe it's more of an issue with C++.

> You can't please everybody. Some people want Python to change even
> faster, some want Python to stop changing so fast...

I don't notice anyone wanting Python to change faster, in the sense of
breaking existing code more often.  Maybe there are some specific
proposals that I missed.

> Programmers... are like cats in that they have a deep distrust for
> anything too different. "Give me something completely new, only
> exactly the same as the old one."

It's more like "don't break the old thing unless the new thing is so
much better that I want to change over anyway".  Python to Erlang or
Ocaml or Go or Haskell are far more drastic changes than Python 2 to
Python 3, but at the same time possibly more attractive.

> I find it hard to imagine anyone seriously preferring Javascript to
> Python because Python has too many language warts. 

There are only a few things about raw JS that are really warty and hard
to avoid, and Python has different but comparable warts.  There's
admittedly a bunch of crap in JS that shouldn't be used, but that's
relatively harmless as long as you ignore it.  Current JS
implementations are far superior to Python in performance and there's a
ton of JS library code out there, both for browsers and on the server
side, that might or might not have Python counterparts.  JS is also a
compilation target for other languages like Purescript, Ocaml, or even C
(emscripten).  I haven't tried Purescript yet but it looks cool, and it
might be the nerd answer to JS cruft.  It's Haskell-like but its data
and evaluation models translate more directly to JS than Haskell does.

> Nobody is stopping you from migrating all your code to Ocaml if you
> like.

We were talking about new projects, not migration.

Do you know if any of the big Python shops (Google maybe?) are using
Python 3 these days?  Have they migrated Python 2 codebases or are they
using both 2 and 3?  I don't personally know (i.e. in person, not
online) anyone using Python 3.  Everyone is still using Python 2.
E.g. I think Twitter was using Python 2 a year or so ago (don't know
about now).  I believe they still use Python for some things but have
gravitated production systems towards Scala.

[toc] | [prev] | [next] | [standalone]

#87594

On 3/16/2015 6:36 PM, Paul Rubin wrote:

> Do you know if any of the big Python shops (Google maybe?) are using
> Python 3 these days?

LibreOffice uses Python3.3 (or later, don't know) both for internal 
scripting and the Python bridge.  The FSR unicode that works everywhere 
for all codepoints had something to do with that.


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#87628

Le mardi 17 mars 2015 03:29:49 UTC+1, Terry Reedy a écrit :
> On 3/16/2015 6:36 PM, Paul Rubin wrote:
> 
> > Do you know if any of the big Python shops (Google maybe?) are using
> > Python 3 these days?
> 
> LibreOffice uses Python3.3 (or later, don't know) both for internal 
> scripting and the Python bridge.  The FSR unicode that works everywhere 
> for all codepoints had something to do with that.
> 
> 

My poor Terry.

I'm recommending (and did recommend) Office Word
instead of LibreOffice Writer. Of course, for
users who wish to produce a little bit more
than a document.

Contrary to you, I *tested" LibreOffice Writer and one of
its internal scripting language.

jmf

[toc] | [prev] | [next] | [standalone]

#87633

On Tue, 17 Mar 2015 09:36 am, Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
[...]
>> If we were designing Python from scratch today, here are some of the
>> changes we would certainly make: [mostly good changes]
> 
> I agree with most of those changes and I'd add some of my own that are
> even more radical.  But, they shouldn't be made in dribs and drabs in
> the production releases: it's better to make a fork of the language that
> does all the changes at the same time.

You mean like Python 3?


> Think of C++ as a fork of C in 
> that manner, although it suffered a lot from trying to be a superset.

C++ is hardly a fork. It is a distinct C-inspired language, although it is
intended to be completely C-compatible.



>> If only we could turn back the clock and redesign it with the benefit
>> of hindsight, without worrying about backwards compatibility!
> 
> We are currently getting the disadvantages of both approaches.  We break
> stuff enough to inflict pain on users, without bringing enough benefits
> to make them want to accept the pain.
> 
>> design fixes all in one lump or spread out of twenty versions, you
>> still have to break backwards compatibility, or keep carrying the
>> obsolete code forever.
> 
> If people are still using it then it's not obsolete.  Code is only
> obsolete when the last user is dead.

That's one way of looking at it, but I don't think a good way. There are
still people using Python 1.5, some in production, some for
experimentation. That doesn't mean that 1.5 isn't obsolete, and it
certainly doesn't imply that the core developers have an obligation to
continue maintaining it.

(I'm one of the later, I keep a copy of 1.5 installed so I can check what
features were around back then and how they have changed.)


>> Of course, not *all* C code written in the 1980s still works fine,
> 
> Can you give an example of some way C changed that broke old code that
> was written in accordance with the language manuals?  Of course tons of
> C code then and now used implementation-dependent hacks that went
> outside the documented behavior, but that doesn't count.

Why shouldn't it count?

People who really take backwards compatibility seriously make even their
undocumented behaviour backwards compatible, sometimes to astonishing
degrees:

    The testers on the Windows team were going through various 
    popular applications, testing them to make sure they worked
    OK, but SimCity kept crashing. They reported this to the
    Windows developers, who disassembled SimCity, stepped through
    it in a debugger, found the bug, and added special code that
    checked if SimCity was running, and if it did, ran the memory
    allocator in a special mode in which you could still use
    memory after freeing it.

http://www.joelonsoftware.com/articles/APIWar.html


As for C, there are a few backwards incompatible changes over the years. One
of my favourites is that the += operator was originally spelled =+ copying
Ken Thompson's earlier language "B". That was changed in 1976.

http://cm.bell-labs.com/who/dmr/chist.html

Although perhaps it is unfair to include changes so far back in time, before
C was an ISO standard.

It is true that C has been remarkably backwards compatible, particularly
compared to the popular languages of the 1970s with their oodles of
proprietary extension, but it has changed. New keywords of course break
code that relied on those words not being keywords. So much of the language
is optional or implementation dependent or undefined that there is lots of
code which will compile but not necessarily do the same thing on two
different systems. As Dennis Richie says (link above):

    There are differing dialects of C—most noticeably, those described
    by the older K&R and the newer Standard C—but on the whole, C has
    remained freer of proprietary extensions than other languages.


Post-standardisation, here are a few other backwards incompatibilities:

- C99 introduced variable-length arrays, but C11 relegated it to an optional
feature which compilers are not required to support;

- C99 changed the behaviour of floating point operations;

- C11 finally removed `gets`.

There are probably others.

But the biggest problem is that C compilers vary (sometimes greatly) in what
subsets of the language they actually support. What works in compiler A may
not even compile on compiler B, let alone behave the same.

Nevertheless, I acknowledge that there is a continuum of "backwards
compatibility strictness", with explicitly unstable languages at one end,
PHP closer to that side, Python more in the middle, and ISO standardised
languages like Fortran and C at the other end.



>> but your point is taken. C, like Fortran and Java, have an even
>> stricter view of backward compatibility than Python.
> 
> Right, they take their deployed base seriously, while Python has been
> more like an experimental language by comparison.
> 
>> The downside of that is that learning and using C means you have to
>> know and learn a whole lot of ugly, dirty cruft, if only so you know
>> what it means when you see it in somebody else's code. Long after
>> feature X has been made obsolete, people still need to deal with it,
>> *even if nobody uses it*.
> 
> Can you give an example?  I wouldn't count things like gets, which
> aren't as much changes in the language, as recognition that using it was
> buggy from the start.

That's exactly the point. `gets` is dangerous and needs to die with extreme
prejudice, and is an extremely strong argument in favour of breaking
backwards compatibility. "Oops, I misspelled referer" is embarrassing but
ultimately harmless. Fixing that bug is a Nice To Have. But removal of
mistakes like `gets` is *critical*, and the failure of C to do so for so
long is why "written in C" is so often a synonym for "security holes up to
your wossname".

 
>> And they will. We both know that no matter how bad and foolish a
>> feature is, no matter how loudly you tell people not to use this
>> deprecated, insecure, unsafe feature, until it is actually gone people
>> will continue to write new code using it.
> 
> I haven't noticed that much in C.  Maybe it's more of an issue with C++.
> 
>> You can't please everybody. Some people want Python to change even
>> faster, some want Python to stop changing so fast...
> 
> I don't notice anyone wanting Python to change faster, in the sense of
> breaking existing code more often.  Maybe there are some specific
> proposals that I missed.

Are you on the python-ideas list?


> Do you know if any of the big Python shops (Google maybe?) are using
> Python 3 these days?  Have they migrated Python 2 codebases or are they
> using both 2 and 3?  I don't personally know (i.e. in person, not
> online) anyone using Python 3.  Everyone is still using Python 2.
> E.g. I think Twitter was using Python 2 a year or so ago (don't know
> about now).  I believe they still use Python for some things but have
> gravitated production systems towards Scala.

There are certainly people using 3, some of them have spoken up about it
here, but 2 is still far more popular. That's only to be expected: after
almost a decade of development, the Python 3 ecosystem is now in a fit
state that people should prefer it for new projects. It will be a long time
before Python 2 is completely gone (although perhaps not as long as some
people expect).


Anyone remember the big backwards incompatible changes made to Visual Basic?
How long did that take to settle down afterwards?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87636

Le mardi 17 mars 2015 12:27:18 UTC+1, Steven D'Aprano a écrit :

> Anyone remember the big backwards incompatible changes made to Visual Basic?

Yes, but

Contrary to LibreOffice + Python (I can show it), MS Office + VBA
works (and it is still working) in the Unicode world
(to my knowledge).

It seems to me logical, MS handles Unicode correctly!

jmf

[toc] | [prev] | [next] | [standalone]

#87638

On Tue, 17 Mar 2015 22:26:58 +1100, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:

>
>> 
>> Can you give an example?  I wouldn't count things like gets, which
>> aren't as much changes in the language, as recognition that using it was
>> buggy from the start.
>
>That's exactly the point. `gets` is dangerous and needs to die with extreme
>prejudice, and is an extremely strong argument in favour of breaking
>backwards compatibility.

Why? You don't know how to use gets without running into a buffer
overflow?

This is the kind of reasoning I don't understand, really. Creating
programming languages that are afraid of their programmers is why we
have been creating ever more weak and limited programming languages,
and why we have been breeding ever more weak and limited programmers.

Getting all worked up because a programming language offers a way for
you to shoot yourself in the foot seems to be the thing of the day.
Program X introduced a vulnerability, blame it on gets() not the
programmer. I'd rather see that programmer die, than gets() but that's
me, I guess...

gets() dying justifies nothing. There's no argument for it. Trying to
make c look more safe because you dicthed gets, is some sort of a very
bad joke. You might as well get rid of alloc and pretty much the whole
standard language, really. And yet, there goes backwards compatibility
down the toilet because programmers aren't men enough to so much as
look at gets() in the documentation without falling into a crying fit
and loosing it in front of the girls.

Don't use it. But let the damn thing stay. For pete's sake, is this
really what we are coming into? Even C?

Notice: Intelligent programmers not needed anymore. Programming is for
dumb people and justin bieber programming languages.

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | comp.lang.python

csiph-web