Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #62898 > unrolled thread

Blog "about python 3"

Back to article view | Back to comp.lang.python

Contents

  Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-30 19:41 +0000
    Re: Blog "about python 3" Steven D'Aprano <steve@pearwood.info> - 2013-12-30 20:49 +0000
      Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-30 21:29 +0000
      Re: Blog "about python 3" Ethan Furman <ethan@stoneleaf.us> - 2013-12-30 14:38 -0800
      Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2013-12-31 12:09 +1100
      Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-31 04:38 +0000
      Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2013-12-31 15:44 +1100
      Re: Blog "about python 3" Ethan Furman <ethan@stoneleaf.us> - 2013-12-30 20:33 -0800
      Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-31 04:59 +0000
      Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-31 08:22 +0000
        Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-31 20:53 +1100
          Re: Blog "about python 3" Antoine Pitrou <solipsis@pitrou.net> - 2013-12-31 14:13 +0000
            Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2013-12-31 10:41 -0500
              Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-01 02:54 +1100
              Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-31 15:55 +0000
              Re: Blog "about python 3" Robin Becker <robin@reportlab.com> - 2014-01-02 17:36 +0000
                Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-03 15:49 +1100
                  Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-03 04:01 -0500
                    Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-03 02:10 -0800
                      Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-03 21:24 +1100
                      Re: Blog "about python 3" Ethan Furman <ethan@stoneleaf.us> - 2014-01-03 08:56 -0800
                  Re: Blog "about python 3" Robin Becker <robin@reportlab.com> - 2014-01-03 12:28 +0000
                    Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-03 09:57 -0500
                      Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-04 02:32 +1100
                  Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-03 17:00 -0500
                  Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-04 04:04 +0000
                    Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-04 08:55 -0500
                      Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 01:17 +1100
                        Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-04 11:10 -0800
                          Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-04 17:46 -0500
                            Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-05 06:23 -0800
                              Re: Blog "about python 3" Ned Batchelder <ned@nedbatchelder.com> - 2014-01-05 10:20 -0500
                              Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-05 17:14 -0500
                                Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-07 05:34 -0800
                                  Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-07 09:54 -0500
                                  Re: Blog "about python 3" Tim Delaney <timothy.c.delaney@gmail.com> - 2014-01-08 09:38 +1100
                                  Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-07 19:02 -0500
                                    Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-08 01:59 -0800
                                      Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-08 14:26 -0500
                                  Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-08 20:04 +0000
                              Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-05 17:48 -0500
                          Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 10:28 +1100
                      Re: Blog "about python 3" Ned Batchelder <ned@nedbatchelder.com> - 2014-01-04 12:51 -0500
                      Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-05 13:27 +1100
                        Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 13:32 +1100
                        Re: Blog "about python 3" MRAB <python@mrabarnett.plus.com> - 2014-01-05 02:41 +0000
                        Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-04 22:20 -0500
                          Re: Blog "about python 3" Rustom Mody <rustompmody@gmail.com> - 2014-01-05 10:12 +0530
                            Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-05 00:11 -0500
                          Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-05 17:28 +1100
                            Re: Blog "about python 3" Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-01-05 14:05 -0500
                          Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 15:01 +1100
                            Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-05 11:34 -0500
                              Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 03:51 +1100
                                Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-05 12:09 -0500
                                Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-06 11:42 +1100
                              Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-05 17:56 -0500
                              Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 10:59 +1100
                              Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-06 12:23 +1100
                                Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 12:54 +1100
                                Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-06 05:53 +0000
                        Re: Blog "about python 3" Devin Jeanpierre <jeanpierreda@gmail.com> - 2014-01-05 00:00 -0800
                          Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-05 23:28 +1100
                            Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 23:48 +1100
                            Re: Blog "about python 3" Roy Smith <roy@panix.com> - 2014-01-05 11:10 -0500
                        Re: Blog "about python 3" Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-01-05 13:51 -0500
              Re: Blog "about python 3" David Hutto <dwightdhutto@gmail.com> - 2014-01-02 13:25 -0500
              Re: Blog "about python 3" Terry Reedy <tjreedy@udel.edu> - 2014-01-02 13:37 -0500
              Re: Blog "about python 3" Antoine Pitrou <solipsis@pitrou.net> - 2014-01-02 23:57 +0000
              Re: Blog "about python 3" Robin Becker <robin@reportlab.com> - 2014-01-03 10:32 +0000
              Re: Blog "about python 3" Robin Becker <robin@reportlab.com> - 2014-01-03 11:14 +0000
                Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-04 05:52 -0800
                  Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-05 13:41 +1100
                    Re: Blog "about python 3" Chris Angelico <rosuav@gmail.com> - 2014-01-05 13:54 +1100
                      Re: Blog "about python 3" wxjmfauth@gmail.com - 2014-01-05 02:39 -0800
              Re: Blog "about python 3" Robin Becker <robin@reportlab.com> - 2014-01-03 11:37 +0000
              Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-04 07:30 +0000
          Re: Blog "about python 3" Johannes Bauer <dfnsonfsduifb@gmx.de> - 2014-01-05 13:14 +0100
            Re: Blog "about python 3" Stefan Behnel <stefan_ml@behnel.de> - 2014-01-05 14:55 +0100
          Re: Blog "about python 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-05 13:10 +0000
      Re: Blog "about python 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-31 20:04 +1100
      Re: Blog "about python 3" Devin Jeanpierre <jeanpierreda@gmail.com> - 2013-12-30 20:25 -0800

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

#63272

On 06/01/2014 01:54, Chris Angelico wrote:
> On Mon, Jan 6, 2014 at 12:23 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> (However, to the extent that Amazon has gained monopoly power over the book
>> market, that reasoning may not apply. Amazon is not *technically* a
>> monopoly, but they are clearly well on the way to becoming one, at which
>> point the customer has no effective choice and the market is no longer
>> free.)
>
> They don't need a monopoly on the whole book market, just on specific
> books - which they did have, in the cited case. I actually asked the
> author (translator, really - it's a translation of "Alice in
> Wonderland") how he would prefer me to buy, as there are some who sell
> on Amazon and somewhere else. There was no alternative to Amazon, ergo
> no choice and the market was not free. Like so many things, one choice
> ("I want to buy Ailice's Anters in Ferlielann") mandates another
> ("Must buy through Amazon").
>
> I don't know what it cost Amazon to ship me two copies of a book, but
> still probably less than they got out of me, so they're still ahead.
> Even if they lost money on this particular deal, they're still way
> ahead because of all the people who decide it's not worth their time
> to spend an hour or so trying to get a replacement. So yep, this
> policy is serving Amazon fairly well.
>
> ChrisA
>

So much for my "You never know, we might even end up with a thread 
whereby the discussion is Python, the whole Python and nothing but the 
Python." :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#63175

On Sat, Jan 4, 2014 at 6:27 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Fast is never more important than correct. It's just that sometimes you
> might compromise a little (or a lot) on what counts as correct in order for
> some speed.

Is this statement even falsifiable? Can you conceive of a circumstance
where someone has traded correctness for speed, but where one couldn't
describe it that latter way? I can't. I think by definition you can
always describe it that way, you just make "what counts as
correctness" be "what the customer wants given the resources
available". The conventional definition, however, is "what the
customer wants, imagining that you have infinite resources". With just
a little redefinition that seems reasonable, you can be made never to
be wrong!


I avoid making unfalsifiable arguments that aren't explicitly labeled
as such. I try to reword them as, "I prefer to look at it as ..." --
it's less aggressive, which means people are more likely to really
listen to what you have to say. It also doesn't pretend to be an
argument when it isn't.

-- Devin

[toc] | [prev] | [next] | [standalone]

#63186

Devin Jeanpierre wrote:

> On Sat, Jan 4, 2014 at 6:27 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Fast is never more important than correct. It's just that sometimes you
>> might compromise a little (or a lot) on what counts as correct in order
>> for some speed.
> 
> Is this statement even falsifiable? Can you conceive of a circumstance
> where someone has traded correctness for speed, but where one couldn't
> describe it that latter way? I can't. 

Every time some programmer "optimises" a piece of code (or, more often,
*thinks* they have optimised it) which introduces bugs into the software,
that's a case where somebody has traded correctness for speed where my
statement doesn't apply. Sometimes the response to the subsequent bug
report is "will not fix", and a retroactive change in the software
requirements. ("Oh, did we say that indexing a string would return a
character? We meant it would return a character, so long as the string only
includes no Unicode characters in the astral planes.") Sometimes it is to
revert the optimisation or otherwise fix the bug.

I accept that there is sometimes a fine line here. I'm assuming that
software applications have their requirements fully documented, which in
the real world is hardly ever the case. Although, even if the requirements
aren't always written down, often they are implicitly understood. (Although
it gets very interesting when the users' understanding and the developers'
understanding is different.)

Take as an example this "torture test" for a mathematical sum function,
where the built-in sum() gets the wrong answer but math.fsum() gets it
right:

py> from math import fsum
py> values = [1e12, 0.0001, -1e12, 0.0001]*10000
py> fsum(values)
2.0
py> sum(values)
2.4413841796875


Here's another example of the same thing, just to prove it's not a fluke:

py> values = [1e17, 1, 1, -1e17]
py> fsum(values)
2.0
py> sum(values)
0.0


The reason for the different results is that fsum() tries hard to account
for intermediate rounding errors and sum() does not. If you benchmark the
two functions, you'll find that sum() is significantly faster than fsum. So
the question to be asked is, does sum() promise to calculate floating point
sums accurately? If so, then this is a bug, probably introduced by the
desire for speed. But in fact, sum() does not promise to calculate floating
point sums accurately. What it promises to do is to calculate the
equivalent of a + b + c + ... for as many values as given, and that's
exactly what it does. Conveniently, that's faster than fsum(), and usually
accurate enough for most uses.

Is sum() buggy? No, of course not. It does what it promises, it's just that
what it promises to do falls short of "calculate floating point summations
to high accuracy".

Now, here's something which *would* be a bug, if sum() did it:

class MyInt(int):
    def __add__(self, other):
        return MyInt(super(MyInt, self).__add__(other))
    def __radd__(self, other):
        return MyInt(super(MyInt, self).__radd__(other))
    def __repr__(self):
        return "MyInt(%d)" % self


Adding a zero MyInt to an int gives a MyInt:

py> MyInt(0) + 23
MyInt(23)

so sum() should do the same thing. If it didn't, if it optimised away the
actual addition because "adding zero to a number can't change anything", it
would be buggy. But in fact, sum() does the right thing:

py> sum([MyInt(0), 23])
MyInt(23)


> I think by definition you can 
> always describe it that way, you just make "what counts as
> correctness" be "what the customer wants given the resources
> available". 

Not quite. "Correct" means "does what the customer wants". Or if there is no
customer, it's "does what you say it will do".

How do we tell when software is buggy? We compare what it actually does to
the promised behaviour, or expected behaviour, and if there is a
discrepancy, we call it a bug. We don't compare it to some ideal that
cannot be met. A bug report that math.pi does not have infinite number of
decimal places would be closed as "Will Not Fix".

Likewise, if your customer pays you to solve the Travelling Salesman Problem
exactly, even if it takes a week to calculate, then nothing short of a
program that solves the Travelling Salesman Problem exactly will satisfy
their requirements. It's no good telling the customer that you can
calculate a non-optimal answer twenty times faster if they want the actual
optimal answer.

(Of course, you may try to persuade them that they don't really need the
optimal solution, or that they cannot afford it, or that you cannot deliver
and they need to compromise.)


> The conventional definition, however, is "what the 
> customer wants, imagining that you have infinite resources".

I don't think the resources really come into it. At least, certainly not
*infinite* resources. fsum() doesn't require infinite resources to
calculate floating point summations to high accuracy. An even more accurate
(but even slower) version would convert each float into a Fraction, then
add the Fractions.


> With just 
> a little redefinition that seems reasonable, you can be made never to
> be wrong!

I'm never wrong because I'm always right! *wink*

Let's bring this back to the claim made at the beginning. Someone (Mark?)
made a facetious comment about preferring fast code to correct code.
Someone else (I forget who, and am too lazy to look it up -- Roy Smith
perhaps?) suggested that we accept incorrect code if it is fast quite
often. But I maintain that we don't. If we did, we'd explicitly say:

"Sure, I know this program calculates the wrong answer, but gosh look how
fast it is!"

much like a anecdote I gave about the roadie driving in the wrong direction
who stated "Who cares, we're making great time!".

I maintain that people don't as a rule justify incorrect code on the basis
of it being fast. They claim the code isn't incorrect, that any compromises
made are deliberate and not bugs:

- "sum() doesn't promise to calculate floats to high accuracy, it 
  promises to give the same answer as if you repeatedly added them 
  with the + operator."

- "We never promised 100% uptime, we promised four nines uptime."

- "Our anti-virus scanner is blindingly fast, while still identifying
  at least 99% of all known computer viruses!"

- "The Unix 'locate' command doesn't do a live search of the file 
  system because that would be too slow, it uses a snapshot of the
  state of the file system."


Is locate buggy because it tells you what files existed the last time the
updatedb command ran, instead of what files exist right now? No, of course
not. locate does exactly what it promises to do.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#63187

On Sun, Jan 5, 2014 at 11:28 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> - "The Unix 'locate' command doesn't do a live search of the file
>   system because that would be too slow, it uses a snapshot of the
>   state of the file system."
>
>
> Is locate buggy because it tells you what files existed the last time the
> updatedb command ran, instead of what files exist right now? No, of course
> not. locate does exactly what it promises to do.

Even more strongly: We say colloquially that Google, DuckDuckGo, etc,
etc, are tools for "searching the web". But they're not. They're tools
for *indexing* the World Wide Web, and then searching that index. It's
plausible to actually search your file system (and there are times
when you want that), but completely implausible to search the (F or
otherwise) web. We accept the delayed appearance of a page in the
search results because we want immediate results, no waiting a month
to find anything! So the difference between what's technically
promised and what's colloquially described may be more than just
concealing bugs - it may be the vital difference between uselessness
and usefulness. And yet we like the handwave.

ChrisA

[toc] | [prev] | [next] | [standalone]

#63197

In article <52c94fec$0$29973$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> How do we tell when software is buggy? We compare what it actually does to
> the promised behaviour, or expected behaviour, and if there is a
> discrepancy, we call it a bug. We don't compare it to some ideal that
> cannot be met. A bug report that math.pi does not have infinite number of
> decimal places would be closed as "Will Not Fix".

That's because it is inherently impossible to "fix" that.  But lots of 
bug reports legitimately get closed with "Will Not Fix" simply because 
the added value from fixing it doesn't justify the cost (whether in 
terms of development effort, or run-time resource consumption).

Go back to the package sorting example I gave.  If the sorting software 
mis-reads the address and sends my package to Newark instead of New York 
by mistake, that's clearly a bug.

Presumably, it's an error which could be eliminated (or, at least, the 
rate of occurrence reduced) by using a more sophisticated OCR algorithm.  
But, if those algorithms take longer to run, the overall expected value 
of implementing the bug fix software may well be negative.

In the real world, nobody cares if software is buggy.  They care that it 
provides value.

[toc] | [prev] | [next] | [standalone]

#63211

On Sun, 05 Jan 2014 13:27:13 +1100, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> declaimed the following:

>
>I know somebody who was once touring in the States, and ended up travelling
>cross-country by road with the roadies rather than flying. She tells me of
>the time someone pointed out that they were travelling in the wrong
>direction, away from their destination. The roadie driving replied "Who
>cares? We're making fantastic time!"
>

	At least it wasn't a neophyte to the Panama Canal... Where the Atlantic
end is to the west of the Pacific end.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#62994

[Multipart message — attachments visible in raw view] — view raw

Just because it's 3.3 doesn't matter...the main interest is in
compatibility. Secondly, you used just one piece of code, which could be a
fluke, try others, and check the PEP. You need to realize that evebn the
older versions are benig worked on, and they have to be refined. So if you
have a problem, use the older and import from the future would be my
suggestion



On Thu, Jan 2, 2014 at 12:36 PM, Robin Becker <robin@reportlab.com> wrote:

> On 31/12/2013 15:41, Roy Smith wrote:
>
>> I'm using 2.7 in production.  I realize that at some point we'll need to
>> upgrade to 3.x.  We'll keep putting that off as long as the "effort +
>> dependencies + risk" metric exceeds the "perceived added value" metric.
>>
>>  We too are using python 2.4 - 2.7 in production. Different clients
> migrate at different speeds.
>
>
>> To be honest, the "perceived added value" in 3.x is pretty low for us.
>> What we're running now works.  Switching to 3.x isn't going to increase
>> our monthly average users, or our retention rate, or decrease our COGS,
>> or increase our revenue.  There's no killer features we need.  In
>> summary, the decision to migrate will be driven more by risk aversion,
>> when the risk of staying on an obsolete, unsupported platform, exceeds
>> the risk of moving to a new one.  Or, there will be some third-party
>> module that we must have which is no longer supported on 2.x.
>>
>>
> +1
>
>  If I were starting a new project today, I would probably start it in 3.x.
>>
> +1
>
> I just spent a large amount of effort porting reportlab to a version which
> works with both python2.7 and python3.3. I have a large number of functions
> etc which handle the conversions that differ between the two pythons.
>
> For fairly sensible reasons we changed the internal default to use unicode
> rather than bytes. After doing all that and making the tests compatible etc
> etc I have a version which runs in both and passes all its tests. However,
> for whatever reason the python 3.3 version runs slower
>
> 2.7 Ran 223 tests in 66.578s
>
> 3.3 Ran 223 tests in 75.703s
>
> I know some of these tests are fairly variable, but even for simple things
> like paragraph parsing 3.3 seems to be slower. Since both use unicode
> internally it can't be that can it, or is python 2.7's unicode faster?
>
> So far the superiority of 3.3 escapes me, but I'm tasked with enjoying
> this process so I'm sure there must be some new 'feature' that will help.
> Perhaps 'yield from' or 'raise from None' or .......
>
> In any case I think we will be maintaining python 2.x code for at least
> another 5 years; the version gap is then a real hindrance.
> --
> Robin Becker
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com <http://www.hitwebdevelopment.com>*

[toc] | [prev] | [next] | [standalone]

#62996

On 1/2/2014 12:36 PM, Robin Becker wrote:

> I just spent a large amount of effort porting reportlab to a version
> which works with both python2.7 and python3.3. I have a large number of
> functions etc which handle the conversions that differ between the two
> pythons.

I am imagine that this was not fun.

[For those who do not know, reportlab produces pdf documents.]

> For fairly sensible reasons we changed the internal default to use
> unicode rather than bytes.

Do you mean 'from __future__ import unicode_literals'?

Am I correct in thinking that this change increases the capabilities of 
reportlab? For instance, easily producing an article with abstracts in 
English, Arabic, Russian, and Chinese?

 > After doing all that and making the tests
> compatible etc etc I have a version which runs in both and passes all
> its tests. However, for whatever reason the python 3.3 version runs slower.

Python 3 is slower in some things, like integer arithmetic with small ints.

> 2.7 Ran 223 tests in 66.578s
>
> 3.3 Ran 223 tests in 75.703s
>
> I know some of these tests are fairly variable, but even for simple
> things like paragraph parsing 3.3 seems to be slower. Since both use
> unicode internally it can't be that can it, or is python 2.7's unicode
> faster?

The new unicode implementation in 3.3 is faster for some operations and 
slower for others. It is definitely more space efficient, especially 
compared to a wide build system. It is definitely less buggy, especially 
compared to a narrow build system.

Do your tests use any astral (non-BMP) chars? If so, do they pass on 
narrow 2.7 builds (like on Windows)?

> So far the superiority of 3.3 escapes me,

For one thing, indexing and slicing just works on all machines for all 
unicode strings. Code for 2.7 and 3.3 either a) does not index or slice, 
b) does not work for all text on 2.7 narrow builds, or c) has extra 
conditional code only for 2.7.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#63010

Hi,

Robin Becker <robin <at> reportlab.com> writes:
> 
> For fairly sensible reasons we changed the internal default to use unicode 
> rather than bytes. After doing all that and making the tests compatible
etc etc 
> I have a version which runs in both and passes all its tests. However, for 
> whatever reason the python 3.3 version runs slower
> 
> 2.7 Ran 223 tests in 66.578s
> 
> 3.3 Ran 223 tests in 75.703s

Running a test suite is a completely broken benchmarking methodology.
You should isolate workloads you are interested in and write a benchmark
simulating them.

Regards

Antoine.

[toc] | [prev] | [next] | [standalone]

#63049

On 02/01/2014 18:25, David Hutto wrote:
> Just because it's 3.3 doesn't matter...the main interest is in
> compatibility. Secondly, you used just one piece of code, which could be a
> fluke, try others, and check the PEP. You need to realize that evebn the
> older versions are benig worked on, and they have to be refined. So if you
> have a problem, use the older and import from the future would be my
> suggestion

Suggesting that I use another piece of code to test python3 against python2 is a 
bit silly. I'm sure I can find stuff which runs faster under python3, but 
reportlab is the code I'm porting and that is going the wrong way.
-- 
Robin Becker

[toc] | [prev] | [next] | [standalone]

#63051

On 02/01/2014 18:37, Terry Reedy wrote:
> On 1/2/2014 12:36 PM, Robin Becker wrote:
>
>> I just spent a large amount of effort porting reportlab to a version
>> which works with both python2.7 and python3.3. I have a large number of
>> functions etc which handle the conversions that differ between the two
>> pythons.
>
> I am imagine that this was not fun.

indeed :)
>
>> For fairly sensible reasons we changed the internal default to use
>> unicode rather than bytes.
>
> Do you mean 'from __future__ import unicode_literals'?

No, previously we had default of utf8 encoded strings in the lower levels of the 
code and we accepted either unicode or utf8 string literals as inputs to text 
functions. As part of the port process we made the decision to change from 
default utf8 str (bytes) to default unicode.

> Am I correct in thinking that this change increases the capabilities of
> reportlab? For instance, easily producing an article with abstracts in English,
> Arabic, Russian, and Chinese?
>
It's made no real difference to what we are able to produce or accept since utf8 
or unicode can encode anything in the input and what can be produced depends on 
fonts mainly.

>  > After doing all that and making the tests
...........
>> I know some of these tests are fairly variable, but even for simple
>> things like paragraph parsing 3.3 seems to be slower. Since both use
>> unicode internally it can't be that can it, or is python 2.7's unicode
>> faster?
>
> The new unicode implementation in 3.3 is faster for some operations and slower
> for others. It is definitely more space efficient, especially compared to a wide
> build system. It is definitely less buggy, especially compared to a narrow build
> system.
>
> Do your tests use any astral (non-BMP) chars? If so, do they pass on narrow 2.7
> builds (like on Windows)?

I'm not sure if we have any non-bmp characters in the tests. Simple CJK etc etc 
for the most part. I'm fairly certain we don't have any ability to handle 
composed glyphs (multi-codepoint) etc etc



....
> For one thing, indexing and slicing just works on all machines for all unicode
> strings. Code for 2.7 and 3.3 either a) does not index or slice, b) does not
> work for all text on 2.7 narrow builds, or c) has extra conditional code only
> for 2.7.
>

probably
-- 
Robin Becker

[toc] | [prev] | [next] | [standalone]

#63133

Le vendredi 3 janvier 2014 12:14:41 UTC+1, Robin Becker a écrit :
> On 02/01/2014 18:37, Terry Reedy wrote:
> 
> > On 1/2/2014 12:36 PM, Robin Becker wrote:
> 
> >
> 
> >> I just spent a large amount of effort porting reportlab to a version
> 
> >> which works with both python2.7 and python3.3. I have a large number of
> 
> >> functions etc which handle the conversions that differ between the two
> 
> >> pythons.
> 
> >
> 
> > I am imagine that this was not fun.
> 
> 
> 
> indeed :)
> 
> >
> 
> >> For fairly sensible reasons we changed the internal default to use
> 
> >> unicode rather than bytes.
> 
> >
> 
> > Do you mean 'from __future__ import unicode_literals'?
> 
> 
> 
> No, previously we had default of utf8 encoded strings in the lower levels of the 
> 
> code and we accepted either unicode or utf8 string literals as inputs to text 
> 
> functions. As part of the port process we made the decision to change from 
> 
> default utf8 str (bytes) to default unicode.
> 
> 
> 
> > Am I correct in thinking that this change increases the capabilities of
> 
> > reportlab? For instance, easily producing an article with abstracts in English,
> 
> > Arabic, Russian, and Chinese?
> 
> >
> 
> It's made no real difference to what we are able to produce or accept since utf8 
> 
> or unicode can encode anything in the input and what can be produced depends on 
> 
> fonts mainly.
> 
> 
> 
> >  > After doing all that and making the tests
> 
> ...........
> 
> >> I know some of these tests are fairly variable, but even for simple
> 
> >> things like paragraph parsing 3.3 seems to be slower. Since both use
> 
> >> unicode internally it can't be that can it, or is python 2.7's unicode
> 
> >> faster?
> 
> >
> 
> > The new unicode implementation in 3.3 is faster for some operations and slower
> 
> > for others. It is definitely more space efficient, especially compared to a wide
> 
> > build system. It is definitely less buggy, especially compared to a narrow build
> 
> > system.
> 
> >
> 
> > Do your tests use any astral (non-BMP) chars? If so, do they pass on narrow 2.7
> 
> > builds (like on Windows)?
> 
> 
> 
> I'm not sure if we have any non-bmp characters in the tests. Simple CJK etc etc 
> 
> for the most part. I'm fairly certain we don't have any ability to handle 
> 
> composed glyphs (multi-codepoint) etc etc
> 
> 
> 
> 
> 
> 
> 
> ....
> 
> > For one thing, indexing and slicing just works on all machines for all unicode
> 
> > strings. Code for 2.7 and 3.3 either a) does not index or slice, b) does not
> 
> > work for all text on 2.7 narrow builds, or c) has extra conditional code only
> 
> > for 2.7.

----

To Robin Becker

I know nothing about ReportLab except its existence.
Your story is very interesting. As I pointed, I know
nothing about the internal of ReportLab, the technical
aspects: the "Python part", "the used api for the PDF creation").
I have however some experience with the unicode TeX engine,
XeTeX, understand I'm understanding a little bit what's
happening behind the scene.

The very interesting aspect in the way you are holding
unicodes (strings). By comparing Python 2 with Python 3.3,
you are comparing utf-8 with the the internal "representation"
of Python 3.3 (the flexible string represenation).
In one sense, more than comparing Py2 with Py3.

It will be much more interesting to compare utf-8/Python
internals at the light of Python 3.2 and Python 3.3. Python
3.2 has a decent unicode handling, Python 3.3 has an absurd
(in mathematical sense) unicode handling. This is really
shining with utf-8, where this flexible string representation
is just doing the opposite of what a correct unicode
implementation does!

On the memory side, it is obvious to see it.

>>> sys.getsizeof('a'*10000 + 'z')
10026
>>> sys.getsizeof('a'*10000 + '€')
20040
>>> sys.getsizeof(('a'*10000 + 'z').encode('utf-8'))
10018
>>> sys.getsizeof(('a'*10000 + '€').encode('utf-8'))
10020

On the performance side, it is much more complexe,
but qualitatively, you may expect the same results.


The funny aspect is that by working with utf-8 in that
case, you are (or one has) forcing Python to work
properly, but one pays on the side of the performance.
And if one wishes to save memory, one has to pay on the
side of performance.

In othe words, attempting to do what Python is
not able to do natively is just impossible!


I'm skipping the very interesting composed glyphs subject
(unicode normalization, ...),  but I wish to point that
with the flexible string representation, one reaches
the top level of surrealism. For a tool which is supposed
to handle these very specific unicode tasks...

jmf

[toc] | [prev] | [next] | [standalone]

#63162

wxjmfauth@gmail.com wrote:

> The very interesting aspect in the way you are holding
> unicodes (strings). By comparing Python 2 with Python 3.3,
> you are comparing utf-8 with the the internal "representation"
> of Python 3.3 (the flexible string represenation).

This is incorrect. Python 2 has never used UTF-8 internally for Unicode
strings. In narrow builds, it uses UTF-16, but makes no allowance for
surrogate pairs in strings. In wide builds, it uses UTF-32.

Other implementations, such as Jython or IronPython, may do something else.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#63164

On Sun, Jan 5, 2014 at 1:41 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> wxjmfauth@gmail.com wrote:
>
>> The very interesting aspect in the way you are holding
>> unicodes (strings). By comparing Python 2 with Python 3.3,
>> you are comparing utf-8 with the the internal "representation"
>> of Python 3.3 (the flexible string represenation).
>
> This is incorrect. Python 2 has never used UTF-8 internally for Unicode
> strings. In narrow builds, it uses UTF-16, but makes no allowance for
> surrogate pairs in strings. In wide builds, it uses UTF-32.

That's for Python's unicode type. What Robin said was that they were
using either a byte string ("str") with UTF-8 data, or a Unicode
string ("unicode") with character data. So jmf was right, except that
it's not specifically to do with Py2 vs Py3.3.

ChrisA

[toc] | [prev] | [next] | [standalone]

#63184

Le dimanche 5 janvier 2014 03:54:29 UTC+1, Chris Angelico a écrit :
> On Sun, Jan 5, 2014 at 1:41 PM, Steven D'Aprano
> 
> <steve+comp.lang.python@pearwood.info> wrote:
> 
> > wxjmfauth@gmail.com wrote:
> 
> >
> 
> >> The very interesting aspect in the way you are holding
> 
> >> unicodes (strings). By comparing Python 2 with Python 3.3,
> 
> >> you are comparing utf-8 with the the internal "representation"
> 
> >> of Python 3.3 (the flexible string represenation).
> 
> >
> 
> > This is incorrect. Python 2 has never used UTF-8 internally for Unicode
> 
> > strings. In narrow builds, it uses UTF-16, but makes no allowance for
> 
> > surrogate pairs in strings. In wide builds, it uses UTF-32.
> 
> 
> 
> That's for Python's unicode type. What Robin said was that they were
> 
> using either a byte string ("str") with UTF-8 data, or a Unicode
> 
> string ("unicode") with character data. So jmf was right, except that
> 
> it's not specifically to do with Py2 vs Py3.3.
> 
> 

Yes, the key point is the preparation of the "unicode
text" for the PDF producer.

This is at this level the different flavours of Python
may be relevant.

I see four possibilites, I do not know what
the PDF producer API is expecting.

- Py2 with utf-8 byte string (ev. utf-16, utf-32)
- Py2 with its internal unicode
- Py3.2 with its internal unicode
- Py3.3 with its internal unicode

jmf

[toc] | [prev] | [next] | [standalone]

#63052

On 02/01/2014 23:57, Antoine Pitrou wrote:
>
..........
>
> Running a test suite is a completely broken benchmarking methodology.
> You should isolate workloads you are interested in and write a benchmark
> simulating them.
>

I'm certain you're right, but individual bits of code like generating our 
reference manual also appear to be slower in 3.3.

> Regards
>
> Antoine.
>
>


-- 
Robin Becker

[toc] | [prev] | [next] | [standalone]

#63115

On 02/01/2014 17:36, Robin Becker wrote:
> On 31/12/2013 15:41, Roy Smith wrote:
>> I'm using 2.7 in production.  I realize that at some point we'll need to
>> upgrade to 3.x.  We'll keep putting that off as long as the "effort +
>> dependencies + risk" metric exceeds the "perceived added value" metric.
>>
> We too are using python 2.4 - 2.7 in production. Different clients
> migrate at different speeds.
>
>>
>> To be honest, the "perceived added value" in 3.x is pretty low for us.
>> What we're running now works.  Switching to 3.x isn't going to increase
>> our monthly average users, or our retention rate, or decrease our COGS,
>> or increase our revenue.  There's no killer features we need.  In
>> summary, the decision to migrate will be driven more by risk aversion,
>> when the risk of staying on an obsolete, unsupported platform, exceeds
>> the risk of moving to a new one.  Or, there will be some third-party
>> module that we must have which is no longer supported on 2.x.
>>
>
> +1
>
>> If I were starting a new project today, I would probably start it in 3.x.
> +1
>
> I just spent a large amount of effort porting reportlab to a version
> which works with both python2.7 and python3.3. I have a large number of
> functions etc which handle the conversions that differ between the two
> pythons.
>
> For fairly sensible reasons we changed the internal default to use
> unicode rather than bytes. After doing all that and making the tests
> compatible etc etc I have a version which runs in both and passes all
> its tests. However, for whatever reason the python 3.3 version runs slower
>
> 2.7 Ran 223 tests in 66.578s
>
> 3.3 Ran 223 tests in 75.703s
>
> I know some of these tests are fairly variable, but even for simple
> things like paragraph parsing 3.3 seems to be slower. Since both use
> unicode internally it can't be that can it, or is python 2.7's unicode
> faster?
>
> So far the superiority of 3.3 escapes me, but I'm tasked with enjoying
> this process so I'm sure there must be some new 'feature' that will
> help. Perhaps 'yield from' or 'raise from None' or .......
>
> In any case I think we will be maintaining python 2.x code for at least
> another 5 years; the version gap is then a real hindrance.

Of interest 
https://mail.python.org/pipermail/python-dev/2012-October/121919.html ?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#63185

On 31.12.2013 10:53, Steven D'Aprano wrote:
> Mark Lawrence wrote:
> 
>> http://blog.startifact.com/posts/alex-gaynor-on-python-3.html.
> 
> I quote:
> 
> "...perhaps a brave group of volunteers will stand up and fork Python 2, and
> take the incremental steps forward. This will have to remain just an idle
> suggestion, as I'm not volunteering myself."
> 
> I expect that as excuses for not migrating get fewer, and the deadline for
> Python 2.7 end-of-life starts to loom closer, more and more haters^W
> Concerned People will whine about the lack of version 2.8 and ask for
> *somebody else* to fork Python.
> 
> I find it, hmmm, interesting, that so many of these Concerned People who say
> that they're worried about splitting the Python community[1] end up
> suggesting that we *split the community* into those who have moved forward
> to Python 3 and those who won't.

Exactly. I don't know what exactly their problem is. I've pushed the
migration of *large* projects at work to Python3 when support was pretty
early and it really wasn't a huge deal.

Specifically because I love pretty much every single aspect that Python3
introduced. The codec support is so good that I've never seen anything
like it in any other programming language and then there's the tons of
beautiful changes (div/intdiv, functools.lru_cache, print(),
datetime.timedelta.total_seconds(), int.bit_length(), bytes/bytearray).

Regards,
Joe

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#63193

Johannes Bauer, 05.01.2014 13:14:
> I've pushed the
> migration of *large* projects at work to Python3 when support was pretty
> early and it really wasn't a huge deal.

I think there are two sides to consider. Those who can switch their code
base to Py3 and be happy (as you did, apparently), and those who cannot
make the switch but have to keep supporting Py2 until 'everyone' else has
switched, too. The latter is a bit more work generally and applies mostly
to Python packages on PyPI, i.e. application dependencies.

There are two ways to approach that problem. One is to try convincing
people that "Py3 has failed, let's stop migrating more code before I have
to start migrating mine", and the other is to say "let's finish the
migration and get it done, so that we can finally drop Py2 support in our
new releases and clean up our code again".

As long as we stick in the middle and keep the status quo, we keep the
worst of both worlds. And, IMHO, pushing loudly for a Py2.8 release
provides a very good excuse for others to not finish their part of the
migration, thus prolonging the maintenance burden for those who already did
their share.

Maybe a couple of major projects should start dropping their Py2 support,
just to make their own life easier and to help others in taking their
decision, too.

(And that's me saying that, who maintains two major projects that still
have legacy support for Py2.4 ...)

Stefan

[toc] | [prev] | [next] | [standalone]

#63188

On 31/12/2013 09:53, Steven D'Aprano wrote:
> Mark Lawrence wrote:
>
>> http://blog.startifact.com/posts/alex-gaynor-on-python-3.html.
>
> I quote:
>
> "...perhaps a brave group of volunteers will stand up and fork Python 2, and
> take the incremental steps forward. This will have to remain just an idle
> suggestion, as I'm not volunteering myself."
>
> I expect that as excuses for not migrating get fewer, and the deadline for
> Python 2.7 end-of-life starts to loom closer, more and more haters^W
> Concerned People will whine about the lack of version 2.8 and ask for
> *somebody else* to fork Python.
>

Should the "somebody else" fork Python, in ten (ish) years time the 
Concerned People will be complaining that they can't port their code to 
Python 4 and will "somebody else" please produce version 2.9.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

Back to top | Article view | comp.lang.python

csiph-web