Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #106266 > unrolled thread

[beginner] What's wrong?

Back to article view | Back to comp.lang.python

Contents

  [beginner] What's wrong? Michael Okuntsov <okuntsov.mikhail@yandex.ru> - 2016-04-02 03:48 +0600
    Re: [beginner] What's wrong? Michael Okuntsov <okuntsov.mikhail@yandex.ru> - 2016-04-02 04:10 +0600
      Re: [beginner] What's wrong? sohcahtoa82@gmail.com - 2016-04-01 15:44 -0700
        Re: [beginner] What's wrong? Random832 <random832@fastmail.com> - 2016-04-02 00:27 -0400
        Re: [beginner] What's wrong? Michael Selik <michael.selik@gmail.com> - 2016-04-02 05:36 +0000
        Re: [beginner] What's wrong? William Ray Wing <wrw@mac.com> - 2016-04-02 00:54 -0400
        Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-02 19:15 +1100
        Re: [beginner] What's wrong? Michael Selik <michael.selik@gmail.com> - 2016-04-02 14:48 +0000
        Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-03 01:55 +1100
          Re: [beginner] What's wrong? Marko Rauhamaa <marko@pacujo.net> - 2016-04-02 18:07 +0300
            Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-03 02:36 +1100
            Re: [beginner] What's wrong? Steven D'Aprano <steve@pearwood.info> - 2016-04-03 02:06 +1000
              Re: [beginner] What's wrong? Marko Rauhamaa <marko@pacujo.net> - 2016-04-02 19:44 +0300
                Re: [beginner] What's wrong? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-04-02 19:12 +0200
                  Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-02 10:28 -0700
                    Re: [beginner] What's wrong? Marko Rauhamaa <marko@pacujo.net> - 2016-04-02 21:43 +0300
                    Re: [beginner] What's wrong? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-04-03 13:47 +0200
                      Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 07:30 -0700
                        Re: [beginner] What's wrong? Dan Sommers <dan@tombstonezero.net> - 2016-04-03 15:25 +0000
                          Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 08:39 -0700
                            Re: [beginner] What's wrong? Dan Sommers <dan@tombstonezero.net> - 2016-04-03 16:22 +0000
                              Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-04 02:44 +1000
                              Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 10:18 -0700
                                Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-04 03:35 +1000
                                Re: [beginner] What's wrong? Dan Sommers <dan@tombstonezero.net> - 2016-04-03 18:26 +0000
                          Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 08:46 -0700
                            Re: [beginner] What's wrong? Larry Martell <larry.martell@gmail.com> - 2016-04-03 11:55 -0400
                            Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-04 01:53 +1000
                              Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 09:49 -0700
                                Re: [beginner] What's wrong? Dan Sommers <dan@tombstonezero.net> - 2016-04-03 18:32 +0000
                            Re: [beginner] What's wrong? Dan Sommers <dan@tombstonezero.net> - 2016-04-03 16:07 +0000
                        Re: [beginner] What's wrong? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-04-06 21:56 +0200
                          Unicode normalisation [was Re: [beginner] What's wrong?] Steven D'Aprano <steve@pearwood.info> - 2016-04-07 11:37 +1000
                            Re: Unicode normalisation [was Re: [beginner] What's wrong?] Marko Rauhamaa <marko@pacujo.net> - 2016-04-07 09:36 +0300
                            Re: Unicode normalisation [was Re: [beginner] What's wrong?] Peter Pearson <pkpearson@nowhere.invalid> - 2016-04-07 16:51 +0000
                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-07 21:43 -0700
                                Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-07 21:47 -0700
                                Re: Unicode normalisation [was Re: [beginner] What's wrong?] Chris Angelico <rosuav@gmail.com> - 2016-04-08 14:54 +1000
                                  Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-08 10:51 -0700
                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Steven D'Aprano <steve@pearwood.info> - 2016-04-08 16:00 +1000
                                Re: Unicode normalisation [was Re: [beginner] What's wrong?] Chris Angelico <rosuav@gmail.com> - 2016-04-08 16:13 +1000
                                Re: Unicode normalisation [was Re: [beginner] What's wrong?] Peter Pearson <pkpearson@nowhere.invalid> - 2016-04-08 17:21 +0000
                                  Re: Unicode normalisation [was Re: [beginner] What's wrong?] Marko Rauhamaa <marko@pacujo.net> - 2016-04-08 20:44 +0300
                                    Re: Unicode normalisation [was Re: [beginner] What's wrong?] Chris Angelico <rosuav@gmail.com> - 2016-04-09 03:50 +1000
                                      Re: Unicode normalisation [was Re: [beginner] What's wrong?] Peter Pearson <pkpearson@nowhere.invalid> - 2016-04-08 18:03 +0000
                                        Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-08 11:17 -0700
                                          Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-08 11:20 -0700
                                    Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-08 11:04 -0700
                                      Re: Unicode normalisation [was Re: [beginner] What's wrong?] Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-08 20:20 -0400
                                        Re: Unicode normalisation [was Re: [beginner] What's wrong?] alister <alister.ware@ntlworld.com> - 2016-04-09 08:30 +0000
                                          Re: Unicode normalisation [was Re: [beginner] What's wrong?] Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-04-09 14:43 +0100
                                            Re: Unicode normalisation [was Re: [beginner] What's wrong?] Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-04-09 15:34 +0100
                                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-09 14:30 -0400
                                            Re: Unicode normalisation [was Re: [beginner] What's wrong?] Rustom Mody <rustompmody@gmail.com> - 2016-04-09 09:08 -0700
                                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-04-09 19:27 +0100
                                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-09 20:25 +0100
                                              Re: Unicode normalisation [was Re: [beginner] What's wrong?] Stephen Hansen <me@ixokai.io> - 2016-04-09 12:45 -0700
                                            Re: Unicode normalisation [was Re: [beginner] What's wrong?] Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-04-10 20:35 +1200
                                      QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Ben Finney <ben+python@benfinney.id.au> - 2016-04-09 10:43 +1000
                                        Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Steven D'Aprano <steve@pearwood.info> - 2016-04-09 13:28 +1000
                                          Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Random832 <random832@fastmail.com> - 2016-04-09 11:44 -0400
                                          Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Random832 <random832@fastmail.com> - 2016-04-09 11:53 -0400
                                            Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Steven D'Aprano <steve@pearwood.info> - 2016-04-18 11:39 +1000
                                              Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Random832 <random832@fastmail.com> - 2016-04-17 22:01 -0400
                                                Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-04-18 17:21 +1000
                                                  Re: QWERTY was not designed to intentionally slow typists down Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-04-18 21:17 +1200
                                              Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Chris Angelico <rosuav@gmail.com> - 2016-04-18 12:09 +1000
                                              Re: QWERTY was not designed to intentionally slow typists down Michael Torrie <torriem@gmail.com> - 2016-04-17 21:50 -0600
                                              Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-18 00:06 -0400
                                          Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-09 14:52 -0400
                                            Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) pyotr filipivich <phamp@mindspring.com> - 2016-04-09 20:09 -0700
                                              Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) Ian Kelly <ian.g.kelly@gmail.com> - 2016-04-10 07:43 -0600
                                                Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) pyotr filipivich <phamp@mindspring.com> - 2016-04-10 19:14 -0700
                                      Re: QWERTY was not designed to intentionally slow typists down Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-09 20:13 +0100
                                        Re: QWERTY was not designed to intentionally slow typists down alister <alister.ware@ntlworld.com> - 2016-04-09 20:22 +0000
                                          Re: QWERTY was not designed to intentionally slow typists down Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-09 22:23 +0100
                                          Re: QWERTY was not designed to intentionally slow typists down Tim Golden <mail@timgolden.me.uk> - 2016-04-09 22:51 +0100
                                      Re: QWERTY was not designed to intentionally slow typists down Tim Golden <mail@timgolden.me.uk> - 2016-04-09 20:25 +0100
                                      Re: QWERTY was not designed to intentionally slow typists down Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-09 20:36 +0100
                                      Re: QWERTY was not designed to intentionally slow typists down Ethan Furman <ethan@stoneleaf.us> - 2016-04-09 14:33 -0700
                                      RE: [E] QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?]) "Coll-Barth, Michael" <Michael.Coll-Barth@VerizonWireless.com> - 2016-04-09 13:31 -0400
                                  Re: Unicode normalisation [was Re: [beginner] What's wrong?] Steven D'Aprano <steve@pearwood.info> - 2016-04-09 04:44 +1000
                                    Re: Unicode normalisation [was Re: [beginner] What's wrong?] Marko Rauhamaa <marko@pacujo.net> - 2016-04-08 21:55 +0300
                                      Re: Unicode normalisation [was Re: [beginner] What's wrong?] Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-04-10 21:25 +1200
                  Re: [beginner] What's wrong? Steven D'Aprano <steve@pearwood.info> - 2016-04-03 09:49 +1000
                    Re: [beginner] What's wrong? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-03 01:26 +0100
                    Re: [beginner] What's wrong? Rustom Mody <rustompmody@gmail.com> - 2016-04-03 07:52 -0700
                      Re: [beginner] What's wrong? Michael Okuntsov <okuntsov.mikhail@yandex.ru> - 2016-04-03 22:24 +0600
                        Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-04 02:28 +1000
                  Re: [beginner] What's wrong? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-04-03 16:57 +1200
                    Re: [beginner] What's wrong? Steven D'Aprano <steve@pearwood.info> - 2016-04-03 15:34 +1000
                Re: [beginner] What's wrong? Terry Reedy <tjreedy@udel.edu> - 2016-04-02 15:07 -0400
                  Re: [beginner] What's wrong? Marko Rauhamaa <marko@pacujo.net> - 2016-04-02 22:36 +0300
                    Re: [beginner] What's wrong? Michael Selik <michael.selik@gmail.com> - 2016-04-02 21:42 +0000
                      Re: [beginner] What's wrong? Steven D'Aprano <steve@pearwood.info> - 2016-04-03 10:48 +1000
                        Re: [beginner] What's wrong? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-03 02:04 +0100
                          Re: [beginner] What's wrong? alister <alister.ware@ntlworld.com> - 2016-04-03 12:37 +0000
            Re: [beginner] What's wrong? Terry Reedy <tjreedy@udel.edu> - 2016-04-02 14:59 -0400
          Re: [beginner] What's wrong? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-04-03 16:43 +1200
        Re: [beginner] What's wrong? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-02 12:31 -0400
        Re: [beginner] What's wrong? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-03 00:58 +0100
        Re: [beginner] What's wrong? sohcahtoa82@gmail.com - 2016-04-08 15:59 -0700
          Re: [beginner] What's wrong? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-09 00:07 +0100
      Re: [beginner] What's wrong? Michael Torrie <torriem@gmail.com> - 2016-04-02 16:49 -0600
        Re: [beginner] What's wrong? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-04-03 10:12 +0200
      Re: [beginner] What's wrong? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-04 15:04 +0100
        Re: [beginner] What's wrong? BartC <bc@freeuk.com> - 2016-04-04 15:51 +0100
      From email addresses sometimes strange on this list - was Re: [beginner] What's wrong? Michael Torrie <torriem@gmail.com> - 2016-04-04 16:55 -0600
      Re: From email addresses sometimes strange on this list - was Re: [beginner] What's wrong? Chris Angelico <rosuav@gmail.com> - 2016-04-05 08:58 +1000
      Re: From email addresses sometimes strange on this list - was Re: [beginner] What's wrong? Michael Torrie <torriem@gmail.com> - 2016-04-04 17:19 -0600

Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6  Next page →

#106651 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Fri, Apr 8, 2016 at 4:00 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> Or for that matter:
>
> a = akjhvciwfdwkejfc2qweoduycwldvqspjcwuhoqwe9fhlcjbqvcbhsiauy37wkg() + 100
> b = 100 + akjhvciwfdwkejfc2qweoduycwldvqspjcwuhoqew9fhlcjbqvcbhsiauy37wkg()
>
> How easily can you tell them apart at a glance?

Ouch! Can't even align them top and bottom. This is evil.

> I think that, beyond normalisation, the compiler need not be too concerned
> by confusables. I wouldn't *object* to the compiler raising a warning if it
> detected confusable identifiers, or mixed script identifiers, but I think
> that's more the job for a linter or human code review.

The compiler should treat as identical anything that an editor should
reasonably treat as identical. I'm not sure whether multiple combining
characters on a single base character are forced into some order prior
to comparison or are kept in the order they were typed, but my gut
feeling is that they should be considered identical.

> They are not, and never have been, in the typesetting business. Perhaps
> characters are not the only things easily confused *wink*

Peter is definitely a character. So are you. QUITE a character. :)

> But really, why should we object? Is "pile-of-poo" any more silly than any
> of the other dingbats, graphics characters, and other non-alphabetical
> characters? Unicode is not just for "letters of the alphabet".

It's less silly than "ZERO-WIDTH NON-BREAKING SPACE", which isn't a
space at all, it's a joiner. Go figure.

(History's a wonderful thing, ain't it? So's backward compatibility
and a guarantee that names will never be changed.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#106696 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Fri, 08 Apr 2016 16:00:10 +1000, Steven D'Aprano <steve@pearwood.info> wrote:
> On Fri, 8 Apr 2016 02:51 am, Peter Pearson wrote:
>> 
>> The Unicode consortium was certifiably insane when it went into the
>> typesetting business.
>
> They are not, and never have been, in the typesetting business. Perhaps
> characters are not the only things easily confused *wink*

Defining codepoints that deal with appearance but not with meaning is
going into the typesetting business.  Examples: ligatures, and spaces of
varying widths with specific typesetting properties like being non-breaking.

Typesetting done in MS Word using such Unicode codepoints will never
be more than a goofy approximation to real typesetting (e.g., TeX), but
it will cost a huge amount of everybody's time, with the current discussion
of ligatures in variable names being just a straw in the wind.  Getting
all the world's writing systems into a single, coherent standard was
an extraordinarily ambitious, monumental undertaking, and I'm baffled
that the urge to broaden its scope in this irrelevant direction was
entertained at all.

(Should this have been in cranky-geezer font?)

-- 
To email me, substitute nowhere->runbox, invalid->com.

[toc] | [prev] | [next] | [standalone]

#106697 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

Peter Pearson <pkpearson@nowhere.invalid>:

> On Fri, 08 Apr 2016 16:00:10 +1000, Steven D'Aprano <steve@pearwood.info> wrote:
>> They are not, and never have been, in the typesetting business.
>> Perhaps characters are not the only things easily confused *wink*
>
> Defining codepoints that deal with appearance but not with meaning is
> going into the typesetting business. Examples: ligatures, and spaces
> of varying widths with specific typesetting properties like being
> non-breaking.
>
> Typesetting done in MS Word using such Unicode codepoints will never
> be more than a goofy approximation to real typesetting (e.g., TeX),
> but it will cost a huge amount of everybody's time, with the current
> discussion of ligatures in variable names being just a straw in the
> wind. Getting all the world's writing systems into a single, coherent
> standard was an extraordinarily ambitious, monumental undertaking, and
> I'm baffled that the urge to broaden its scope in this irrelevant
> direction was entertained at all.

I agree completely but at the same time have a lot of understanding for
the reasons why Unicode had to become such a mess. Part of it is
historical, part of it is political, yet part of it is in the
unavoidable messiness of trying to define what a character is.

For example, is "ä" one character or two: "a" plus "¨"? Is "i" one
character of two: "ı" plus "˙"? Is writing linear or two-dimensional?

Unicode heroically and definitively solved the problems ASCII had posed
but introduced a bag of new, trickier problems.

(As for ligatures, I understand that there might be quite a bit of
legacy software that dedicated code points and code pages for ligatures.
Translating that legacy software to Unicode was made more
straightforward by introducing analogous codepoints to Unicode. Unicode
has quite many such codepoints: µ, K, Ω etc.)


Marko

[toc] | [prev] | [next] | [standalone]

#106698 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Sat, Apr 9, 2016 at 3:44 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Unicode heroically and definitively solved the problems ASCII had posed
> but introduced a bag of new, trickier problems.
>
> (As for ligatures, I understand that there might be quite a bit of
> legacy software that dedicated code points and code pages for ligatures.
> Translating that legacy software to Unicode was made more
> straightforward by introducing analogous codepoints to Unicode. Unicode
> has quite many such codepoints: µ, K, Ω etc.)

More specifically, Unicode solved the problems that *codepages* had
posed. And one of the principles of its design was that every
character in every legacy encoding had a direct representation as a
Unicode codepoint, allowing bidirectional transcoding for
compatibility. Perhaps if Unicode had existed from the dawn of
computing, we'd have less characters; but backward compatibility is
way too important to let a narrow purity argument sway it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#106701 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Sat, 9 Apr 2016 03:50:16 +1000, Chris Angelico <rosuav@gmail.com> wrote:
> On Sat, Apr 9, 2016 at 3:44 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
[snip]
>> (As for ligatures, I understand that there might be quite a bit of
>> legacy software that dedicated code points and code pages for ligatures.
>> Translating that legacy software to Unicode was made more
>> straightforward by introducing analogous codepoints to Unicode. Unicode
>> has quite many such codepoints: µ, K, Ω etc.)
>
> More specifically, Unicode solved the problems that *codepages* had
> posed. And one of the principles of its design was that every
> character in every legacy encoding had a direct representation as a
> Unicode codepoint, allowing bidirectional transcoding for
> compatibility. Perhaps if Unicode had existed from the dawn of
> computing, we'd have less characters; but backward compatibility is
> way too important to let a narrow purity argument sway it.

I guess with that historical perspective the current situation
seems almost inevitable.  Thanks.  And thanks to Steven D'Aprano
for other relevant insights.

-- 
To email me, substitute nowhere->runbox, invalid->com.

[toc] | [prev] | [next] | [standalone]

#106704 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Friday, April 8, 2016 at 11:33:38 PM UTC+5:30, Peter Pearson wrote:
> On Sat, 9 Apr 2016 03:50:16 +1000, Chris Angelico wrote:
> > On Sat, Apr 9, 2016 at 3:44 AM, Marko Rauhamaa  wrote:
> [snip]
> >> (As for ligatures, I understand that there might be quite a bit of
> >> legacy software that dedicated code points and code pages for ligatures.
> >> Translating that legacy software to Unicode was made more
> >> straightforward by introducing analogous codepoints to Unicode. Unicode
> >> has quite many such codepoints: µ, K, Ω etc.)
> >
> > More specifically, Unicode solved the problems that *codepages* had
> > posed. And one of the principles of its design was that every
> > character in every legacy encoding had a direct representation as a
> > Unicode codepoint, allowing bidirectional transcoding for
> > compatibility. Perhaps if Unicode had existed from the dawn of
> > computing, we'd have less characters; but backward compatibility is
> > way too important to let a narrow purity argument sway it.
> 
> I guess with that historical perspective the current situation
> seems almost inevitable.  Thanks.  And thanks to Steven D'Aprano
> for other relevant insights.

Strange view
In fact the unicode standard itself encourages not using the standard in its
entirety

5.12 Deprecation

In the Unicode Standard, the term deprecation is used somewhat differently than it is in some other standards. Deprecation is used to mean that a character or other feature is strongly discouraged from use. This should not, however, be taken as indicating that anything has been removed from the standard, nor that anything is planned for removal from the standard. Any such change is constrained by the Unicode Consortium Stability Policies [Stability].

For the Unicode Character Database, there are two important types of deprecation to be noted. First, an encoded character may be deprecated. Second, a character property may be deprecated.

When an encoded character is strongly discouraged from use, it is given the property value Deprecated=True. The Deprecated property is a binary property defined specifically to carry this information about Unicode characters. Very few characters are ever formally deprecated this way; it is not enough that a character be uncommon, obsolete, disliked, or not preferred. Only those few characters which have been determined by the UTC to have serious architectural defects or which have been determined to cause significant implementation problems are ever deprecated. Even in the most severe cases, such as the deprecated format control characters (U+206A..U+206F), an encoded character is never removed from the standard. Furthermore, although deprecated characters are strongly discouraged from use, and should be avoided in favor of other, more appropriate mechanisms, they may occur in data. Conformant implementations of Unicode processes such a Unicode normalization must handle even deprecated characters correctly.

I read this as saying that -- in addition to officially deprecated chars --
there ARE "uncommon, obsolete, disliked, or not preferred" chars
which sensible users should avoid using even though unicode as a standard is
compelled to keep supporting

Which translates into
- python as a language *implementing* unicode (eg in strings) needs to
do it completely if it is to be standard compliant
- python as a *user* of unicode (eg in identifiers) can (and IMHO should)
use better judgement

[toc] | [prev] | [next] | [standalone]

#106705 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

Adding link

On Friday, April 8, 2016 at 11:48:07 PM UTC+5:30, Rustom Mody wrote:
<Quote>
> 5.12 Deprecation
> 
> In the Unicode Standard, the term deprecation is used somewhat differently than it is in some other standards. Deprecation is used to mean that a character or other feature is strongly discouraged from use. This should not, however, be taken as indicating that anything has been removed from the standard, nor that anything is planned for removal from the standard. Any such change is constrained by the Unicode Consortium Stability Policies [Stability].
> 
> For the Unicode Character Database, there are two important types of deprecation to be noted. First, an encoded character may be deprecated. Second, a character property may be deprecated.
> 
> When an encoded character is strongly discouraged from use, it is given the property value Deprecated=True. The Deprecated property is a binary property defined specifically to carry this information about Unicode characters. Very few characters are ever formally deprecated this way; it is not enough that a character be uncommon, obsolete, disliked, or not preferred. Only those few characters which have been determined by the UTC to have serious architectural defects or which have been determined to cause significant implementation problems are ever deprecated. Even in the most severe cases, such as the deprecated format control characters (U+206A..U+206F), an encoded character is never removed from the standard. Furthermore, although deprecated characters are strongly discouraged from use, and should be avoided in favor of other, more appropriate mechanisms, they may occur in data. Conformant implementations of Unicode processes such a Unicode normalization must handle even deprecated characters correctly.

</Quote>

Link: http://unicode.org/reports/tr44/#Deprecation

[toc] | [prev] | [next] | [standalone]

#106703 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Friday, April 8, 2016 at 11:14:21 PM UTC+5:30, Marko Rauhamaa wrote:
> Peter Pearson :
> 
> > On Fri, 08 Apr 2016 16:00:10 +1000, Steven D'Aprano  wrote:
> >> They are not, and never have been, in the typesetting business.
> >> Perhaps characters are not the only things easily confused *wink*
> >
> > Defining codepoints that deal with appearance but not with meaning is
> > going into the typesetting business. Examples: ligatures, and spaces
> > of varying widths with specific typesetting properties like being
> > non-breaking.
> >
> > Typesetting done in MS Word using such Unicode codepoints will never
> > be more than a goofy approximation to real typesetting (e.g., TeX),
> > but it will cost a huge amount of everybody's time, with the current
> > discussion of ligatures in variable names being just a straw in the
> > wind. Getting all the world's writing systems into a single, coherent
> > standard was an extraordinarily ambitious, monumental undertaking, and
> > I'm baffled that the urge to broaden its scope in this irrelevant
> > direction was entertained at all.
> 
> I agree completely but at the same time have a lot of understanding for
> the reasons why Unicode had to become such a mess. Part of it is
> historical, part of it is political, yet part of it is in the
> unavoidable messiness of trying to define what a character is.

There are standards and standards.
Just because they are standard does not make them useful, well-designed,
reasonable etc..

Its reasonably likely that all our keyboards start QWERT...
 Doesn't make it a sane design.

Likewise using NFKC to define the equivalence relation on identifiers
is analogous to saying: Since QWERTY has been in use for over a hundred years
its a perfectly good design. Just because NFKC has the stamp of the unicode
consortium it does not straightaway make it useful for all purposes

[toc] | [prev] | [next] | [standalone]

#106713 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Fri, 8 Apr 2016 11:04:53 -0700 (PDT), Rustom Mody
<rustompmody@gmail.com> declaimed the following:

>Its reasonably likely that all our keyboards start QWERT...
> Doesn't make it a sane design.
>
	It was a sane design -- for early mechanical typewrites. It fulfills
its goal of slowing down a typist to reduce jamming print-heads at the
platen.* And since so many of us who had formal touch typing training
probably learned on said mechanical typewriters, it hangs around.
Fortunately, even though the typewriters at school had European dead-keys,
we were plain English and I never had to pick them up.

	For a few years I did have problems with ()... They were on different
keys (8 and 9, respectively) on old typewriters (the type that also had no
1) vs IBM Selectrics (never used by be) and computer terminals...



* Except I kept jamming two letters of my last name... I and E are reached
with the same finger on opposite hands, which made a fast stroke-pair
(compare moving the same finger on both hands to moving different fingers).
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#106722 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Fri, 08 Apr 2016 20:20:02 -0400, Dennis Lee Bieber wrote:

> On Fri, 8 Apr 2016 11:04:53 -0700 (PDT), Rustom Mody
> <rustompmody@gmail.com> declaimed the following:
> 
>>Its reasonably likely that all our keyboards start QWERT...
>> Doesn't make it a sane design.
>>
> 	It was a sane design -- for early mechanical typewrites. It 
fulfills
> its goal of slowing down a typist to reduce jamming print-heads at the
> platen.* And since so many of us who had formal touch typing training
> probably learned on said mechanical typewriters, it hangs around.
> Fortunately, even though the typewriters at school had European
> dead-keys, we were plain English and I never had to pick them up.
> 
> 	For a few years I did have problems with ()... They were on 
different
> keys (8 and 9, respectively) on old typewriters (the type that also had
> no 1) vs IBM Selectrics (never used by be) and computer terminals...
> 
> 
> 
> * Except I kept jamming two letters of my last name... I and E are
> reached with the same finger on opposite hands, which made a fast
> stroke-pair (compare moving the same finger on both hands to moving
> different fingers).

<pedant Mode on>
the design of qwerty was not to "Slow" the typist bu to ensure that the 
hammers for letters commonly used together are spaced widely apart, 
reducing the portion of trier travel arc were the could jam.
I and E are actually such a pair which is why they are at opposite ends 
of the hammer rack (I doubt that is the correct technical term).
they are on opposite hands to make typing of them faster.
unfortunately as you found it is still possible to jam them if they are 
hit almost simultaneously
<Pedant Mode Off>



-- 
There's a trick to the Graceful Exit.  It begins with the vision to
recognize when a job, a life stage, a relationship is over -- and to let
go.  It means leaving what's over without denying its validity or its
past importance in our lives.  It involves a sense of future, a belief
that every exit line is an entry, that we are moving on, rather than out.
The trick of retiring well may be the trick of living well.  It's hard to
recognize that life isn't a holding action, but a process.  It's hard to
learn that we don't leave the best parts of ourselves behind, back in the
dugout or the office. We own what we learned back there.  The experiences
and the growth are grafted onto our lives.  And when we exit, we can take
ourselves along -- quite gracefully.
		-- Ellen Goodman

[toc] | [prev] | [next] | [standalone]

#106726 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

alister <alister.ware@ntlworld.com> writes:
<snip>
> <pedant Mode on>
> the design of qwerty was not to "Slow" the typist bu to ensure that the 
> hammers for letters commonly used together are spaced widely apart, 
> reducing the portion of trier travel arc were the could jam.
> I and E are actually such a pair which is why they are at opposite ends 
> of the hammer rack (I doubt that is the correct technical term).
> they are on opposite hands to make typing of them faster.
> unfortunately as you found it is still possible to jam them if they are 
> hit almost simultaneously
> <Pedant Mode Off>

The problem with that theory is that 'er/re' (this is e and r in either
order) is the 3rd most common pair in English but have been placed
together.  ou and et (in either order) are the 15th and 22nd most common
and they are separated by only one hammer position.  On the other hand,
the QWERTY layout puts jk together, but they almost never appear
together in English text.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#106732 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:

> alister <alister.ware@ntlworld.com> writes:
> <snip>
>> <pedant Mode on>
>> the design of qwerty was not to "Slow" the typist bu to ensure that the 
>> hammers for letters commonly used together are spaced widely apart, 
>> reducing the portion of trier travel arc were the could jam.
>> I and E are actually such a pair which is why they are at opposite ends 
>> of the hammer rack (I doubt that is the correct technical term).
>> they are on opposite hands to make typing of them faster.
>> unfortunately as you found it is still possible to jam them if they are 
>> hit almost simultaneously
>> <Pedant Mode Off>
>
> The problem with that theory is that 'er/re' (this is e and r in either
> order) is the 3rd most common pair in English but have been placed
> together.  ou and et (in either order) are the 15th and 22nd most common
> and they are separated by only one hammer position.  On the other hand,
> the QWERTY layout puts jk together, but they almost never appear
> together in English text.

This last part came out muddled.  It's obviously wise to put infrequent
combinations together (like jk), but j and k are both also rare letters
so putting them together represents a wasted opportunity for meeting the
supposed design objective.  Swapping, say, k and r, or splitting jk but
putting e in the middle would surely result in a net gain of "hammer
separation".

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#106750 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Sat, 09 Apr 2016 15:34:29 +0100, Ben Bacarisse <ben.usenet@bsb.me.uk>
declaimed the following:

>supposed design objective.  Swapping, say, k and r, or splitting jk but
>putting e in the middle would surely result in a net gain of "hammer
>separation".

	JEK would not help... 

key
keep
kernel
jerk
jelly
jenny

That E is too common with either J or K

	Then again... E is too common with most of the alphabet <G>
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#106744 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Saturday, April 9, 2016 at 7:14:05 PM UTC+5:30, Ben Bacarisse wrote:
> The problem with that theory is that 'er/re' (this is e and r in either
> order) is the 3rd most common pair in English but have been placed
> together.  ou and et (in either order) are the 15th and 22nd most common
> and they are separated by only one hammer position.  On the other hand,
> the QWERTY layout puts jk together, but they almost never appear
> together in English text.

Where do you get this (kind of) statistical data?

[toc] | [prev] | [next] | [standalone]

#106749 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

Rustom Mody <rustompmody@gmail.com> writes:

> On Saturday, April 9, 2016 at 7:14:05 PM UTC+5:30, Ben Bacarisse wrote:
>> The problem with that theory is that 'er/re' (this is e and r in either
>> order) is the 3rd most common pair in English but have been placed
>> together.  ou and et (in either order) are the 15th and 22nd most common
>> and they are separated by only one hammer position.  On the other hand,
>> the QWERTY layout puts jk together, but they almost never appear
>> together in English text.
>
> Where do you get this (kind of) statistical data?

It was generated by counting the pairs found in a corpus of texts taken
from Project Gutenberg.  The numbers do very depending on what you pick
(for the complete works of Mark Twain er/re is second, for example), and
the none of the texts are very modern (because of the source) but I
doubt that matters too much.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#106759 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On 09/04/2016 17:08, Rustom Mody wrote:
> On Saturday, April 9, 2016 at 7:14:05 PM UTC+5:30, Ben Bacarisse wrote:
>> The problem with that theory is that 'er/re' (this is e and r in either
>> order) is the 3rd most common pair in English but have been placed
>> together.  ou and et (in either order) are the 15th and 22nd most common
>> and they are separated by only one hammer position.  On the other hand,
>> the QWERTY layout puts jk together, but they almost never appear
>> together in English text.
>
> Where do you get this (kind of) statistical data?
>

Again, where is the relevance to Python in this discussion, as we're on 
the main Python mailing list?  Please can the moderators take this stuff 
out, it is getting beyond the pale.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#106768 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

On Sat, Apr 9, 2016, at 12:25 PM, Mark Lawrence via Python-list wrote:
> Again, where is the relevance to Python in this discussion, as we're on 
> the main Python mailing list?  Please can the moderators take this stuff 
> out, it is getting beyond the pale.

You need to come to grip with the fact that python-list is only
moderated in the vaguest sense of the word. 

Quote:

https://www.python.org/community/lists/
"Pretty much anything Python-related is fair game for discussion, and
the group is even fairly tolerant of off-topic digressions; there have
been entertaining discussions of topics such as floating point, good
software design, and other programming languages such as Lisp and
Forth."

If you don't like it, sorry. We all have our burdens to bear.

--S

[toc] | [prev] | [next] | [standalone]

#106776 — Re: Unicode normalisation [was Re: [beginner] What's wrong?]

Ben Bacarisse wrote:
> The problem with that theory is that 'er/re' (this is e and r in either
> order) is the 3rd most common pair in English but have been placed
> together.

No, they haven't. The order of the characters in the type
basket goes down the slanted columns of keys, so E and R
are separated by D and C.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#106714 — QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?])

Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:

> [The QWERTY keyboard layout] was a sane design -- for early mechanical
> typewrites. It fulfills its goal of slowing down a typist to reduce
> jamming print-heads at the platen.

This is an often-repeated myth, with citations back as far as the 1970s.
It is false.

The design is intended to reduce jamming the print heads together, but
the goal of this is not to reduce speed, but to enable *fast* typing.

It aims to maximise the frequency in which (English-language) text has
consecutive letters alternating either side of the middle of the
keyboard. This should thus reduce collisions of nearby heads — and hence
*increase* the effective typing speed that can be achieved on such a
mechanical typewriter.

The degree to which this maximum was achieved is arguable. Certainly the
relevance to keyboards today, with no connection from the layout to
whether print heads will jam, is negligible.

What is not arguable is that there is no evidence the design had any
intention of *slowing* typists in any way. Quite the opposite, in fact.

<URL:http://www.straightdope.com/columns/read/221/was-the-qwerty-keyboard-purposely-designed-to-slow-typists>,
and other links from the Wikipedia article
<URL:https://en.wikipedia.org/wiki/QWERTY#History_and_purposes>, should
allow interested people to get the facts right on this canard.

-- 
 \     “I used to think that the brain was the most wonderful organ in |
  `\   my body. Then I realized who was telling me this.” —Emo Philips |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#106720 — Re: QWERTY was not designed to intentionally slow typists down (was: Unicode normalisation [was Re: [beginner] What's wrong?])

On Sat, 9 Apr 2016 10:43 am, Ben Finney wrote:

> Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
> 
>> [The QWERTY keyboard layout] was a sane design -- for early mechanical
>> typewrites. It fulfills its goal of slowing down a typist to reduce
>> jamming print-heads at the platen.
> 
> This is an often-repeated myth, with citations back as far as the 1970s.
> It is false.
> 
> The design is intended to reduce jamming the print heads together, but
> the goal of this is not to reduce speed, but to enable *fast* typing.

And how did it enable fast typing? By *slowing down the typist*, and thus
having fewer jams.

Honestly, I have the greatest respect for the Straight Dope, but this is one
of those times when they miss the forest for the trees. The conventional
wisdom about typewriters isn't wrong -- or at least there's no evidence
that it's wrong.

As far as I can, *every single* argument against the conventional wisdom
comes down to an argument that it is ridiculous or silly that anyone might
have wanted to slow typing down. For example, Wikipedia links to this page:

http://www.smithsonianmag.com/arts-culture/fact-of-fiction-the-legend-of-the-qwerty-keyboard-49863249/?no-ist

which quotes researchers:

“The speed of Morse receiver should be equal to the Morse sender, of course.
If Sholes really arranged the keyboard to slow down the operator, the
operator became unable to catch up the Morse sender. We don’t believe that
Sholes had such a nonsense intention during his development of
Type-Writer.”

This is merely argument from personal incredibility:

http://rationalwiki.org/wiki/Argument_from_incredulity

and is trivially answerable: how well do you think the receiver can keep up
with the sender if they have to stop every few dozen keystrokes to unjam
the typewriter?

Wikipedia states:

"Contrary to popular belief, the QWERTY layout was not designed to slow the
typist down,[3]"

with the footnote [3] linking to

http://www.maltron.com/media/lillian_kditee_001.pdf

which clearly and prominently states in the THIRD paragraph:

"It has been said of the Sholes letter layout [QWERTY] this it would
probably have been chosen if the objective was to find the least
efficient -- in terms of learning time and speed achievable -- and the most
error producing character arrangement. This is not surprising when one
considers that a team of people spent one year developing this layout so
that it should provide THE GREATEST INHIBITION TO FAST KEYING. [Emphasis
added.] This was no Machiavellian plot, but necessary because the mechanism
of the early typewriters required slow operation."

This is the power of the "slowing typists down is a myth" meme: same
Wikipedia contributor takes an article which *clearly and obviously*
repeats the conventional narrative that QWERTY was designed to decrease the
number of key presses per second, and uses that to defend the counter-myth
that QWERTY wasn't designed to decrease the number of key presses per
second!

These are the historical facts:

- early typewriters had varying layouts, some of which allow much more rapid
keying than QWERTY;

- early typewriters were prone to frequent and difficult jamming;

- Sholes spend significant time developing a layout which reduced the number
of jams by intentionally moving frequently typed characters far apart,
which has the effect of slowing down the rate at which the typist can hit
keys;

- which results in greater typing speed do to a reduced number of jams.

In other words the conventional story.

Jams have such a massively negative effect on typing speed that reducing the
number of jams gives you a *huge* win on overall speed even if the rate of
keying is significantly lower. At first glance, it may seem paradoxical,
but it's not. Which is faster?

- typing at a steady speed of (lets say) 100 words per minute;

- typing in bursts of (say) 200 wpm for a minute, followed by three minutes
of 0 wpm.

The second case averages half the speed of the first, even though the typist
is hitting keys at a faster rate. This shouldn't be surprising to any car
driver who has raced from one red light to the next, only to be caught up
and even overtaken by somebody driving at a more sedate speed who caught
nothing but green lights. Or to anyone who has heard the story of the
Tortoise and the Hare.

The moral of QWERTY is "less haste, more speed".

The myth of the "QWERTY myth" is based on the idea that people are unable to
distinguish between peak speed and average speed. But ironically, in my
experience, it's only those repeating the myth who seem confused by that
difference (as in the quote from the Smithsonian above). Most people don't
need the conventional narrative explained:

"Speed up typing by slowing the typist down? Yeah, that makes sense. When I
try to do things in a rush, I make more mistakes and end up taking longer
than I otherwise would have. This is exactly the same sort of principle."

while others, like our dear Cecil from the Straight Dope, wrongly imagine
that ordinary folks cannot understand this rather simple concept, and
therefore must believe something which nobody has every said:

"QWERTY was designed to make typing slower."

Well, maybe Dvorak proponents, but they have an ulterior motive to
misrepresent the situation.





-- 
Steven

[toc] | [prev] | [next] | [standalone]

Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6  Next page →

Back to top | Article view | comp.lang.python

csiph-web