Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92257

Re: Ah Python, you have spoiled me for all other languages

References (4 earlier) <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com> <2483375.eHyISxeWLQ@PointedEars.de> <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com> <mailman.243.1433678894.13271.python-list@python.org> <557445f6$0$12997$c3e8da3$5496439d@news.astraweb.com>
Date 2015-06-08 00:47 +1000
Subject Re: Ah Python, you have spoiled me for all other languages
From Chris Angelico <rosuav@gmail.com>
Newsgroups comp.lang.python
Message-ID <mailman.250.1433688442.13271.python-list@python.org> (permalink)

Show all headers | View raw

On Sun, Jun 7, 2015 at 11:24 PM, Steven D'Aprano <steve@pearwood.info> wrote:
>> (Unless you want to say that all strings are guaranteed to
>> be NFC/NFD normalized, such that s1 and s2 would actually be
>> identical, which I suppose is plausible. I'm not sure what the
>> advantage would be, though. And certainly you wouldn't want to
>> K-normalize strings automatically.)
>
> I believe that filenames on Apple file systems (HFS+ if I remember
> correctly) are guaranteed to be both normalised and correctly encoded as
> UTF-8. If you could live in a purely Apple world, you'd have far fewer
> filename hassles.

Yep. Actually, there should be nothing stopping the next Linux file
system ("ext5" or whatever) from enforcing the same thing;
byte-oriented filename APIs would still work just fine, but you could
have some confidence that at least local file systems will normally be
decodable as UTF-8. Then the only time you'd have to worry about
encoding problems would be network or removable file systems - no
worrying about "what's the FS encoding", because it'll just be UTF-8.
(Hmm. Point of interest: What happens on a Mac if you network-mount
something that isn't Unicode? If the enforcement of UTF-8 and
normalization is done at the file system level, it's no different from
the current Linux situation, where basically anything goes.)

But that's file names, not strings in a program. I'm not sure that
mandating that strings be normalized is particularly useful, but on
the flip side, I'm not sure of any situation where it'd be majorly
problematic either. There are ambiguities in some encodings, and as
soon as you decode from them and re-encode, you've effectively folded
those ambiguities to some canonical form; if your language
automatically normalized strings, you'd just have the same effect of
folding. And then you could have encode methods that stipulate the
other form of normalization - say you NFD everything internally, you
could then have a method "a\u0301".encode("utf-8", combine=True) which
NFC normalizes prior to encoding (and would thus be C3 A1 instead of
61 CC 81). Are there any languages out there that work this way?

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-23 00:58 +1000
  Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 01:29 +1000
    Re: Ah Python, you have spoiled me for all other languages wxjmfauth@gmail.com - 2015-05-22 10:57 -0700
    Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-22 16:40 -0500
    Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-22 16:40 -0500
      Re: Ah Python, you have spoiled me for all other languages Terry Reedy <tjreedy@udel.edu> - 2015-05-22 21:54 -0400
        Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:12 -0500
        Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:12 -0500
          Re: Ah Python, you have spoiled me for all other languages Terry Reedy <tjreedy@udel.edu> - 2015-05-23 13:26 -0400
      Re: Ah Python, you have spoiled me for all other languages Michael Torrie <torriem@gmail.com> - 2015-05-22 21:31 -0600
        Re: Ah Python, you have spoiled me for all other languages Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-05-23 08:55 +0200
          Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:21 -0500
            Re: Ah Python, you have spoiled me for all other languages Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-05-23 15:24 +0200
              Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-23 20:05 +0300
                Re: Ah Python, you have spoiled me for all other languages Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-05-24 20:29 +0200
          Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-23 15:44 +0300
            Re: Ah Python, you have spoiled me for all other languages Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-05-23 15:17 +0200
            Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-24 00:00 +1000
              Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-23 19:53 +0300
                Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-24 03:41 +1000
                Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-23 22:02 +0300
                Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-24 20:26 +1000
                Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-24 18:26 +0300
                Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-25 01:35 +1000
                Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-25 09:57 +0300
                Re: Ah Python, you have spoiled me for all other languages Laura Creighton <lac@openend.se> - 2015-05-25 11:39 +0200
                Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-25 21:09 +1000
            Re: Ah Python, you have spoiled me for all other languages Michael Torrie <torriem@gmail.com> - 2015-05-23 21:00 -0600
              Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-24 11:23 +0300
      Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-22 22:10 -0600
      Re: Ah Python, you have spoiled me for all other languages amber <amber.of.luxor@gmail.com> - 2015-05-23 04:11 +0000
        Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:11 -0500
        Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:11 -0500
      Re: Ah Python, you have spoiled me for all other languages Ben Finney <ben+python@benfinney.id.au> - 2015-05-23 14:20 +1000
      Re: Ah Python, you have spoiled me for all other languages Michael Torrie <torriem@gmail.com> - 2015-05-22 22:30 -0600
        Re: Ah Python, you have spoiled me for all other languages Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-05-23 11:10 +0000
          Re: Ah Python, you have spoiled me for all other languages Tim Chase <python.list@tim.thechases.com> - 2015-05-23 06:34 -0500
          Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 21:40 +1000
          Re: Ah Python, you have spoiled me for all other languages Michael Torrie <torriem@gmail.com> - 2015-05-23 20:57 -0600
          Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-24 01:22 -0600
      Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-22 22:29 -0600
      Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-22 22:49 -0600
      Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 14:49 +1000
        Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-23 06:29 -0500
      Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 14:55 +1000
      Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 14:28 +1000
      Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 14:21 +1000
    Re: Ah Python, you have spoiled me for all other languages Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-05-23 14:33 +0200
      Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-23 23:01 +1000
        Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 23:12 +1000
          Re: Ah Python, you have spoiled me for all other languages wxjmfauth@gmail.com - 2015-05-23 23:37 -0700
        Re: Ah Python, you have spoiled me for all other languages Ned Batchelder <ned@nedbatchelder.com> - 2015-05-23 06:35 -0700
          Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-24 00:09 +1000
          Re: Ah Python, you have spoiled me for all other languages Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-06-07 10:21 +0200
            Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-06-07 21:42 +1000
              Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-06-07 22:08 +1000
                Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-06-07 23:24 +1000
                Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-06-08 00:47 +1000
              Re: Ah Python, you have spoiled me for all other languages random832@fastmail.us - 2015-06-07 10:58 -0400
                Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-06-08 02:28 +1000
  Re: Ah Python, you have spoiled me for all other languages Tony the Tiger <tony@tiger.invalid> - 2015-05-22 16:31 +0000
    Re: Ah Python, you have spoiled me for all other languages Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-22 17:57 +0100
    Re: Ah Python, you have spoiled me for all other languages Tim Daneliuk <tundra@tundraware.com> - 2015-05-22 16:41 -0500
      Re: Ah Python, you have spoiled me for all other languages Tony the Tiger <tony@tiger.invalid> - 2015-05-23 20:25 +0000
  Re: Ah Python, you have spoiled me for all other languages Grant Edwards <invalid@invalid.invalid> - 2015-05-22 17:47 +0000
    Re: Ah Python, you have spoiled me for all other languages Chris Angelico <rosuav@gmail.com> - 2015-05-23 04:11 +1000
    Re: Ah Python, you have spoiled me for all other languages mm0fmf <none@mailinator.com> - 2015-05-22 19:19 +0100
    Re: Ah Python, you have spoiled me for all other languages Laura Creighton <lac@openend.se> - 2015-05-22 21:14 +0200
      Re: Ah Python, you have spoiled me for all other languages Steven D'Aprano <steve@pearwood.info> - 2015-05-23 11:36 +1000
    Re: Ah Python, you have spoiled me for all other languages MRAB <python@mrabarnett.plus.com> - 2015-05-22 20:34 +0100
    Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-22 13:56 -0600
      Re: Ah Python, you have spoiled me for all other languages Marko Rauhamaa <marko@pacujo.net> - 2015-05-22 23:34 +0300
        Re: Ah Python, you have spoiled me for all other languages Tim Chase <python.list@tim.thechases.com> - 2015-05-22 15:55 -0500
        Re: Ah Python, you have spoiled me for all other languages Ethan Furman <ethan@stoneleaf.us> - 2015-05-22 14:15 -0700
        Re: Ah Python, you have spoiled me for all other languages Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-22 15:20 -0600
  Re: Ah Python, you have spoiled me for all other languages Paul Rubin <no.email@nospam.invalid> - 2015-05-22 16:00 -0700
    Re: Ah Python, you have spoiled me for all other languages Michael Torrie <torriem@gmail.com> - 2015-05-22 21:33 -0600

csiph-web