Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #81089 > unrolled thread

Re: python 2.7 and unicode (one more time)

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
                  Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
          Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
                      Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
                        Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
                          Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
                          Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
                  Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
                      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
                          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
                            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
                            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
                              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
                                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
                                  Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
      Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
            Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
        Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100

Page 1 of 4  [1] 2 3 4  Next page →

#81089 — Re: python 2.7 and unicode (one more time)

On 11/20/2014 09:32 AM, Peter Otten wrote:
> Chris Angelico wrote:
> 
>> On Fri, Nov 21, 2014 at 2:40 AM, Peter Otten <__peter__@web.de> wrote:
>>> I think that you may get a Unicode/Encode/Error when you try to /decode/
>>> a unicode string is more confusing...
>>
>> Hang on a minute, what does it even mean to decode a Unicode string?
> 
> Let's not get philosophical ;)

It's not philosophical.  It's an important distinction that folks need
to be clear on when understanding unicode and the errors that python can
throw.

Unicode can only be encoded to bytes.
Bytes can only be decoded to unicode.

Without understanding that, the exception errors about decoding won't be
properly understood, nor will one know how to fix them.

[toc] | [next] | [standalone]

#81105

Michael Torrie <torriem@gmail.com>:

> Unicode can only be encoded to bytes.
> Bytes can only be decoded to unicode.

I don't really like it how Unicode is equated with text, or even
character strings.

There's barely any difference between the truth value of these
statements:

   Python strings are ASCII.

   Python strings are Latin-1.

   Python strings are Unicode.

Each of those statements is true as long as you stay within the
respective character sets, and cease to be true when your text contains
characters outside the character sets.

Now, it is true that Python currently limits itself to the 1,114,112
Unicode code points. And it likely won't adopt more characters unless
Unicode does it first. However, text is something more lofty and
abstract than a sequence of Unicode code points.

We shouldn't call strings Unicode any more than we call numbers IEEE or
times ISO.


Marko

[toc] | [prev] | [next] | [standalone]

#81146

On Fri, Nov 21, 2014 at 5:56 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Michael Torrie <torriem@gmail.com>:
>
>> Unicode can only be encoded to bytes.
>> Bytes can only be decoded to unicode.
>
> I don't really like it how Unicode is equated with text, or even
> character strings.
>
> There's barely any difference between the truth value of these
> statements:
>
>    Python strings are ASCII.
>
>    Python strings are Latin-1.
>
>    Python strings are Unicode.
>
> Each of those statements is true as long as you stay within the
> respective character sets, and cease to be true when your text contains
> characters outside the character sets.

The difference is that ASCII and Latin-1 cut out a large number of
active world languages, UCS-2 (the intermediate option you didn't
mention) cuts out a small proportion (by usage) of significant
characters, and Unicode cuts out only those characters which fall
under issues like Han unification. (Plus any that haven't yet been
allocated. But since Python doesn't actually validate code points to
ensure that they've been given meanings, you can use today's Python to
work with tomorrow's Unicode.)

Do you have actual text that you're unable to represent in Unicode? If
so, you are going to have major problems using it with *any* computer
system. There are Japanese encodings that can represent additional
characters, but they also *cannot* represent a lot of the other
characters we use, so there'll be fundamental incompatibilities.

> Now, it is true that Python currently limits itself to the 1,114,112
> Unicode code points. And it likely won't adopt more characters unless
> Unicode does it first. However, text is something more lofty and
> abstract than a sequence of Unicode code points.
>
> We shouldn't call strings Unicode any more than we call numbers IEEE or
> times ISO.

We don't call numbers IEEE, but if we're working with Python floats,
we *do* require all numbers to be representable as IEEE
floating-point. Don't like that? Pick decimal.Decimal instead, or
fractions.Fraction, and pick a different set of limitations... but
ultimately, you *will* have restrictions - and much tighter
restrictions than Unicode places on text.

Do you genuinely have text that you can't represent in Unicode, or are
you just arguing against Unicode to try to justify "Python strings are
<something else>" as a basis for your code?

ChrisA

[toc] | [prev] | [next] | [standalone]

#81171

Chris Angelico <rosuav@gmail.com>:

> On Fri, Nov 21, 2014 at 5:56 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>> I don't really like it how Unicode is equated with text, or even
>> character strings.
> [...]
> Do you have actual text that you're unable to represent in Unicode?

Not my point at all.

I'm saying equating an abstract data type (string) with its
representation (Unicode vector) is bad taste.

> We don't call numbers IEEE,

Exactly.

> Do you genuinely have text that you can't represent in Unicode, or are
> you just arguing against Unicode to try to justify "Python strings are
> <something else>" as a basis for your code?

Nobody is arguing against Unicode. I'm saying, let's talk about the
forest instead of the trees (except when the trees really are the
focus).


Marko

[toc] | [prev] | [next] | [standalone]

#81174

On Fri, Nov 21, 2014 at 5:36 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> On Fri, Nov 21, 2014 at 5:56 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>> I don't really like it how Unicode is equated with text, or even
>>> character strings.
>> [...]
>> Do you have actual text that you're unable to represent in Unicode?
>
> Not my point at all.
>
> I'm saying equating an abstract data type (string) with its
> representation (Unicode vector) is bad taste.

What about "sequence of Unicode code points" is "representation"? What
is your abstraction over that?

ChrisA

[toc] | [prev] | [next] | [standalone]

#81177

Chris Angelico <rosuav@gmail.com>:

> On Fri, Nov 21, 2014 at 5:36 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>> I'm saying equating an abstract data type (string) with its
>> representation (Unicode vector) is bad taste.
>
> What about "sequence of Unicode code points" is "representation"? What
> is your abstraction over that?

The letter 'A' is a character. Unicode for the letter 'A' is 65. It is
very rarely that you care about that number. You are only interested in
the letter 'A', which you can use to spell people's names, for instance.

When you read a book, you read the text, not the ink.


Marko

[toc] | [prev] | [next] | [standalone]

#81178

On Fri, Nov 21, 2014 at 6:14 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> On Fri, Nov 21, 2014 at 5:36 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>> I'm saying equating an abstract data type (string) with its
>>> representation (Unicode vector) is bad taste.
>>
>> What about "sequence of Unicode code points" is "representation"? What
>> is your abstraction over that?
>
> The letter 'A' is a character. Unicode for the letter 'A' is 65. It is
> very rarely that you care about that number. You are only interested in
> the letter 'A', which you can use to spell people's names, for instance.
>
> When you read a book, you read the text, not the ink.

Then you need to read more about Unicode. The *codepoint* for the
letter 'A' is 65. That is not Unicode, that is one part of the Unicode
spec.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81180

Chris Angelico <rosuav@gmail.com>:

> Then you need to read more about Unicode. The *codepoint* for the
> letter 'A' is 65. That is not Unicode, that is one part of the Unicode
> spec.

I don't think Python users need to know anything more about Unicode than
they need to know about IEEE-754.

How many bits are reserved for the mantissa? I don't remember and I
don't see why I should care.


Marko

[toc] | [prev] | [next] | [standalone]

#81181

On Fri, Nov 21, 2014 at 7:16 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> Then you need to read more about Unicode. The *codepoint* for the
>> letter 'A' is 65. That is not Unicode, that is one part of the Unicode
>> spec.
>
> I don't think Python users need to know anything more about Unicode than
> they need to know about IEEE-754.
>
> How many bits are reserved for the mantissa? I don't remember and I
> don't see why I should care.

At what point can a Python float no longer represent every integer?
That's why you should care.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81220

On Friday, November 21, 2014 12:06:54 PM UTC+5:30, Marko Rauhamaa wrote:
> Chris Angelico :
> 
> > On Fri, Nov 21, 2014 at 5:56 AM, Marko Rauhamaa  wrote:
> >> I don't really like it how Unicode is equated with text, or even
> >> character strings.
> > [...]
> > Do you have actual text that you're unable to represent in Unicode?
> 
> Not my point at all.
> 
> I'm saying equating an abstract data type (string) with its
> representation (Unicode vector) is bad taste.
> 
> > We don't call numbers IEEE,
> 
> Exactly.
> 
> > Do you genuinely have text that you can't represent in Unicode, or are
> > you just arguing against Unicode to try to justify "Python strings are
> > <something else>" as a basis for your code?
> 
> Nobody is arguing against Unicode. I'm saying, let's talk about the
> forest instead of the trees (except when the trees really are the
> focus).

Ive always felt the makers of C showed remarkably good taste in 
the names 'int' and 'float'. Unlike:
Pascal: Int and Real
PL/1: Fixed and Float

IOW the more leaky abstraction used for real numbers is explicitly reminded.

Likewise in 2014, and given the arguments, inconsistencies, etc
remembering the nuts-n-bolts below the strings-represented-as-unicode
abstraction may be in order.

[toc] | [prev] | [next] | [standalone]

#81225

Rustom Mody <rustompmody@gmail.com>:

> Likewise in 2014, and given the arguments, inconsistencies, etc
> remembering the nuts-n-bolts below the strings-represented-as-unicode
> abstraction may be in order.

No need to hide Unicode, but talking about a

   Unicode string

is like talking about an

   electronic computer

   visible spectrum display

   mouse user interface

   ethernet socket

   magnetic file

   electric power supply

The language spec calls the things just "strings," as it should.


Marko

[toc] | [prev] | [next] | [standalone]

#81226

On Sat, Nov 22, 2014 at 3:36 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> No need to hide Unicode, but talking about a
>
>    Unicode string
>
> is like talking about an
>
>    electronic computer
>
>    visible spectrum display
>
>    mouse user interface
>
>    ethernet socket
>
>    magnetic file
>
>    electric power supply
>
> The language spec calls the things just "strings," as it should.

I'm not sure what you mean here, because the adjectives all cut out
other common constructs - a byte string, an analog computer, an IR or
UV display, a blind-compatible UI, a Unix domain socket, an in-memory
file, and a diesel power supply. Okay, I'm pushing it with the last
one (they're usually called gen sets, not power supplies), and I don't
often hear people talk about "magnetic files", but the rest are
definitely valid comparison/contrast terms.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81266

Marko Rauhamaa wrote:

> Rustom Mody <rustompmody@gmail.com>:
> 
>> Likewise in 2014, and given the arguments, inconsistencies, etc
>> remembering the nuts-n-bolts below the strings-represented-as-unicode
>> abstraction may be in order.
> 
> No need to hide Unicode, but talking about a
> 
>    Unicode string
> 
> is like talking about an
> 
>    electronic computer

versus a hydraulic computer, a mechanical computer, an optical computer, a
human computer, a genetic (DNA) computer, ... 

>    visible spectrum display

I'm not sure that many people actually do refer to "visible spectrum
display", or what you mean by it, but I can easily imagine that being in
contrast with a non-visible spectrum display.


>    mouse user interface

As opposed to a commandline user interface, direct brain-to-computer user
interface, touch UI, etc. Not to mention non-user interfaces, like SCSI
interface, SATA interface, USB interface, ...


>    ethernet socket

Telephone socket, Appletalk socket, Firewire socket, ADB socket ...


>    magnetic file

I have no idea what you mean here. Do you mean magnetic *field*? As opposed
to an electric field, gravitational field, Higgs field, strong nuclear
force field, weak nuclear force field ...


>    electric power supply
> 
> The language spec calls the things just "strings," as it should.


I really don't understand what bothers you about this. In Python, we have
Unicode strings and byte strings. In computing in general, strings can
consist of Unicode characters, ASCII characters, Tron characters, EBCDID
characters, ISO-8859-7 characters, and literally dozens of others. It
boogles my mind that you are so opposed to being explicit about what sort
of string we are dealing with.

Are you equally disturbed when people distinguish between tablespoon,
teaspoon, dessert spoon and serving spoon?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#81277

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> In Python, we have Unicode strings and byte strings.

No, you don't. You have strings and bytes:

  Textual data in Python is handled with str objects, or strings.
  Strings are immutable sequences of Unicode code points. String
  literals are written in a variety of ways: [...]

  <URL: https://docs.python.org/3/library/stdtypes.html#text-sequence-typ
  e-str>

  The core built-in types for manipulating binary data are bytes and bytearray.

  <URL: https://docs.python.org/3/library/stdtypes.html#binary-sequence-t
  ypes-bytes-bytearray-memoryview


Equivalently, I wouldn't mind "character strings" vs "byte strings".
Unicode strings is not wrong but the technical emphasis on Unicode is as
strange as a "tire car" or "rectangular door" when "car" and "door" are
what you usually mean.


Marko

[toc] | [prev] | [next] | [standalone]

#81285

Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
> 
>> In Python, we have Unicode strings and byte strings.
> 
> No, you don't. You have strings and bytes:

Python has strings of Unicode code points, a.k.a. "Unicode strings",
or "text strings", and strings of bytes, a.k.a. "byte strings". These are
the plain English descriptive names of the types "str" and "bytes". 


>   Textual data in Python is handled with str objects, or strings.
>   Strings are immutable sequences of Unicode code points. String
>   literals are written in a variety of ways: [...]

Hence, Unicode string.


>   <URL: https://docs.python.org/3/library/stdtypes.html#text-sequence-typ
>   e-str>
> 
>   The core built-in types for manipulating binary data are bytes and
>   bytearray.

Which are strings of bytes.


>   <URL: https://docs.python.org/3/library/stdtypes.html#binary-sequence-t
>   ypes-bytes-bytearray-memoryview
> 
> 
> Equivalently, I wouldn't mind "character strings" vs "byte strings".

Unicode strings are not strings of characters, except informally. Some code
points represent non-characters:

http://www.unicode.org/faq/private_use.html#nonchar1

They are strings of Unicode code points, but "code point string" is firstly
an inelegant and ugly phrase, and secondly ambiguous. What sort of code
points? Baudot codes? ASCII codes? Big5 codes? Tron codes? No, none of the
above, they are *Unicode* code points.

You haven't given any good reason for objecting to calling Unicode strings
by what they are. Maybe you think that it is an implementation detail, and
that some version of Python might suddenly and without warning change to
only supporting KOI8-R strings or GB2312 strings? If so, you are badly
mistaken. The fact that Python strings are Unicode is not an implementation
detail, it is part of the language semantics.


> Unicode strings is not wrong but the technical emphasis on Unicode is as
> strange as a "tire car" or "rectangular door" when "car" and "door" are
> what you usually mean.

"Tire car" makes no sense. "Rectangular door" makes perfect sense, and in a
world where there are dozens of legacy non-rectangular doors, it would be
very sensible to specify the kind of door. Just as we specify sliding door,
glass door, security door, fire door, flyscreen wire door, and so on.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#81286

On Sun, Nov 23, 2014 at 12:50 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> "Tire car" makes no sense. "Rectangular door" makes perfect sense, and in a
> world where there are dozens of legacy non-rectangular doors, it would be
> very sensible to specify the kind of door. Just as we specify sliding door,
> glass door, security door, fire door, flyscreen wire door, and so on.

Not just legacy - scifi often has non-rectangular doors. (And they're
often HORRIBLY impractical. I think the rectangular door is here to
stay.) But English is a strange beast. A glass door is made of
glass... a flyscreen wire door is made of (at least, has a significant
component of) flyscreen, but a fire door isn't made of fire...

ChrisA

[toc] | [prev] | [next] | [standalone]

#81287

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> You haven't given any good reason for objecting to calling Unicode
> strings by what they are. Maybe you think that it is an implementation
> detail, and that some version of Python might suddenly and without
> warning change to only supporting KOI8-R strings or GB2312 strings? If
> so, you are badly mistaken. The fact that Python strings are Unicode
> is not an implementation detail, it is part of the language semantics.

To me, repeating the word Unicode everywhere is giving the (in and of
itself impressive) standard too primary a status. While understanding
how Unicode, IEEE-754, 2's complement, mark-and-sweep etc work is very
useful and occasionally can be taken explicit advantage of, those really
are mundane techniques to implement abstractions.

Python's strings exist (primarily) so you can express utterances in a
human language, aka plain text. They don't exist to express Unicode code
points. That would be putting the cart before the horse.

> "Rectangular door" makes perfect sense, and in a world where there are
> dozens of legacy non-rectangular doors, it would be very sensible to
> specify the kind of door.

It makes sense, and yet, I've never heard anyone talk about rectangular
doors even though I use numerous doors every day. Why is it, then, that
people feel the constant need to add the "Unicode" epithet to Python's
strings, which -- according to its own specification -- are just
strings?


Marko

[toc] | [prev] | [next] | [standalone]

#81288

In article <87y4r348uf.fsf@elektro.pacujo.net>,
 Marko Rauhamaa <marko@pacujo.net> wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
> 
> > You haven't given any good reason for objecting to calling Unicode
> > strings by what they are. Maybe you think that it is an implementation
> > detail, and that some version of Python might suddenly and without
> > warning change to only supporting KOI8-R strings or GB2312 strings? If
> > so, you are badly mistaken. The fact that Python strings are Unicode
> > is not an implementation detail, it is part of the language semantics.
> 
> To me, repeating the word Unicode everywhere is giving the (in and of
> itself impressive) standard too primary a status. While understanding
> how Unicode, IEEE-754, 2's complement, mark-and-sweep etc work is very
> useful and occasionally can be taken explicit advantage of, those really
> are mundane techniques to implement abstractions.
> 
> Python's strings exist (primarily) so you can express utterances in a
> human language, aka plain text. They don't exist to express Unicode code
> points. That would be putting the cart before the horse.
> 
> > "Rectangular door" makes perfect sense, and in a world where there are
> > dozens of legacy non-rectangular doors, it would be very sensible to
> > specify the kind of door.
> 
> It makes sense, and yet, I've never heard anyone talk about rectangular
> doors even though I use numerous doors every day. Why is it, then, that
> people feel the constant need to add the "Unicode" epithet to Python's
> strings, which -- according to its own specification -- are just
> strings?
> 
> 
> Marko

There's a old joke to the effect that the fields of study which are 
confident that they're really doing science (i.e. chemistry, biology, 
physics, astronomy, etc) don't put the word "science" in their names.  
It's only the fields of study that are less confident about their status 
as sciences (computer science, behavioral science, political science, 
etc) that feel the need to explicitly say "science".  As if repeating it 
enough times makes it true.  I wonder if something of the same thing 
applies here?  <ducking and running>

Somewhat more seriously, the IEEE-754 point is quite apropos.  Back when 
754 first came out, there were lots of different floating point 
implementations.  Machines that used 754 touted it in their sales 
literature and mentioned it all over their documentation.  These days, 
754 is so ubiquitous, nobody even thinks to mention it, in the same way 
nobody bothers to mention 2's complement integers.  I suspect that some 
day, the same thing will happen with Unicode.  For that matter, we will 
eventually get to the point where when people say, "just plain text", 
they will mean Unicode, in the same way that "just plain text" today 
really means ASCII (and the text/plain MIME type will become a 
historical curiosity).

[toc] | [prev] | [next] | [standalone]

#81289

Roy Smith <roy@panix.com>:

> For that matter, we will eventually get to the point where when people
> say, "just plain text", they will mean Unicode, in the same way that
> "just plain text" today really means ASCII (and the text/plain MIME
> type will become a historical curiosity).

MIME has:

   Content-Type: text/plain; charset="UTF-8"

(even though UTF-8 isn't a character set but a content encoding).


Marko

[toc] | [prev] | [next] | [standalone]

#81290

Le samedi 22 novembre 2014 15:18:43 UTC+1, Marko Rauhamaa a écrit :
> 
> 
> To me, repeating the word Unicode everywhere ...

When I read the PEP393 (?) Flexible String Representation
(and I re-read it many times) I noticed these brave people
were not able to discuss Unicode with a correct unicode
terminology.

Other curiosities:
- I had once to inform a French py dev with commit access,
one can not write French with Latin-1 !
- By chance, I found on the web a German py dev who
was commenting and he had not an updated "DUDEN" (a
German dictionnary).

+ plenty of goodies like these.

And it continues ...

jmf

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | comp.lang.python

csiph-web