Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #81089 > unrolled thread

Re: python 2.7 and unicode (one more time)

Started byMichael Torrie <torriem@gmail.com>
First post2014-11-20 09:51 -0700
Last post2014-11-21 12:39 +1100
Articles 20 on this page of 62 — 13 participants

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
                  Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
          Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
                      Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
                        Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
                          Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
                          Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
                  Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
                      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
                          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
                            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
                            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
                              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
                                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
                                  Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
      Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
            Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
        Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →


#81291

FromRustom Mody <rustompmody@gmail.com>
Date2014-11-22 07:57 -0800
Message-ID<1e79c8d1-f574-4d38-a826-972175e30d27@googlegroups.com>
In reply to#81290
On Saturday, November 22, 2014 8:14:15 PM UTC+5:30, Roy Smith wrote:
>  Marko Rauhamaa wrote:
> 
> > Steven D'Aprano:
> > 
> > > You haven't given any good reason for objecting to calling Unicode
> > > strings by what they are. Maybe you think that it is an implementation
> > > detail, and that some version of Python might suddenly and without
> > > warning change to only supporting KOI8-R strings or GB2312 strings? If
> > > so, you are badly mistaken. The fact that Python strings are Unicode
> > > is not an implementation detail, it is part of the language semantics.
> > 
> > To me, repeating the word Unicode everywhere is giving the (in and of
> > itself impressive) standard too primary a status. While understanding
> > how Unicode, IEEE-754, 2's complement, mark-and-sweep etc work is very
> > useful and occasionally can be taken explicit advantage of, those really
> > are mundane techniques to implement abstractions.
> > 
> > Python's strings exist (primarily) so you can express utterances in a
> > human language, aka plain text. They don't exist to express Unicode code
> > points. That would be putting the cart before the horse.
> > 
> > > "Rectangular door" makes perfect sense, and in a world where there are
> > > dozens of legacy non-rectangular doors, it would be very sensible to
> > > specify the kind of door.
> > 
> > It makes sense, and yet, I've never heard anyone talk about rectangular
> > doors even though I use numerous doors every day. Why is it, then, that
> > people feel the constant need to add the "Unicode" epithet to Python's
> > strings, which -- according to its own specification -- are just
> > strings?
> > 
> > 
> > Marko
> 
> There's a old joke to the effect that the fields of study which are 
> confident that they're really doing science (i.e. chemistry, biology, 
> physics, astronomy, etc) don't put the word "science" in their names.  
> It's only the fields of study that are less confident about their status 
> as sciences (computer science, behavioral science, political science, 
> etc) that feel the need to explicitly say "science".  As if repeating it 
> enough times makes it true.  I wonder if something of the same thing 
> applies here?  <ducking and running>
> 
> Somewhat more seriously, the IEEE-754 point is quite apropos.  Back when 
> 754 first came out, there were lots of different floating point 
> implementations.  Machines that used 754 touted it in their sales 
> literature and mentioned it all over their documentation.  These days, 
> 754 is so ubiquitous, nobody even thinks to mention it, in the same way 
> nobody bothers to mention 2's complement integers.  I suspect that some 
> day, the same thing will happen with Unicode.  For that matter, we will 
> eventually get to the point where when people say, "just plain text", 
> they will mean Unicode, in the same way that "just plain text" today 
> really means ASCII (and the text/plain MIME type will become a 
> historical curiosity).

Yes this was my point also -- encodings in general and unicode in
particular is a mess (as of 2014).  Maybe in a few years the dust 
will settle.  Then saying 'unicode' will become redundant.
But until then when we have a rather leaky abstraction having
sealing liquid on the hands is preferable to sewage in the house.

[toc] | [prev] | [next] | [standalone]


#81293

FromMarko Rauhamaa <marko@pacujo.net>
Date2014-11-22 19:49 +0200
Message-ID<87mw7j3z2r.fsf@elektro.pacujo.net>
In reply to#81290
wxjmfauth@gmail.com:

> - By chance, I found on the web a German py dev who was commenting and
> he had not an updated "DUDEN" (a German dictionnary).

That... leaves me utterly speachless!


Marko

[toc] | [prev] | [next] | [standalone]


#81294

FromMark Lawrence <breamoreboy@yahoo.co.uk>
Date2014-11-22 18:17 +0000
Message-ID<mailman.16141.1416680242.18130.python-list@python.org>
In reply to#81293
On 22/11/2014 17:49, Marko Rauhamaa wrote:
> wxjmfauth@gmail.com:
>
>> - By chance, I found on the web a German py dev who was commenting and
>> he had not an updated "DUDEN" (a German dictionnary).
>
> That... leaves me utterly speachless!
>
>
> Marko
>

Please don't feed him.  Your average troll is bad enough but he really 
takes the biscuit.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]


#81298

Fromwxjmfauth@gmail.com
Date2014-11-22 11:32 -0800
Message-ID<b75a1ad1-143c-476b-a6e8-3866fa10226a@googlegroups.com>
In reply to#81294
Le samedi 22 novembre 2014 19:17:33 UTC+1, Mark Lawrence a écrit :
> On 22/11/2014 17:49, Marko Rauhamaa wrote:
> > wxjmfauth@gmail.com:
> >
> >> - By chance, I found on the web a German py dev who was commenting and
> >> he had not an updated "DUDEN" (a German dictionnary).
> >
> > That... leaves me utterly speachless!
> >
> >
> > Marko
> >
> 
> Please don't feed him.  Your average troll is bad enough but he really 
> takes the biscuit.
> 

German again. It's no a too dramatic case.

>>> timeit.timeit("('a'*1000 + '\u1e9e')[:-1]")
2.197046143133785
>>> timeit.timeit("('a'*1000 + 'z')[:-1]")
0.9829807244318012

Py 32 does better.

jmf

[toc] | [prev] | [next] | [standalone]


#81297

Fromwxjmfauth@gmail.com
Date2014-11-22 11:17 -0800
Message-ID<d12bc043-f225-41b3-b091-c1cc8247c585@googlegroups.com>
In reply to#81293
Le samedi 22 novembre 2014 18:49:43 UTC+1, Marko Rauhamaa a écrit :
> wxjmfauth@gmail.com:
> 
> > - By chance, I found on the web a German py dev who was commenting and
> > he had not an updated "DUDEN" (a German dictionnary).
> 
> That... leaves me utterly speachless!

Nothin wrong.

----

As a real experience, I recommended in a private mail
exchage a Py core dev to test this wonderful new Python
with its unicode handling with the set of characters,
Germany is officaly using or will use. (Technically Latin
is quite a hard chars set).

[...]

A link in English:
http://ilovetypography.com/2014/07/12/bundessans-and-bundesserif-truly-democratic-typefaces/

Far different from a naive way to present Python/Unicode with a few
simple Chinese or Japanese chars.

jmf

[toc] | [prev] | [next] | [standalone]


#81301

FromChris Angelico <rosuav@gmail.com>
Date2014-11-23 07:17 +1100
Message-ID<mailman.16143.1416687478.18130.python-list@python.org>
In reply to#81293
On Sun, Nov 23, 2014 at 5:17 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> Please don't feed him.  Your average troll is bad enough but he really takes
> the biscuit.

... someone was feeding him biscuits?

ChrisA

[toc] | [prev] | [next] | [standalone]


#81304

FromMark Lawrence <breamoreboy@yahoo.co.uk>
Date2014-11-22 22:04 +0000
Message-ID<mailman.16146.1416693857.18130.python-list@python.org>
In reply to#81293
On 22/11/2014 20:17, Chris Angelico wrote:
> On Sun, Nov 23, 2014 at 5:17 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>> Please don't feed him.  Your average troll is bad enough but he really takes
>> the biscuit.
>
> ... someone was feeding him biscuits?
>
> ChrisA
>

Surely it's better than feeding him unicode?

As I needed cheering up I ventured over to gg and wasn't disappointed 
reading his latest rubbish. My favourite "find thousand and one ways to 
make Python crashing or failing." but I don't recall a single bug report 
in the last two years from anybody regarding problems with the FSR, or 
have I missed something?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]


#81305

FromChris Angelico <rosuav@gmail.com>
Date2014-11-23 09:31 +1100
Message-ID<mailman.16147.1416695512.18130.python-list@python.org>
In reply to#81293
On Sun, Nov 23, 2014 at 9:04 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> My favourite "find thousand and one ways to make Python crashing or
> failing." but I don't recall a single bug report in the last two years from
> anybody regarding problems with the FSR, or have I missed something?

What you've missed is the grammar of the sentence you've (partially)
quoted. Clearly he is seeking to make Python, and he is crashing or
failing. My advice to him: Stop trying to build complex software while
in command of a car.

ChrisA

[toc] | [prev] | [next] | [standalone]


#81306

FromMark Lawrence <breamoreboy@yahoo.co.uk>
Date2014-11-22 23:38 +0000
Message-ID<mailman.16148.1416699516.18130.python-list@python.org>
In reply to#81293
On 22/11/2014 22:31, Chris Angelico wrote:
> On Sun, Nov 23, 2014 at 9:04 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>> My favourite "find thousand and one ways to make Python crashing or
>> failing." but I don't recall a single bug report in the last two years from
>> anybody regarding problems with the FSR, or have I missed something?
>
> What you've missed is the grammar of the sentence you've (partially)
> quoted. Clearly he is seeking to make Python, and he is crashing or
> failing. My advice to him: Stop trying to build complex software while
> in command of a car.
>
> ChrisA
>


What?  The entire message follows.

<quote>
I think you are not understanding the point very well.

Py32 and Qt derivative + plenty of dirty tricks.
(It will probably not be rendered correctly.)

Write something like this (an interactive interpreter)
in Py32 and Py33 and see what happens:

 >>> print(999)
999
 >>> sys.version
'3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)]'
 >>> # note the emoji and the private use area (plane 15)
 >>> a = 'abc\u00e9\u0153\u20ac\u1e9e\U0001f300\udb80\udc00z'
 >>> print(a)
abc需ẞ🌀󰀀z
 >>>

Note: it can be "cut/copied/pasted" with a MS product.

jmf

PS I have to recognized, I'm slowly getting tired to
find thousand and one ways to make Python crashing
or failing.
</quote>

That is a standard Windows build. He is again conflating problems with 
using the Windows command line for a given code page with the FSR.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]


#81363

Fromwxjmfauth@gmail.com
Date2014-11-23 04:52 -0800
Message-ID<9d17bd95-6dc2-4a39-9d2d-8243ee5020bd@googlegroups.com>
In reply to#81306
Le dimanche 23 novembre 2014 00:38:53 UTC+1, Mark Lawrence a écrit :
> On 22/11/2014 22:31, Chris Angelico wrote:
> > On Sun, Nov 23, 2014 at 9:04 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> >> My favourite "find thousand and one ways to make Python crashing or
> >> failing." but I don't recall a single bug report in the last two years from
> >> anybody regarding problems with the FSR, or have I missed something?
> >
> > What you've missed is the grammar of the sentence you've (partially)
> > quoted. Clearly he is seeking to make Python, and he is crashing or
> > failing. My advice to him: Stop trying to build complex software while
> > in command of a car.
> >
> > ChrisA
> >
> 
> 
> What?  The entire message follows.
> 
> <quote>
> I think you are not understanding the point very well.
> 
> Py32 and Qt derivative + plenty of dirty tricks.
> (It will probably not be rendered correctly.)
> 
> Write something like this (an interactive interpreter)
> in Py32 and Py33 and see what happens:
> 
>  >>> print(999)
> 999
>  >>> sys.version
> '3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)]'
>  >>> # note the emoji and the private use area (plane 15)
>  >>> a = 'abc\u00e9\u0153\u20ac\u1e9e\U0001f300\udb80\udc00z'
>  >>> print(a)
> abc需ẞ🌀󰀀z
>  >>>
> 
> Note: it can be "cut/copied/pasted" with a MS product.
> 
> jmf
> 
> PS I have to recognized, I'm slowly getting tired to
> find thousand and one ways to make Python crashing
> or failing.
> </quote>
> 
> That is a standard Windows build. He is again conflating problems with 
> using the Windows command line for a given code page with the FSR.
> 

Certainly not. It is a GUI application.

jmf

[toc] | [prev] | [next] | [standalone]


#81309

Fromrandom832@fastmail.us
Date2014-11-22 20:52 -0500
Message-ID<mailman.16150.1416707559.18130.python-list@python.org>
In reply to#81293
On Sat, Nov 22, 2014, at 18:38, Mark Lawrence wrote:
> ...
> That is a standard Windows build. He is again conflating problems with 
> using the Windows command line for a given code page with the FSR.

The thing is, with a truetype font selected, a correctly written win32
console problem should be able to print any character without caring
about codepages (via use of WriteConsoleW instead of WriteFile). You
cannot rely on having the codepage set to 65001, especially since 65001
isn't actually a fully supported codepage.

In my opinion it is a deficiency in the win32 support, rather than
unicode support (and certainly nothing to do with the FSR), but it _is_
a deficiency.

[toc] | [prev] | [next] | [standalone]


#81312

FromChris Angelico <rosuav@gmail.com>
Date2014-11-23 13:11 +1100
Message-ID<mailman.16152.1416708725.18130.python-list@python.org>
In reply to#81293
On Sun, Nov 23, 2014 at 12:52 PM,  <random832@fastmail.us> wrote:
> On Sat, Nov 22, 2014, at 18:38, Mark Lawrence wrote:
>> ...
>> That is a standard Windows build. He is again conflating problems with
>> using the Windows command line for a given code page with the FSR.
>
> The thing is, with a truetype font selected, a correctly written win32
> console problem should be able to print any character without caring
> about codepages (via use of WriteConsoleW instead of WriteFile). You
> cannot rely on having the codepage set to 65001, especially since 65001
> isn't actually a fully supported codepage.

Is that true? Does WriteConsoleW support every Unicode character? It's
not obvious from the docs whether it uses UCS-2 or UTF-16 (or maybe
something else).

ChrisA

[toc] | [prev] | [next] | [standalone]


#81318

Fromrandom832@fastmail.us
Date2014-11-22 22:10 -0500
Message-ID<mailman.16155.1416712258.18130.python-list@python.org>
In reply to#81293
On Sat, Nov 22, 2014, at 21:11, Chris Angelico wrote:
> Is that true? Does WriteConsoleW support every Unicode character? It's
> not obvious from the docs whether it uses UCS-2 or UTF-16 (or maybe
> something else).

I was defining "every unicode character" loosely. There are certainly
display problems (there are display problems with wide characters on
non-CJK windows versions, too), but if you write a surrogate pair,
you'll get something that can copy to the clipboard as a surrogate pair,
and get the same thing that writing a non-BMP UTF-8 character with
codepage 65001 will give you. And you certainly won't get an error.

[toc] | [prev] | [next] | [standalone]


#81383

FromDennis Lee Bieber <wlfraed@ix.netcom.com>
Date2014-11-23 11:33 -0500
Message-ID<mailman.16194.1416760440.18130.python-list@python.org>
In reply to#81293
On Sat, 22 Nov 2014 20:52:37 -0500, random832@fastmail.us declaimed the
following:

>On Sat, Nov 22, 2014, at 18:38, Mark Lawrence wrote:
>> ...
>> That is a standard Windows build. He is again conflating problems with 
>> using the Windows command line for a given code page with the FSR.
>
>The thing is, with a truetype font selected, a correctly written win32
>console problem should be able to print any character without caring

	Why would that be possible? Many truetype fonts only supply glyphs for
single-byte encodings (ISO-Latin-1, for example -- pop up the Windows
character map utility and see what some of the font files contain.

	Heck -- on my current machine, the True Type fonts are all old
third-party items. All the standard fonts are now Open Type.

	The only font that appears to have "full" Unicode is "Arial MS Unicode"
-- regular Arial family looks to be Latin-1, Cyrillic and/or Greek, Hebrew,
various Arabic squiggles (I don't know how many variations of similar
squiggles count as separate languages).
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]


#81385

FromChris Angelico <rosuav@gmail.com>
Date2014-11-24 03:39 +1100
Message-ID<mailman.16196.1416760756.18130.python-list@python.org>
In reply to#81293
On Mon, Nov 24, 2014 at 3:33 AM, Dennis Lee Bieber
<wlfraed@ix.netcom.com> wrote:
> On Sat, 22 Nov 2014 20:52:37 -0500, random832@fastmail.us declaimed the
> following:
>
>>On Sat, Nov 22, 2014, at 18:38, Mark Lawrence wrote:
>>> ...
>>> That is a standard Windows build. He is again conflating problems with
>>> using the Windows command line for a given code page with the FSR.
>>
>>The thing is, with a truetype font selected, a correctly written win32
>>console problem should be able to print any character without caring
>
>         Why would that be possible? Many truetype fonts only supply glyphs for
> single-byte encodings (ISO-Latin-1, for example -- pop up the Windows
> character map utility and see what some of the font files contain.

A program should be able to print those characters even if they all
look identical. Chances are you can copy and paste them into something
else. But yes, finding a suitable font that covers the whole Unicode
range is *hard*. I've struggled with this one with a few programs (and
I still haven't managed to get VLC to satisfactorily display subtitles
that include Chinese characters).

ChrisA

[toc] | [prev] | [next] | [standalone]


#81397

Fromrandom832@fastmail.us
Date2014-11-23 13:13 -0500
Message-ID<mailman.16203.1416766415.18130.python-list@python.org>
In reply to#81293
On Sun, Nov 23, 2014, at 11:33, Dennis Lee Bieber wrote:
> 	Why would that be possible? Many truetype fonts only supply glyphs for
> single-byte encodings (ISO-Latin-1, for example -- pop up the Windows
> character map utility and see what some of the font files contain.

With a bitmap font selected, the characters will be immediately replaced
with characters present in the font's codepage, and will copy to
clipboard as such.

With a truetype font (Lucida Console or Consolas) selected, the
characters will be displayed as replacement glyphs (box with a question
mark in it) if not present in the font, but *will still copy to the
clipboard as the original code point* (which you might notice is where
we started, with someone claiming success by being able to do so with
codepage 65001 selected). And in any case, all characters that *are* in
the font will work and display correctly, rather than only those in the
OEM codepage.

> 	Heck -- on my current machine, the True Type fonts are all old
> third-party items. All the standard fonts are now Open Type.

The win32 console's configuration UI refers to opentype fonts as
truetype. Opentype fonts can use either truetype or type 1 as the
underlying format, and all opentype fonts supplied with windows use
truetype. You are being excessively pedantic in objecting to my use of
the term "truetype".

[toc] | [prev] | [next] | [standalone]


#81401

FromDave Angel <d@davea.name>
Date2014-11-23 15:31 -0500
Message-ID<mailman.16207.1416774684.18130.python-list@python.org>
In reply to#81293
On 11/23/2014 01:13 PM, random832@fastmail.us wrote:
> On Sun, Nov 23, 2014, at 11:33, Dennis Lee Bieber wrote:
>> 	Why would that be possible? Many truetype fonts only supply glyphs for
>> single-byte encodings (ISO-Latin-1, for example -- pop up the Windows
>> character map utility and see what some of the font files contain.
>
> With a bitmap font selected, the characters will be immediately replaced
> with characters present in the font's codepage, and will copy to
> clipboard as such.

I didn't realize Windows shell (DOS box) had that bug.  Course I don't 
use Windows much the last few years.

it's one thing to not display it properly.  It's quite another to supply 
faulty data to the clipboard.  Especially since the Windows clipboard 
has a separate Unicode type available.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]


#81403

FromChris Angelico <rosuav@gmail.com>
Date2014-11-24 09:22 +1100
Message-ID<mailman.16208.1416781370.18130.python-list@python.org>
In reply to#81293
On Mon, Nov 24, 2014 at 7:31 AM, Dave Angel <d@davea.name> wrote:
> On 11/23/2014 01:13 PM, random832@fastmail.us wrote:
>>
>> On Sun, Nov 23, 2014, at 11:33, Dennis Lee Bieber wrote:
>>>
>>>         Why would that be possible? Many truetype fonts only supply
>>> glyphs for
>>> single-byte encodings (ISO-Latin-1, for example -- pop up the Windows
>>> character map utility and see what some of the font files contain.
>>
>>
>> With a bitmap font selected, the characters will be immediately replaced
>> with characters present in the font's codepage, and will copy to
>> clipboard as such.
>
>
> I didn't realize Windows shell (DOS box) had that bug.  Course I don't use
> Windows much the last few years.

Likewise. I've been accustomed to copying and pasting unrecognized
characters (one of the easiest solutions is to paste them into a
Python console - ord() for one character, or a Py2 repr() for multiple
- to quickly see what the codepoints are), relying on the clipboard
getting the exact same sequence that was printed by the application.
Thanks, Windows, just what I always wanted to hear.

ChrisA

[toc] | [prev] | [next] | [standalone]


#81411

FromDennis Lee Bieber <wlfraed@ix.netcom.com>
Date2014-11-23 20:24 -0500
Message-ID<mailman.16214.1416792276.18130.python-list@python.org>
In reply to#81293
On Sun, 23 Nov 2014 13:13:32 -0500, random832@fastmail.us declaimed the
following:

>The win32 console's configuration UI refers to opentype fonts as
>truetype. Opentype fonts can use either truetype or type 1 as the
>underlying format, and all opentype fonts supplied with windows use
>truetype. You are being excessively pedantic in objecting to my use of
>the term "truetype".

	I was commenting on what I found displayed in the Character Map
utility... Only a few of my third party fonts had the icon for TrueType;
all the Window's supplied fonts had the icon for OpenType (and then there
are the PostScript fonts from other places -- someday I need to pull my
WinXP box out of storage and reboot it to see if I can find my copy of
Critter; a PostScript font I actually paid money for, but seem to not have
transferred to my newest computers).
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]


#81427

Fromrandom832@fastmail.us
Date2014-11-24 00:05 -0500
Message-ID<mailman.16223.1416805510.18130.python-list@python.org>
In reply to#81293
On Sun, Nov 23, 2014, at 15:31, Dave Angel wrote:
> I didn't realize Windows shell (DOS box) had that bug.  Course I don't 
> use Windows much the last few years.
> 
> it's one thing to not display it properly.  It's quite another to supply 
> faulty data to the clipboard.  Especially since the Windows clipboard 
> has a separate Unicode type available.

It's because console bitmap fonts almost always (always?) only have one
codepage's worth of characters, and it's considered better to display A
for U+0100 than a blank space, and the clipboard has always been a bit
of an afterthought for the windows console. Meanwhile, a truetype font
is considered likely to have real glyphs for most characters a user
would want to display, so no conversion is done. And there's no font
rendering routine for bitmap fonts that will allow for dynamic
substitution of glyphs, so it becomes a real A (or whatever) in the
console buffer itself - this isn't a conversion done at clipboard-copy
time.

[toc] | [prev] | [next] | [standalone]


Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.lang.python


csiph-web