Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #194435
| From | <tomas@tuxteam.de> |
|---|---|
| Newsgroups | linux.debian.user |
| Subject | Re: utf |
| Date | 2018-04-03 11:30 +0200 |
| Message-ID | <vAqql-5Zj-7@gated-at.bofh.it> (permalink) |
| References | <vzLQd-4cg-1@gated-at.bofh.it> <vzRsC-8m1-3@gated-at.bofh.it> <vA2o1-6N4-3@gated-at.bofh.it> <vApXj-5wU-1@gated-at.bofh.it> <vAqgG-5VJ-23@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Tue, Apr 03, 2018 at 09:14:22PM +1200, Richard Hector wrote:
> On 03/04/18 20:55, Darac Marjal wrote:
> > If these things matter to you, it's better to convert from UTF-8 to
> > Unicode, first. I tend to think of Unicode as an arbitrarily large code
> > page. Each character maps to a number, but that number could be 1, 1000
> > or 500_000 (Unicode seems to be growing without might end in sight).
> > Internally, you might store those code points as Integers or QUad Words
> > or whatever you like. Only once you're ready to transfer the text to
> > another process (print on screen, save to a file, stream across a
> > network), do you convert the Unicode back into UTF-8.
> >
> > Basically, you consider UTF-8 to be a transfer-only format (like
> > Base64). If you want to do anything non-trivial with it, decode it into
> > Unicode.
>
> Eh? UTF-8 is an encoding of Unicode. You can't "convert UTF-8 to
> Unicode" - it already is Unicode. You could convert it to another
> encoding, eg UTF-16 or UTF-32. Perhaps UTF-32 is what you mean, being
> fixed-width.
I think Darac was talking about UTF-32 [1], which is a fixed-width encoding
of Unicode. Yes, Unicode is strictly speaking the abstract "mapping" between
integers ("code points") and characters. A computer has no integers...
What's curious is that there's no UTF-24 (although Unicode currently has
all its code points below 2^21). That would make for a slightly more
compact fixed-width encoding.
I think these days fixed-width encodings are losing their charm a bit,
since memory access is getting much more expensive than CPU power.
Things might change once again when the Chinese dominate culturally,
since UTF-8 plays its advantage only with ASCII dominated text.
But perhaps then, another encoding will make more sense. Or just UTF-24
is born, for a 25% savings :-)
Cheers
[1] https://en.wikipedia.org/wiki/UTF-32
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iEYEARECAAYFAlrDST0ACgkQBcgs9XrR2kadzQCeO8N1Kjua/p0aOdfE8QQTvv6R
PisAn0+0DbwWXb+cWYsUmMwqSqQN+BKQ
=y+Fu
-----END PGP SIGNATURE-----
Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
utf mess-mate <mess-mate@gmx.com> - 2018-04-01 16:10 +0200
Re: utf Curt <curty@free.fr> - 2018-04-01 17:50 +0200
Re: utf Ionel Mugurel Ciobîcă <I.M.Ciobica@upcmail.nl> - 2018-04-01 18:30 +0200
Re: utf Andre Majorel <aym-naibed@teaser.fr> - 2018-04-01 18:40 +0200
Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-01 22:10 +0200
Re: utf Cindy-Sue Causey <butterflybytes@gmail.com> - 2018-04-02 00:50 +0200
Re: utf Curt <curty@free.fr> - 2018-04-02 09:50 +0200
Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-02 11:40 +0200
Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-02 15:30 +0200
Re: utf Andre Majorel <aym-naibed@teaser.fr> - 2018-04-02 09:50 +0200
Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 14:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-02 15:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-02 15:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 19:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-02 20:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-02 20:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) tomas@tuxteam.de - 2018-04-02 21:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 21:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 00:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-03 13:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-03 14:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-03 14:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 14:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 14:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-03 14:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 21:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 21:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 22:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 14:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 03:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 03:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 13:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 14:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 14:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 14:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 15:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Andre Majorel <aym-naibed@teaser.fr> - 2018-04-04 16:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 16:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 18:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 18:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 20:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 14:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 15:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Don Armstrong <don@debian.org> - 2018-04-04 19:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 19:50 +0200
mbox vs maildir vs better formats [Re: Invalid UTF-8 byte? (was: Re: utf)] Don Armstrong <don@debian.org> - 2018-04-04 20:00 +0200
Re: mbox vs maildir vs better formats [Re: Invalid UTF-8 byte? (was: Re: utf)] Nicolas George <george@nsup.org> - 2018-04-04 20:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 20:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Don Armstrong <don@debian.org> - 2018-04-04 20:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-05 14:40 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) deloptes <deloptes@gmail.com> - 2018-04-05 00:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 21:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 13:20 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-04 16:10 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 20:50 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-04 21:30 +0200
Re: Invalid UTF-8 byte? Ben Caradoc-Davies <ben@transient.nz> - 2018-04-04 23:50 +0200
Re: Invalid UTF-8 byte? Michael Stone <mstone@debian.org> - 2018-04-05 01:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 21:00 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 20:30 +0200
Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 21:30 +0200
Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-02 23:40 +0200
Re: utf Darac Marjal <mailinglist@darac.org.uk> - 2018-04-03 11:00 +0200
Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-03 11:20 +0200
Re: utf <tomas@tuxteam.de> - 2018-04-03 11:30 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-03 12:20 +0200
Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-03 22:50 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-03 23:00 +0200
Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 23:00 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-03 23:30 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 19:00 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:10 +0200
Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 19:30 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:40 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 19:40 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:40 +0200
Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 19:50 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-04 20:00 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 23:20 +0200
Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-05 02:30 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-05 14:10 +0200
Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 00:00 +0200
Re: utf rhkramer@gmail.com - 2018-04-04 20:30 +0200
Re: utf Joel Roth <joelz@pobox.com> - 2018-04-04 21:30 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 23:40 +0200
Re: utf <tomas@tuxteam.de> - 2018-04-05 08:30 +0200
Re: utf rhkramer@gmail.com - 2018-04-05 14:50 +0200
Re: utf <tomas@tuxteam.de> - 2018-04-05 15:00 +0200
Re: utf Nicolas George <george@nsup.org> - 2018-04-05 15:00 +0200
Re: utf tomas@tuxteam.de - 2018-04-05 15:10 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-05 21:40 +0200
Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 23:30 +0200
Re: utf deloptes <deloptes@gmail.com> - 2018-04-05 23:40 +0200
Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-06 03:20 +0200
Re: utf <tomas@tuxteam.de> - 2018-04-06 09:00 +0200
Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-06 04:20 +0200
Re: utf <tomas@tuxteam.de> - 2018-04-06 09:10 +0200
Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 19:00 +0200
Re: utf rhkramer@gmail.com - 2018-04-05 20:00 +0200
Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-03 23:40 +0200
Re: utf Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 14:20 +0200
csiph-web