Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53880

Re: Chardet, file, ... and the Flexible String Representation

Show all headers | View raw

On 09/09/2013 08:28 AM, wxjmfauth@gmail.com wrote:
> Comment: Such differences never happen with utf.

But with utf, slicing strings is O(n) (well that's a simplification as
someone showed an algorithm that is log n), whereas a fixed-width
encoding (Latin-1, UCS-2, UCS-4) is O(1).  Do you understand what this
means?

> Complicate and full of side effects, eg :
> 
>>>> sys.getsizeof('a')
> 26
>>>> sys.getsizeof('aé')
> 39

Why on earth are you doing getsizeof?  What are you expecting to prove?
 Why are you even trying to concern yourself with implementation
details?  As a programmer you should deal with unicode.  Period.  All
you should care about is that you can properly index or slice a unicode
string and that unicode strings can be operated on at a reasonable speed.

IE string[4] should give you the character at position 4.  len(string)
should return the length of the string in *characters*.

The byte encoding used behind the scenes is of no consequence other than
speed (and you have not shown any problem with speed).

> 
> Is not a latin-1 "é" supposed to count as a latin-1 "a" ?

Of course it does.  'aé'[0] == 'a' and 'aé'[1] == 'é'.  len('aé') returns 2.

> I picked up random methods, there may be variations, basically
> this general behaviour is always expected.

Eh?  Can you point to something in the unicode spec that doesn't work?

I don't even know that much about unicode yet it's clear you're either
deliberately muddying the waters with your stupid and pointless
arguments against FCS or you don't really understand the difference
between unicode and byte encoding.  Which is it?

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Chardet, file, ... and the Flexible String Representation wxjmfauth@gmail.com - 2013-09-06 02:11 -0700
  Re: Chardet, file, ... and the Flexible String Representation Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-09-06 10:57 +0000
  Re: Chardet, file, ... and the Flexible String Representation Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-09-06 13:10 +0200
  Re: Chardet, file, ... and the Flexible String Representation Ned Batchelder <ned@nedbatchelder.com> - 2013-09-06 07:02 -0400
  Re: Chardet, file, ... and the Flexible String Representation Piet van Oostrum <piet@vanoostrum.org> - 2013-09-06 11:46 -0400
    Re: Chardet, file, ... and the Flexible String Representation Chris Angelico <rosuav@gmail.com> - 2013-09-07 02:04 +1000
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-06 12:59 -0400
    Re: Chardet, file, ... and the Flexible String Representation Chris Angelico <rosuav@gmail.com> - 2013-09-07 03:04 +1000
    Re: Chardet, file, ... and the Flexible String Representation wxjmfauth@gmail.com - 2013-09-09 07:28 -0700
      Re: Chardet, file, ... and the Flexible String Representation Ned Batchelder <ned@nedbatchelder.com> - 2013-09-09 12:38 -0400
      Re: Chardet, file, ... and the Flexible String Representation Michael Torrie <torriem@gmail.com> - 2013-09-09 11:05 -0600
        Re: Chardet, file, ... and the Flexible String Representation Steven D'Aprano <steve@pearwood.info> - 2013-09-10 04:58 +0000
      Re: Chardet, file, ... and the Flexible String Representation Terry Reedy <tjreedy@udel.edu> - 2013-09-09 16:47 -0400
      Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-10 11:36 -0400
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-09 14:34 -0400
    Re: Chardet, file, ... and the Flexible String Representation Ian Kelly <ian.g.kelly@gmail.com> - 2013-09-09 13:03 -0600
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-09 15:27 -0400
    Re: Chardet, file, ... and the Flexible String Representation Serhiy Storchaka <storchaka@gmail.com> - 2013-09-12 00:11 +0300

csiph-web