Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #6378

Re: UTF-8 and strings

From Nobody <nobody@nowhere.com>
Subject Re: UTF-8 and strings
Date 2011-06-07 21:35 +0100
Message-Id <pan.2011.06.07.20.35.15.703000@nowhere.com>
Newsgroups comp.lang.c++
References <c51336b5-cc02-4db9-8f7a-31792552a2da@c41g2000yqm.googlegroups.com>
Organization Zen Internet

Show all headers | View raw

On Tue, 07 Jun 2011 08:08:02 -0700, John M. Dlugosz wrote:

> But std::string models a sequence of *characters*, and that doesn't
> fit with multi-byte variable-length character encodings of any kind.
> Furthermore, one character won't fit in a char.
> 
> Before I get in too deep with my thoughts, I want to see what others
> are doing.  Is there any existing information on this topic?  Is there
> a better place to discuss it?

If you want to manipulate "characters", then use std::wstring internally.
For converting between std::string and std::wstring, possible approaches
include:

1. Writing the code yourself (simple if you only use UTF-8)
2. mbstowcs and wcstombs (C99, <cstdlib>).
3. iconv (Unix)
4. MultiByteToWideChar and WideCharToMultiByte (Windows)

Each has at least one drawback. 1 requires that you write the encoding
and decoding functions. 2 requires setting the locale to a UTF-8 locale
before any conversion then changing it back afterwards. 3 and 4 aren't
portable.

Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-07 08:08 -0700
  Re: UTF-8 and strings Asger-P <junk@asger-p.dk> - 2011-06-07 17:26 +0200
    Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-08 17:58 -0700
      Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 18:21 -0700
        Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-09 06:38 -0700
  Re: UTF-8 and strings "Balog Pal" <pasa@lib.hu> - 2011-06-07 17:35 +0200
    Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-08 18:08 -0700
  Re: UTF-8 and strings Jeff Flinn <TriumphSprint2000@hotmail.com> - 2011-06-07 12:55 -0400
    Re: UTF-8 and strings Paavo Helde <myfirstname@osa.pri.ee> - 2011-06-07 15:29 -0500
      Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-08 07:15 -0500
        Re: UTF-8 and strings yatremblay@bel1lin202.(none) (Yannick Tremblay) - 2011-06-08 15:45 +0000
          Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-08 12:58 -0500
            Re: UTF-8 and strings yatremblay@bel1lin202.(none) (Yannick Tremblay) - 2011-06-09 09:18 +0000
              Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-09 07:36 -0500
                Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-10 13:18 -0700
                Re: UTF-8 and strings Ruben Safir <mrbrklyn@panix.com> - 2011-06-11 00:29 +0000
    Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-10 12:58 -0700
  Re: UTF-8 and strings diamondback <christopher.rau@gmail.com> - 2011-06-07 10:57 -0700
    Re: UTF-8 and strings Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-06-08 06:38 +0000
  Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-07 13:30 -0700
    Re: UTF-8 and strings Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-06-08 06:52 +0000
      Re: UTF-8 and strings Marc <marc.glisse@gmail.com> - 2011-06-08 08:38 +0000
        Re: UTF-8 and strings Miles Bader <miles@gnu.org> - 2011-06-08 18:02 +0900
          Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 02:05 -0700
            Re: UTF-8 and strings Miles Bader <miles@gnu.org> - 2011-06-08 18:11 +0900
          Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-08 07:22 -0500
        Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 02:03 -0700
          Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 02:06 -0700
        Re: UTF-8 and strings tm <thomas.mertes@gmx.at> - 2011-06-08 02:59 -0700
        Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-08 18:20 -0700
      Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 01:56 -0700
      Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-08 07:19 -0500
        Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-10 13:12 -0700
          Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-10 18:34 -0500
            Re: UTF-8 and strings Miles Bader <miles@gnu.org> - 2011-06-11 13:12 +0900
            Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-12 20:52 -0700
              Re: UTF-8 and strings ruben safir <ruben@mrbrklyn.com> - 2011-06-13 03:57 +0000
                Re: UTF-8 and strings Asger-P <junk@asger-p.dk> - 2011-06-13 10:51 +0200
    Re: UTF-8 and strings Nobody <nobody@nowhere.com> - 2011-06-08 08:01 +0100
  Re: UTF-8 and strings Nobody <nobody@nowhere.com> - 2011-06-07 21:35 +0100
    Re: UTF-8 and strings Joshua Maurice <joshuamaurice@gmail.com> - 2011-06-08 01:56 -0700
      Re: UTF-8 and strings Nobody <nobody@nowhere.com> - 2011-06-09 03:22 +0100
  Re: UTF-8 and strings yatremblay@bel1lin202.(none) (Yannick Tremblay) - 2011-06-08 15:55 +0000
    Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-08 17:55 -0700
      Re: UTF-8 and strings yatremblay@bel1lin202.(none) (Yannick Tremblay) - 2011-06-09 10:02 +0000
        Re: UTF-8 and strings "MikeP" <mp011011@some.org> - 2011-06-09 08:41 -0500
          Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-12 20:57 -0700
            Re: UTF-8 and strings ruben safir <ruben@mrbrklyn.com> - 2011-06-13 04:46 +0000
              Re: UTF-8 and strings Asger-P <junk@asger-p.dk> - 2011-06-13 11:58 +0200
        Re: UTF-8 and strings "John M. Dlugosz" <uwqejpnp92@snkmail.com> - 2011-06-09 06:52 -0700

csiph-web