Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #5994

Re: fgetwc doesn't read Unicode

Show all headers | View raw

felix.leg <felix.leg@vp.pl> wrote:
> W dniu 09.06.2011 03:39, Nobody pisze:
> > On Wed, 08 Jun 2011 13:41:37 +0200, felix.leg wrote:
> >
> > fgetwc() assumes that the file is in the locale's encoding. If the file
> > is in e.g. UTF-8, then you need to use a UTF-8 locale. This means
> > calling setlocale(LC_CTYPE, ...); if you use "" as the locale, then the
> > appropriate environment variables need to indicate a UTF-8 locale.
> [..cut..]
> > At startup, all locale categories default to "C" (which has ASCII as its
> > encoding). You have to call setlocale() to change it.

> So dealing with Unicode in C is inextricably linked with Locale?

It's not realy about C. The fundamental problem is that you have
a file in some encoding. That could be plain ASCII, Latin-1, one
of the cyrillic or chinese encodings, UTF-8, UTF-16, UTF-32 etc.
But what encoding it is can't be determined from the file (one,
at best, could try to make an educated guess by reading the whole
file and rule out some of the encodings due to some data being 
impossible and perhaps a frequency analysis). So what encoding is
fgetwc() supposed to assume when it reads from the file? It relies
on you to tell it what encoding it's to use when reading and con-
verting to the encoding used for wide chars. And that's what you
do by setting the LC_CTYPE value of the locale. That's actually
not different from telling e.g. scanf() if to expect a dot as the
decimal point or a comma instead in floating point numbers. Per
default it will expect a dot but by setting LC_NUMERIC you can
make it expect a comma, i.e to understand that 0,75 is meant to
represent three quarters.
                                Regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

fgetwc doesn't read Unicode "felix.leg" <felix.leg@vp.pl> - 2011-06-08 13:41 +0200
  Re: fgetwc doesn't read Unicode jt@toerring.de (Jens Thoms Toerring) - 2011-06-08 20:18 +0000
  Re: fgetwc doesn't read Unicode Nobody <nobody@nowhere.com> - 2011-06-09 02:39 +0100
    Re: fgetwc doesn't read Unicode "felix.leg" <felix.leg@vp.pl> - 2011-06-12 11:30 +0200
      Re: fgetwc doesn't read Unicode Nobody <nobody@nowhere.com> - 2011-06-13 02:09 +0100
      Re: fgetwc doesn't read Unicode jt@toerring.de (Jens Thoms Toerring) - 2011-06-13 21:38 +0000
  Re: fgetwc doesn't read Unicode Dirk Krause <devnull@fh-schmalkalden.de> - 2011-06-09 13:19 +0000

csiph-web