Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #13412 > unrolled thread

when is typecasting (unsigned char*) to (char*) dangerous?

Back to article view | Back to comp.lang.c

Contents

  when is typecasting (unsigned char*) to (char*) dangerous? tim <nospam@nospam.com> - 2011-11-15 19:59 +0000
    Re: when is typecasting (unsigned char*) to (char*) dangerous? Kaz Kylheku <kaz@kylheku.com> - 2011-11-15 20:51 +0000
      Re: when is typecasting (unsigned char*) to (char*) dangerous? tim <nospam@nospam.com> - 2011-11-16 16:55 +0000
        Re: when is typecasting (unsigned char*) to (char*) dangerous? Kaz Kylheku <kaz@kylheku.com> - 2011-11-16 17:54 +0000
          Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 11:27 -0800
            Re: when is typecasting (unsigned char*) to (char*) dangerous? Kaz Kylheku <kaz@kylheku.com> - 2011-11-16 19:49 +0000
              Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 11:52 -0800
                Re: when is typecasting (unsigned char*) to (char*) dangerous? Kaz Kylheku <kaz@kylheku.com> - 2011-11-16 20:06 +0000
                  Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 14:55 -0800
                    Re: when is typecasting (unsigned char*) to (char*) dangerous? Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-25 04:42 -0800
                Re: when is typecasting (unsigned char*) to (char*) dangerous? James Kuyper <jameskuyper@verizon.net> - 2011-11-16 15:15 -0500
            Re: when is typecasting (unsigned char*) to (char*) dangerous? James Kuyper <jameskuyper@verizon.net> - 2011-11-16 15:24 -0500
              Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-16 12:41 -0800
                Re: when is typecasting (unsigned char*) to (char*) dangerous? James Kuyper <jameskuyper@verizon.net> - 2011-11-16 16:02 -0500
                  Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-16 13:23 -0800
                    Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 15:32 -0800
                      Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-16 17:02 -0800
                        Re: when is typecasting (unsigned char*) to (char*) dangerous? pete <pfiland@mindspring.com> - 2011-11-16 21:38 -0500
                        Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 19:22 -0800
                          Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-16 23:08 -0800
                            Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 23:46 -0800
                              Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-17 09:15 -0800
                          Re: when is typecasting (unsigned char*) to (char*) dangerous? Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-25 04:54 -0800
            Re: when is typecasting (unsigned char*) to (char*) dangerous? Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> - 2011-11-16 21:44 +0100
        Re: when is typecasting (unsigned char*) to (char*) dangerous? Kenneth Brody <kenbrody@spamcop.net> - 2011-11-16 13:17 -0500
    Re: when is typecasting (unsigned char*) to (char*) dangerous? Nick Keighley <nick_keighley_nospam@hotmail.com> - 2011-11-16 00:58 -0800
    Re: when is typecasting (unsigned char*) to (char*) dangerous? Malcolm McLean <malcolm.mclean5@btinternet.com> - 2011-11-16 01:36 -0800
      Re: when is typecasting (unsigned char*) to (char*) dangerous? Markus Wichmann <nullplan@gmx.net> - 2011-11-16 13:28 +0100
      Re: when is typecasting (unsigned char*) to (char*) dangerous? nroberts <roberts.noah@gmail.com> - 2011-11-16 08:14 -0800
        Re: when is typecasting (unsigned char*) to (char*) dangerous? BGB <cr88192@hotmail.com> - 2011-11-16 12:03 -0700
    Re: when is typecasting (unsigned char*) to (char*) dangerous? blp@cs.stanford.edu (Ben Pfaff) - 2011-11-16 08:58 -0800
      Re: when is typecasting (unsigned char*) to (char*) dangerous? Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> - 2011-11-16 19:04 +0100
        Re: when is typecasting (unsigned char*) to (char*) dangerous? Ben Pfaff <blp@cs.stanford.edu> - 2011-11-16 10:05 -0800
        Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-16 11:19 -0800
          Re: when is typecasting (unsigned char*) to (char*) dangerous? Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> - 2011-11-16 20:58 +0100
            Re: when is typecasting (unsigned char*) to (char*) dangerous? Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-11-18 15:37 +0200
              Re: when is typecasting (unsigned char*) to (char*) dangerous? Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> - 2011-11-18 19:41 +0100
                Re: when is typecasting (unsigned char*) to (char*) dangerous? Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> - 2011-11-18 19:43 +0100
    Re: when is typecasting (unsigned char*) to (char*) dangerous? Lauri Alanko <la@iki.fi> - 2011-11-17 19:09 +0000
      Re: when is typecasting (unsigned char*) to (char*) dangerous? Harald van Dijk <truedfx@gmail.com> - 2011-11-17 11:44 -0800
      Re: when is typecasting (unsigned char*) to (char*) dangerous? Keith Thompson <kst-u@mib.org> - 2011-11-17 13:16 -0800
    Re: when is casting (unsigned char*) to (signed char*) dangerous? Shao Miller <sha0.miller@gmail.com> - 2012-01-22 02:22 -0500
      Re: when is casting (unsigned char*) to (signed char*) dangerous? Harald van Dijk <haraldvdijk@gmail.com> - 2012-01-22 12:14 -0800
      Re: when is casting (unsigned char*) to (signed char*) dangerous? Keith Thompson <kst-u@mib.org> - 2012-01-23 00:37 -0800
        Re: when is casting (unsigned char*) to (signed char*) dangerous? Jens Gustedt <jens.gustedt@loria.fr> - 2012-01-24 15:10 +0100
        Re: when is casting (unsigned char*) to (signed char*) dangerous? Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-26 08:55 -0800
          Re: when is casting (unsigned char*) to (signed char*) dangerous? Keith Thompson <kst-u@mib.org> - 2012-01-26 11:32 -0800
            Re: when is casting (unsigned char*) to (signed char*) dangerous? Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 21:57 -0800
              Re: when is casting (unsigned char*) to (signed char*) dangerous? Shao Miller <sha0.miller@gmail.com> - 2012-02-01 01:12 -0500
      Re: when is casting (unsigned char*) to (signed char*) dangerous? Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-26 08:48 -0800

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#13547

Harald van Dijk <truedfx@gmail.com> writes:
> On Nov 17, 4:22 am, Keith Thompson <ks...@mib.org> wrote:
>> Here's the paragraph:
>>
>>     Certain object representations need not represent a value of
>>     the object type. If the stored value of an object has such a
>>     representation and is read by an lvalue expression that does
>>     not have character type, the behavior is undefined. If such
>>     a representation is produced by a side effect that modifies
>>     all or any part of the object by an lvalue expression that
>>     does not have character type, the behavior is undefined.
>>     Such a representation is called a *trap representation*.
>>
>> I still think that it refers to (or *should* refer to) a trap
>> representation for the type of the lvalue.
>>
>> Consider:
>>
>>     void *ptr = malloc(sizeof (int));
>>     assert(ptr != NULL);
>>     /* code to set the bytes pointed to by ptr to a
>>        trap representation for type int */
>>     int n = *(int*)ptr;
>>
>> The object created by the malloc() call isn't an object of type int;
>> it's just raw storage.  If 6.2.6.1p5 doesn't imply that accessing
>> it via an lvalue of type int has undefined behavior, then what does?
>
> If *ptr doesn't hold an int, then reading it as an int is a violation
> of the aliasing rules.

6.5p7:

    An object shall have its stored value accessed only by an lvalue
    expression that has one of the following types:

    -- a type compatible with the effective type of the object,
    [...]

Going back to paragraph 6:

    For all other accesses to an object having no declared type, the
    effective type of the objec simply the type of the lvalue used for
    the access.

So the effective type of the object is int.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#13564

On Nov 17, 8:46 am, Keith Thompson <ks...@mib.org> wrote:
> Harald van Dijk <true...@gmail.com> writes:
> > On Nov 17, 4:22 am, Keith Thompson <ks...@mib.org> wrote:
> >> Here's the paragraph:
>
> >>     Certain object representations need not represent a value of
> >>     the object type. If the stored value of an object has such a
> >>     representation and is read by an lvalue expression that does
> >>     not have character type, the behavior is undefined. If such
> >>     a representation is produced by a side effect that modifies
> >>     all or any part of the object by an lvalue expression that
> >>     does not have character type, the behavior is undefined.
> >>     Such a representation is called a *trap representation*.
>
> >> I still think that it refers to (or *should* refer to) a trap
> >> representation for the type of the lvalue.
>
> >> Consider:
>
> >>     void *ptr = malloc(sizeof (int));
> >>     assert(ptr != NULL);
> >>     /* code to set the bytes pointed to by ptr to a
> >>        trap representation for type int */
> >>     int n = *(int*)ptr;
>
> >> The object created by the malloc() call isn't an object of type int;
> >> it's just raw storage.  If 6.2.6.1p5 doesn't imply that accessing
> >> it via an lvalue of type int has undefined behavior, then what does?
>
> > If *ptr doesn't hold an int, then reading it as an int is a violation
> > of the aliasing rules.
>
> 6.5p7:
>
>     An object shall have its stored value accessed only by an lvalue
>     expression that has one of the following types:
>
>     -- a type compatible with the effective type of the object,
>     [...]
>
> Going back to paragraph 6:
>
>     For all other accesses to an object having no declared type, the
>     effective type of the objec simply the type of the lvalue used for
>     the access.
>
> So the effective type of the object is int.

Depending on the omitted "code to set the bytes pointed to by ptr to a
trap representation for type int", there are three possibilities right
before n is initialised:
1) *ptr has no effective type
2) *ptr has an effective type that is int
3) *ptr has an effective type that is not int

You're right, 1) and 2) are effectively the same, the effective type
becomes int. In both of those cases, 6.2.6.1p5 says the behaviour is
undefined. For possibility 3), 6.5p7 says the behaviour is undefined.

[toc] | [prev] | [next] | [standalone]

#16189

Keith Thompson <kst-u@mib.org> writes:

> Harald van Dijk <truedfx@gmail.com> writes:
> [...]
>> Yes, but that was not the point I was trying to make. I'm reading
>> s2.ptr, which potentially holds a trap representation, and 6.2.6.1p5
>> disallows reading trap representations. It doesn't refer to
>> representations that do not represent a value in the type of the
>> lvalue expression, it refers to representations that do not represent
>> a value in the type of the object. The object has type void *, even if
>> it's accessed using unsigned char. That's why there needs to be a
>> specific exception for when the lvalue expression has character type.
>
> That's an interesting interpretation, but I'm still not quite convinced.
>
> Here's the paragraph:
>
>     Certain object representations need not represent a value of
>     the object type. If the stored value of an object has such a
>     representation and is read by an lvalue expression that does
>     not have character type, the behavior is undefined. If such
>     a representation is produced by a side effect that modifies
>     all or any part of the object by an lvalue expression that
>     does not have character type, the behavior is undefined.
>     Such a representation is called a *trap representation*.
>
> I still think that it refers to (or *should* refer to) a trap
> representation for the type of the lvalue.  [snip example]

The point of 6.2.6.1p5 is that, when an object holds a trap
representation for its own type, reading it using another type
(that isn't a character type) is undefined behavior, _even if_
the particular object representation is not a trap representation
for the accessing type.  For example:

    int foo;
    memcpy( &foo, &int_trap_representation, sizeof foo );
    * (unsigned int *) &foo;  /* BAM! */

The BAM line is undefined behavior, even if 'unsigned int'
has no trap representations.  (If both types have no trap
representations then the access on the BAM line is well-
defined.)

Of course, trying to access an object that holds a trap
representation of the accessing (lvalue) type is always
undefined behavior, as I just explained in my last posting.

[toc] | [prev] | [next] | [standalone]

#13517

Keith Thompson ha scritto:
[...]
>
> I have a vague memory of a statement that plain and signed char cannot
> have trap representations, but I can't confirm that from the standard.
>

You probably refer to the fact that there is no undefined-behavior
when a trap representation is read by an lvalue expression that has
character type. Nevertheless, the value of such an object should not
be used until a proper value is stored in it.

-- 
Vincenzo Mercuri | www.backbox.org

[toc] | [prev] | [next] | [standalone]

#13496

(Top-posting corrected.  And question copied from subject into body.)

On 11/16/2011 11:55 AM, tim wrote:
> On Tue, 15 Nov 2011 20:51:01 +0000, Kaz Kylheku wrote:
>> On 2011-11-15, tim<nospam@nospam.com>  wrote:
>>> when is typecasting (unsigned char*) to (char*) dangerous?
 >>>
>>> thanks in advance for your help, tim
>>
>> It's dangerous when it's done by someone who got other people to answer
>> his technical interview or homework questions.
 >>
 > THIS IS NOT A HOMEWORK

It's a very "homework-sounding" question.  Perhaps if you were to explain 
what you're trying to do, so we can better answer the "real" question at hand?

-- 
Kenneth Brody

[toc] | [prev] | [next] | [standalone]

#13467

include the subject of your post in the body of your post

"when is typecasting (unsigned char*) to (char*) dangerous? Options"

On Nov 15, 7:59 pm, tim <nos...@nospam.com> wrote:
> thanks in advance for your help, tim

what is "typecasting"? It isn't defined by the C Standard so I'm
guessing you're talking about the problems older actors have? Must be
some extended pun to do with stars and character actors not being
asked for autographs...

[toc] | [prev] | [next] | [standalone]

#13468

On Nov 15, 9:59 pm, tim <nos...@nospam.com> wrote:
> thanks in advance for your help, tim
>
In narrow technical sense, it's almost never dangerous. chars can have
trap representations whilst unsigned chars can't, but it's most
unlikely your code will ever need to run on such a machine.

However casting unsigned char to char, though not the other way round,
usually means that someone doesn't know what they are doing. char
should be used for characters, i.e human-readable text unsigned char
for bytes, usually arbitrary bits, occasionally for small integers. If
you need a tiny signed integer, use signed char. It doesn't normally
make sense to convert an arbitrary bit pattern to a human-readable
character.
--
MiniBasic - how to write a script interpreter. Working ANSI C Basic
interpreter
http://www.malcolmmclean.site11.com/www

[toc] | [prev] | [next] | [standalone]

#13473

On 16.11.2011 10:36, Malcolm McLean wrote:
> On Nov 15, 9:59 pm, tim <nos...@nospam.com> wrote:
>> thanks in advance for your help, tim
>>
> In narrow technical sense, it's almost never dangerous. chars can have
> trap representations whilst unsigned chars can't, but it's most
> unlikely your code will ever need to run on such a machine.
> 
> However casting unsigned char to char, though not the other way round,
> usually means that someone doesn't know what they are doing. char
> should be used for characters, i.e human-readable text unsigned char
> for bytes, usually arbitrary bits, occasionally for small integers. If
> you need a tiny signed integer, use signed char. It doesn't normally
> make sense to convert an arbitrary bit pattern to a human-readable
> character.

Heh! Tell that to the guys who wrote libbzip2! They have a routine
called BZ2_bzBuffToBuffDecompress() and they think that the compressed
buffer pointer is of type "char*". Nice one. So I had to cast that
pointer to make the warning go away.

CYA,
Markus

[toc] | [prev] | [next] | [standalone]

#13484

On Nov 16, 1:36 am, Malcolm McLean <malcolm.mcle...@btinternet.com>
wrote:
> On Nov 15, 9:59 pm, tim <nos...@nospam.com> wrote:> thanks in advance for your help, tim
>
> In narrow technical sense, it's almost never dangerous. chars can have
> trap representations whilst unsigned chars can't, but it's most
> unlikely your code will ever need to run on such a machine.
>
> However casting unsigned char to char, though not the other way round,
> usually means that someone doesn't know what they are doing. char
> should be used for characters, i.e human-readable text unsigned char
> for bytes, usually arbitrary bits, occasionally for small integers.

Not since the 70's...maybe earlier.  The character sets on today's
systems extend the full length of the 8 bit, unsigned character if not
further.

> If
> you need a tiny signed integer, use signed char.

Not unless you've got a really good reason.  It doesn't buy you
anything.

[toc] | [prev] | [next] | [standalone]

#13499

On 11/16/2011 9:14 AM, nroberts wrote:
> On Nov 16, 1:36 am, Malcolm McLean<malcolm.mcle...@btinternet.com>
> wrote:
>> On Nov 15, 9:59 pm, tim<nos...@nospam.com>  wrote:>  thanks in advance for your help, tim
>>
>> In narrow technical sense, it's almost never dangerous. chars can have
>> trap representations whilst unsigned chars can't, but it's most
>> unlikely your code will ever need to run on such a machine.
>>
>> However casting unsigned char to char, though not the other way round,
>> usually means that someone doesn't know what they are doing. char
>> should be used for characters, i.e human-readable text unsigned char
>> for bytes, usually arbitrary bits, occasionally for small integers.
> Not since the 70's...maybe earlier.  The character sets on today's
> systems extend the full length of the 8 bit, unsigned character if not
> further.

tradition dictates use of "char *", but UTF-8 doesn't work so well with 
plain char, so casts both ways are often needed when working with UTF-8...

IMO, UTF-8 is still generally preferable to UTF-16 as:
the vast majority of text is smaller or the same size with UTF-8 (the 
main cases it is on-average bigger is with lots of CJK characters);
the raw performance difference usually does not matter (raw strings can 
be normalized, and compared as bytes, and cases where more complex logic 
is needed, UTF-16 will not save much performance-wise);
...

accessing UTF-8 strings by character index can also be handled (by 
faking it some).


>> If
>> you need a tiny signed integer, use signed char.
> Not unless you've got a really good reason.  It doesn't buy you
> anything.

I disagree...

often, saving memory is a good idea, as even with multiple GB of RAM, 
one can still end up using up most of it, and saving memory if one has a 
lot of data of a given type may often make sense.

using signed char arrays rather than int or similar, if the data will 
fit in a signed char, is often a good idea.
very often, people make use of more elaborate strategies (bit packing or 
mathematical transformations, or sometimes traditional data-compression 
strategies) to allow encoding data in less bits as well.

a lot depends on the type of app though...

for an app which never uses more than maybe a few MB of RAM, yes, saving 
a few bytes may not matter so much...


or such...

[toc] | [prev] | [next] | [standalone]

#13488

When "char" has trap representations that "unsigned char" does
not?
-- 
"When I have to rely on inadequacy, I prefer it to be my own."
--Richard Heathfield

[toc] | [prev] | [next] | [standalone]

#13492

Ben Pfaff ha scritto:
> When "char" has trap representations that "unsigned char" does
> not?

I thought about this as well, but the typecast "per se" is in fact
a conversion between pointer types so I think it would be safe.

-- 
Vincenzo Mercuri | www.backbox.org

[toc] | [prev] | [next] | [standalone]

#13493

Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> writes:

> Ben Pfaff ha scritto:
>> When "char" has trap representations that "unsigned char" does
>> not?
>
> I thought about this as well, but the typecast "per se" is in fact
> a conversion between pointer types so I think it would be safe.

I agree that the cast itself is not the problem.
-- 
Ben Pfaff 
http://benpfaff.org

[toc] | [prev] | [next] | [standalone]

#13502

Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> writes:
> Ben Pfaff ha scritto:
>> When "char" has trap representations that "unsigned char" does
>> not?
>
> I thought about this as well, but the typecast "per se" is in fact
> a conversion between pointer types so I think it would be safe.

If you're going to cast (not "typecast") an unsigned char* to char*,
surely you have a reason for doing so, presumably to access the
pointed-to memory as char.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#13508

Keith Thompson ha scritto:
[...]
> If you're going to cast (not "typecast") an unsigned char* to char*,
> surely you have a reason for doing so, presumably to access the
> pointed-to memory as char.

Yes, I can't imagine of a reason for making such a cast without
accessing the pointed-to memory... but I can't say that this is
what the OP means. Let's just say that the following code can
have undefined behavior:

#include <stdio.h>
int main(void)
{
     unsigned char a = 255;
     char b = *    (char *)&a;
     /*      ^^^   ^^^^^^^   */
     /*      risk  no-risk   */
     printf("b is %d\n", (int)b);
     return 0;
}


-- 
Vincenzo Mercuri | www.backbox.org

[toc] | [prev] | [next] | [standalone]

#13590

Vincenzo Mercuri <vincenzo.mercuri@yahoo.it> writes:
> Keith Thompson ha scritto:
> [...]
> > If you're going to cast (not "typecast") an unsigned char* to char*,
> > surely you have a reason for doing so, presumably to access the
> > pointed-to memory as char.
> 
> Yes, I can't imagine of a reason for making such a cast without
> accessing the pointed-to memory...

When it's an arg for a call-back function, which you will use
by first casting back to the right type?

Phil
-- 
Unix is simple. It just takes a genius to understand its simplicity
-- Dennis Ritchie (1941-2011), Unix Co-Creator

[toc] | [prev] | [next] | [standalone]

#13602

Phil Carmody ha scritto:
> Vincenzo Mercuri<vincenzo.mercuri@yahoo.it>  writes:
>> Keith Thompson ha scritto:
>> [...]
>>> If you're going to cast (not "typecast") an unsigned char* to char*,
>>> surely you have a reason for doing so, presumably to access the
>>> pointed-to memory as char.
>>
>> Yes, I can't imagine of a reason for making such a cast without
>> accessing the pointed-to memory...
>
> When it's an arg for a call-back function, which you will use
> by first casting back to the right type?

Neither, I'd use a "void*", and I would then cast it back
to the right type (whereas an explicity cast is really needed).

-- 
Vincenzo Mercuri | www.backbox.org

[toc] | [prev] | [next] | [standalone]

#13603

Vincenzo Mercuri ha scritto:

> to the right type (whereas an explicity cast is really needed).

*explicit*

-- 
Vincenzo Mercuri | www.backbox.org

[toc] | [prev] | [next] | [standalone]

#13572

I actually stumbled into this problem just now.

I need to represent UTF-8 strings in C, but although the common wisdom
is that UTF-8 is nicely compatible with legacy C code, it seems that
this isn't strictly true: we cannot portably cast an (unsigned char*)
buffer containing a UTF-8-encoded, 0-terminated string to (char*) and
operate on it with standard C functions. A UTF-8 encoded buffer may
contain the byte value 0x80, which, when casted to char, might be a
trap representation on platforms where CHAR_MIN is -127.

This is awful shame, since there are byte values that never occur in
well-formed UTF-8: 0xc0, 0xc1 and 0xf5-0xff. If one of those had been
0x80, to my understanding there wouldn't have been a problem.

But if char might not be able to represent all the possible bytes of
UTF-8, how can C1X have UTF-8 encoded string literals? Maybe I'll
write a separate post about this to comp.std.c.


Lauri

[toc] | [prev] | [next] | [standalone]

#13573

On Nov 17, 8:09 pm, Lauri Alanko <l...@iki.fi> wrote:
> I actually stumbled into this problem just now.
>
> I need to represent UTF-8 strings in C, but although the common wisdom
> is that UTF-8 is nicely compatible with legacy C code, it seems that
> this isn't strictly true: we cannot portably cast an (unsigned char*)
> buffer containing a UTF-8-encoded, 0-terminated string to (char*) and
> operate on it with standard C functions. A UTF-8 encoded buffer may
> contain the byte value 0x80, which, when casted to char, might be a
> trap representation on platforms where CHAR_MIN is -127.

The standard C functions in <string.h> treat their arguments as
unsigned char * (7.21.1p3 for the interested), so strlen() etc. work
even on the system you describe, of course assuming you expect a
string length in bytes.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web