Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #68899 > unrolled thread

On endianness, many #ifdefs, and the need thereof

Back to article view | Back to comp.lang.c++

Contents

  On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-02 04:00 -0700
    Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-02 13:26 +0200
      Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-02 08:59 -0700
    Re: On endianness, many #ifdefs, and the need thereof Öö Tiib <ootiib@hot.ee> - 2019-10-02 05:42 -0700
    Re: On endianness, many #ifdefs, and the need thereof Juha Nieminen <nospam@thanks.invalid> - 2019-10-02 12:53 +0000
      Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-02 15:05 +0200
        Re: On endianness, many #ifdefs, and the need thereof scott@slp53.sl.home (Scott Lurndal) - 2019-10-02 13:54 +0000
          Re: On endianness, many #ifdefs, and the need thereof Keith Thompson <kst-u@mib.org> - 2019-10-02 12:54 -0700
            Re: On endianness, many #ifdefs, and the need thereof scott@slp53.sl.home (Scott Lurndal) - 2019-10-02 20:16 +0000
          Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-03 09:48 +0200
      Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-02 07:52 -0700
        Re: On endianness, many #ifdefs, and the need thereof Öö Tiib <ootiib@hot.ee> - 2019-10-02 08:04 -0700
          Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-02 08:31 -0700
            Re: On endianness, many #ifdefs, and the need thereof Juha Nieminen <nospam@thanks.invalid> - 2019-10-02 21:32 +0000
        Re: On endianness, many #ifdefs, and the need thereof Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-10-03 11:33 +0000
          Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-03 13:46 +0200
            Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-03 05:56 -0700
              Re: On endianness, many #ifdefs, and the need thereof scott@slp53.sl.home (Scott Lurndal) - 2019-10-03 13:17 +0000
                Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-03 16:03 +0200
                  Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-03 07:13 -0700
            Re: On endianness, many #ifdefs, and the need thereof Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-10-03 18:51 +0000
            Re: On endianness, many #ifdefs, and the need thereof Keith Thompson <kst-u@mib.org> - 2019-10-03 14:28 -0700
              Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-04 10:11 +0200
                Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-04 09:00 -0700
          Re: On endianness, many #ifdefs, and the need thereof Paavo Helde <myfirstname@osa.pri.ee> - 2019-10-03 14:46 +0300
          Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-03 05:40 -0700
    Re: On endianness, many #ifdefs, and the need thereof Bonita Montero <Bonita.Montero@gmail.com> - 2019-10-02 15:10 +0200
    Re: On endianness, many #ifdefs, and the need thereof Barry Schwarz <schwarzb@dqel.com> - 2019-10-02 08:33 -0700
      Re: On endianness, many #ifdefs, and the need thereof David Brown <david.brown@hesbynett.no> - 2019-10-03 09:54 +0200
    Re: On endianness, many #ifdefs, and the need thereof "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-10-02 19:35 +0200
      Re: On endianness, many #ifdefs, and the need thereof Daniel <danielaparker@gmail.com> - 2019-10-02 15:39 -0700
    Re: On endianness, many #ifdefs, and the need thereof "Oliver S." <Follow.Me@gmx.net> - 2019-10-02 19:59 +0200

Page 1 of 2  [1] 2  Next page →

#68899 — On endianness, many #ifdefs, and the need thereof

Most C++ software that needs to know whether the host is big endian or little 
endian comes with a header file with a very long sequence of #ifdef's, along 
the lines of

#    if (defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__) || \
    (defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && __BYTE_ORDER == __BIG_ENDIAN) || \
    (defined(BYTE_ORDER) && defined(BIG_ENDIAN) && BYTE_ORDER == BIG_ENDIAN) || \
    (defined(_BIG_ENDIAN) && !defined(_LITTLE_ENDIAN)) || (defined(__BIG_ENDIAN__) && !defined(__LITTLE_ENDIAN__)) || \
    defined(__ARMEB__) || defined(__MIPSEB__) || defined(__s390__) || defined(__sparc__)

On the other hand, at least one library merely has
 
static constexpr bool little_endianess(int num = 1) noexcept
{
    return *reinterpret_cast<char*>(&num) == 1;
}

referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.

So why the many #ifdef's, if this is enough? Or is it enough?

Thanks,
Daniel 

[toc] | [next] | [standalone]

#68900

On 02/10/2019 13:00, Daniel wrote:
> Most C++ software that needs to know whether the host is big endian or little 
> endian comes with a header file with a very long sequence of #ifdef's, along 
> the lines of
> 
> #    if (defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__) || \
>     (defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && __BYTE_ORDER == __BIG_ENDIAN) || \
>     (defined(BYTE_ORDER) && defined(BIG_ENDIAN) && BYTE_ORDER == BIG_ENDIAN) || \
>     (defined(_BIG_ENDIAN) && !defined(_LITTLE_ENDIAN)) || (defined(__BIG_ENDIAN__) && !defined(__LITTLE_ENDIAN__)) || \
>     defined(__ARMEB__) || defined(__MIPSEB__) || defined(__s390__) || defined(__sparc__)
> 
> On the other hand, at least one library merely has
>  
> static constexpr bool little_endianess(int num = 1) noexcept
> {
>     return *reinterpret_cast<char*>(&num) == 1;
> }
> 
> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> 
> So why the many #ifdef's, if this is enough? Or is it enough?
> 

This function is cannot be evaluated as a constant expression, despite
the "constexpr" qualifier.  So you can't use in in places where you
actually need a constant expression (size of an array, template
parameter, etc.).

It will still be good enough in some contexts, especially as compilers
can optimise knowing the result of the function.

For constant expression endianness detection, conditional compilation is
your main choice (the complexity depends on the portability you need),
with pre-build configuration being an alternative, until C++20
standardises endianness.

[toc] | [prev] | [next] | [standalone]

#68920

On Wednesday, October 2, 2019 at 7:26:44 AM UTC-4, David Brown wrote:
> On 02/10/2019 13:00, Daniel wrote:
> > 
> > On the other hand, at least one library merely has
> >  
> > static constexpr bool little_endianess(int num = 1) noexcept
> > {
> >     return *reinterpret_cast<char*>(&num) == 1;
> > }
> > 
> 
> This function is cannot be evaluated as a constant expression, despite
> the "constexpr" qualifier.  So you can't use in in places where you
> actually need a constant expression (size of an array, template
> parameter, etc.).
> 
> It will still be good enough in some contexts, especially as compilers
> can optimise knowing the result of the function.
> 
> For constant expression endianness detection, conditional compilation is
> your main choice (the complexity depends on the portability you need),
> with pre-build configuration being an alternative, until C++20
> standardises endianness.

Thanks for the explanation, appreciated.

Daniel

[toc] | [prev] | [next] | [standalone]

#68902

On Wednesday, 2 October 2019 14:00:51 UTC+3, Daniel  wrote:

> On the other hand, at least one library merely has
>  
> static constexpr bool little_endianess(int num = 1) noexcept
> {
>     return *reinterpret_cast<char*>(&num) == 1;
> }
> 
> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> 
> So why the many #ifdef's, if this is enough? Or is it enough?

The "constexpr" qualifier to functions is meaning something like
"pure function". It can be that it is impossible to expand it
compile time.

Result of reinterpret_cast is not constant expression and so your
function does not work compile time. Demo with online compiler: 
<http://coliru.stacked-crooked.com/a/0801518db168d629>

[toc] | [prev] | [next] | [standalone]

#68903

Daniel <danielaparker@gmail.com> wrote:
> On the other hand, at least one library merely has
>  
> static constexpr bool little_endianess(int num = 1) noexcept
> {
>     return *reinterpret_cast<char*>(&num) == 1;
> }
> 
> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> 
> So why the many #ifdef's, if this is enough? Or is it enough?

Endianess cannot be resolved at compile time, no matter what you do, not
even with modern C++. (Believe me, I have tried. It's not possible.)
The only way to know endianess at compile time is if an external tool
(such as some kind of configure script) provides you with a macro or
constant that says so.

I haven't tried, but I believe the above example will not be evaluated
at compile time (assuming it compiles at all, which I don't think it does;
haven't tried, though).

There may be a technical reason why the compiler cannot provide
endianess information at compile time.

[toc] | [prev] | [next] | [standalone]

#68904

On 02/10/2019 14:53, Juha Nieminen wrote:
> Daniel <danielaparker@gmail.com> wrote:
>> On the other hand, at least one library merely has
>>  
>> static constexpr bool little_endianess(int num = 1) noexcept
>> {
>>     return *reinterpret_cast<char*>(&num) == 1;
>> }
>>
>> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
>>
>> So why the many #ifdef's, if this is enough? Or is it enough?
> 
> Endianess cannot be resolved at compile time, no matter what you do, not
> even with modern C++. (Believe me, I have tried. It's not possible.)
> The only way to know endianess at compile time is if an external tool
> (such as some kind of configure script) provides you with a macro or
> constant that says so.
> 
> I haven't tried, but I believe the above example will not be evaluated
> at compile time (assuming it compiles at all, which I don't think it does;
> haven't tried, though).
> 
> There may be a technical reason why the compiler cannot provide
> endianess information at compile time.
> 

The /compiler/ can handle it at compile time - it can figure out the
result of this function at compile time and use that in optimisation.
(Compilers also usually have pre-defined macros for endianness - but
they are not standardised.)  The issue is that the language says this
can't be part of a constant expression.

I suspect the technical reason will be that constant expressions can't
access memory - they need to be independent of the state of the system.

What I don't really understand, is why endianness macros have never been
standardised.  gcc has __BYTE_ORDER__, __ORDER_LITTLE_ENDIAN__,
__ORDER_BIG_ENDIAN__.  Other compilers usually have similar macros.  It
would not have taken a great deal of effort to make __STDC__ equivalents
part of the C and C++ standards.

With C++20, we will have be able to write:

static constexpr bool little_endianess() noexcept
{
	return std::endian::native == std::endian::little;
}

and it will be a constant expression.

[toc] | [prev] | [next] | [standalone]

#68913

David Brown <david.brown@hesbynett.no> writes:
>On 02/10/2019 14:53, Juha Nieminen wrote:
>> Daniel <danielaparker@gmail.com> wrote:
>>> On the other hand, at least one library merely has
>>>  
>>> static constexpr bool little_endianess(int num = 1) noexcept
>>> {
>>>     return *reinterpret_cast<char*>(&num) == 1;
>>> }
>>>
>>> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
>>>
>>> So why the many #ifdef's, if this is enough? Or is it enough?
>> 
>> Endianess cannot be resolved at compile time, no matter what you do, not
>> even with modern C++. (Believe me, I have tried. It's not possible.)
>> The only way to know endianess at compile time is if an external tool
>> (such as some kind of configure script) provides you with a macro or
>> constant that says so.
>> 
>> I haven't tried, but I believe the above example will not be evaluated
>> at compile time (assuming it compiles at all, which I don't think it does;
>> haven't tried, though).
>> 
>> There may be a technical reason why the compiler cannot provide
>> endianess information at compile time.
>> 
>
>The /compiler/ can handle it at compile time - it can figure out the
>result of this function at compile time and use that in optimisation.

Can it?  How does that work when you're cross-compiling to a different
architecture (like a microcontroller, for example)?

[toc] | [prev] | [next] | [standalone]

#68925

scott@slp53.sl.home (Scott Lurndal) writes:
> David Brown <david.brown@hesbynett.no> writes:
[...]
>>The /compiler/ can handle it at compile time - it can figure out the
>>result of this function at compile time and use that in optimisation.
>
> Can it?  How does that work when you're cross-compiling to a different
> architecture (like a microcontroller, for example)?

Presumably the compiler knows the endianness of the target architecture
it's generating code for.  There might be an issue with the compiler not
having that information soon enough, but it could certainly get the
information if it's worthwhile.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#68926

Keith Thompson <kst-u@mib.org> writes:
>scott@slp53.sl.home (Scott Lurndal) writes:
>> David Brown <david.brown@hesbynett.no> writes:
>[...]
>>>The /compiler/ can handle it at compile time - it can figure out the
>>>result of this function at compile time and use that in optimisation.
>>
>> Can it?  How does that work when you're cross-compiling to a different
>> architecture (like a microcontroller, for example)?
>
>Presumably the compiler knows the endianness of the target architecture
>it's generating code for.  There might be an issue with the compiler not
>having that information soon enough, but it could certainly get the
>information if it's worthwhile.

Each ring in ARM64 can be set to run big- or little-endian.   In ARMv7,
the application itself can change the endianness dynamically (SETEND
instruction).   Linux supports reading the endianness from the ELF
header on Arm64 and will configure the process state accordingly.

So, absent some indication to the compiler on the command line
which endianness is desired, there doesn't seem to be a way for the
compiler to figure it out itself.

There is absolutely nothing wrong with using the pre-processor and
implementation defined (or specified by  the programmer with -D)
macros to determine which endianness is being used.

We write a lot of code that is required to run in both big- and
little-endian environments, and one of the larger issues with
portability is related to bitfields.  All of our structures with
bitfields have declarations similar to:

    struct INTCTLR_CMD_CLEAR_s {
#if __BYTE_ORDER == __BIG_ENDIAN
        uint64_t dev_id                      : 32; /**< [ 63: 32] Interrupt device ID. */
        uint64_t reserved_8_31               : 24;
        uint64_t cmd_type                    :  8; /**< [  7:  0] Command type. Indicates GITS_CMD_TYPE_E::CMD_CLEAR. */
#else
        uint64_t cmd_type                    :  8;
        uint64_t reserved_8_31               : 24;
        uint64_t dev_id                      : 32;
#endif
#if __BYTE_ORDER == __BIG_ENDIAN
        uint64_t reserved_96_127             : 32;
        uint64_t int_id                      : 32; /**< [ 95: 64] Interrupt ID to be translated. */
#else
        uint64_t int_id                      : 32;
        uint64_t reserved_96_127             : 32;
#endif
        uint64_t reserved_128_191            : 64;
        uint64_t reserved_192_255            : 64;
    } s;

[toc] | [prev] | [next] | [standalone]

#68939

On 02/10/2019 15:54, Scott Lurndal wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 02/10/2019 14:53, Juha Nieminen wrote:
>>> Daniel <danielaparker@gmail.com> wrote:
>>>> On the other hand, at least one library merely has
>>>>  
>>>> static constexpr bool little_endianess(int num = 1) noexcept
>>>> {
>>>>     return *reinterpret_cast<char*>(&num) == 1;
>>>> }
>>>>
>>>> referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
>>>>
>>>> So why the many #ifdef's, if this is enough? Or is it enough?
>>>
>>> Endianess cannot be resolved at compile time, no matter what you do, not
>>> even with modern C++. (Believe me, I have tried. It's not possible.)
>>> The only way to know endianess at compile time is if an external tool
>>> (such as some kind of configure script) provides you with a macro or
>>> constant that says so.
>>>
>>> I haven't tried, but I believe the above example will not be evaluated
>>> at compile time (assuming it compiles at all, which I don't think it does;
>>> haven't tried, though).
>>>
>>> There may be a technical reason why the compiler cannot provide
>>> endianess information at compile time.
>>>
>>
>> The /compiler/ can handle it at compile time - it can figure out the
>> result of this function at compile time and use that in optimisation.
> 
> Can it?  How does that work when you're cross-compiling to a different
> architecture (like a microcontroller, for example)?
> 

Try this on <https://godbolt.org>


static constexpr bool little_endianess(int num = 1) noexcept
{
    return *reinterpret_cast<char*>(&num) == 1;
}

bool is_little_endian() {
    return little_endianess();
}



You'll see gcc generate code such as:

is_little_endian():
	movl $1, %eax
	ret


This is, of course, an optimisation issue - not all compilers can handle
it, and you will need to enable optimisation.  (And clang throws an
wobbly when you have a "reinterpret_cast" in a "constexpr" function.
You can just remove the "constexpr", since it doesn't help anyway.)


Compilers know their targets, and they know if they are targeting
little-endian or big-endian (or mixed-endian).  They know that, even if
the cpu in question supports both endiannesses - binaries are generated
for only one endianness at a time.

[toc] | [prev] | [next] | [standalone]

#68914

On Wednesday, October 2, 2019 at 8:53:51 AM UTC-4, Juha Nieminen wrote:
> Daniel wrote:
> > On the other hand, at least one library merely has
> >  
> > static constexpr bool little_endianess(int num = 1) noexcept
> > {
> >     return *reinterpret_cast<char*>(&num) == 1;
> > }
> > 
> > referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> > 
> 
> I haven't tried, but I believe the above example will not be evaluated
> at compile time (assuming it compiles at all, which I don't think it does;
> 
The example is from the nlohmann json library, where I first saw it, 
https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp#L5159, 
and it does compile.

Daniel

[toc] | [prev] | [next] | [standalone]

#68915

On Wednesday, 2 October 2019 17:52:29 UTC+3, Daniel  wrote:
> On Wednesday, October 2, 2019 at 8:53:51 AM UTC-4, Juha Nieminen wrote:
> > Daniel wrote:
> > > On the other hand, at least one library merely has
> > >  
> > > static constexpr bool little_endianess(int num = 1) noexcept
> > > {
> > >     return *reinterpret_cast<char*>(&num) == 1;
> > > }
> > > 
> > > referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> > > 
> > 
> > I haven't tried, but I believe the above example will not be evaluated
> > at compile time (assuming it compiles at all, which I don't think it does;
> > 
> The example is from the nlohmann json library, where I first saw it, 
> https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp#L5159, 
> and it does compile.

On what compiler does that compile:

    constexpr bool b = nlohmann:detail::little_endianess();
 
???

[toc] | [prev] | [next] | [standalone]

#68918

On Wednesday, October 2, 2019 at 11:04:48 AM UTC-4, Öö Tiib wrote:
> On Wednesday, 2 October 2019 17:52:29 UTC+3, Daniel  wrote:
> > On Wednesday, October 2, 2019 at 8:53:51 AM UTC-4, Juha Nieminen wrote:
> > > Daniel wrote:
> > > > On the other hand, at least one library merely has
> > > >  
> > > > static constexpr bool little_endianess(int num = 1) noexcept
> > > > {
> > > >     return *reinterpret_cast<char*>(&num) == 1;
> > > > }
> > > > 
> > > > referencing a contribution on stackoverflow, http://stackoverflow.com/a/1001328/266378.
> > > > 
> > > 
> > > I haven't tried, but I believe the above example will not be evaluated
> > > at compile time (assuming it compiles at all, which I don't think it does;
> > > 
> > The example is from the nlohmann json library, where I first saw it, 
> > https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp#L5159, 
> > and it does compile.
> 
> On what compiler does that compile:
> 
>     constexpr bool b = nlohmann:detail::little_endianess();
>  
> ???

That wouldn't :-) little_endianess() is a static member function of the 
template class binary_reader. I meant that the function compiles in the 
contexts in which it is used in nlohmann's library, it is used after all. But 
the result can't be assigned to a constexpr bool. 

Daniel

[toc] | [prev] | [next] | [standalone]

#68927

Daniel <danielaparker@gmail.com> wrote:
>> > > > static constexpr bool little_endianess(int num = 1) noexcept
>> > > > {
>> > > >     return *reinterpret_cast<char*>(&num) == 1;
>> > > > }

>> > The example is from the nlohmann json library, where I first saw it, 
>> > https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp#L5159, 
>> > and it does compile.
>> 
>> On what compiler does that compile:
>> 
>>     constexpr bool b = nlohmann:detail::little_endianess();
>>  
>> ???
> 
> That wouldn't :-) little_endianess() is a static member function of the 
> template class binary_reader. I meant that the function compiles in the 
> contexts in which it is used in nlohmann's library, it is used after all. But 
> the result can't be assigned to a constexpr bool. 

I find it surprising that it will compile with that 'constexpr'
modifier. I have always had the understanding that a constexpr
function can only contain code that could at least in theory be
evaluated at compile time (even if it actually isn't). At the very
least the potential must be there.

Taking the address of a variable and dereferencing it doesn't
sound like something that should be doable at compile time,
and therefore it sounds like it wouldn't be allowed in a
constexpr function (in the same way as it isn't allowed
eg. as a non-type template parameter).

[toc] | [prev] | [next] | [standalone]

#68948

On Wed, 2019-10-02, Daniel wrote:
> On Wednesday, October 2, 2019 at 8:53:51 AM UTC-4, Juha Nieminen wrote:
>> Daniel wrote:
>> > On the other hand, at least one library merely has
>> >  
>> > static constexpr bool little_endianess(int num = 1) noexcept
>> > {
>> >     return *reinterpret_cast<char*>(&num) == 1;
>> > }
>> > 
>> > referencing a contribution on stackoverflow,
>> > http://stackoverflow.com/a/1001328/266378.
>> > 
>> 
>> I haven't tried, but I believe the above example will not be evaluated
>> at compile time (assuming it compiles at all, which I don't think it does;
>> 
> The example is from the nlohmann json library, where I first saw it, 
> https://github.com/nlohmann/json/ [...]

Weird and slightly worrying that a JSON library would have to be
endianness-aware ...

/Jorgen,
 too lazy to check the source code

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#68951

On 03/10/2019 13:33, Jorgen Grahn wrote:
> On Wed, 2019-10-02, Daniel wrote:
>> On Wednesday, October 2, 2019 at 8:53:51 AM UTC-4, Juha Nieminen wrote:
>>> Daniel wrote:
>>>> On the other hand, at least one library merely has
>>>>  
>>>> static constexpr bool little_endianess(int num = 1) noexcept
>>>> {
>>>>     return *reinterpret_cast<char*>(&num) == 1;
>>>> }
>>>>
>>>> referencing a contribution on stackoverflow,
>>>> http://stackoverflow.com/a/1001328/266378.
>>>>
>>>
>>> I haven't tried, but I believe the above example will not be evaluated
>>> at compile time (assuming it compiles at all, which I don't think it does;
>>>
>> The example is from the nlohmann json library, where I first saw it, 
>> https://github.com/nlohmann/json/ [...]
> 
> Weird and slightly worrying that a JSON library would have to be
> endianness-aware ...
> 
> /Jorgen,
>  too lazy to check the source code
> 

A little look suggests that it is for parsing binary JSON formats (like
UBJSON), rather than standard text-based JSON.  But is still not
necessary, and I doubt if the code's algorithm is more efficient than
the obvious endian independent method.

[toc] | [prev] | [next] | [standalone]

#68954

On Thursday, October 3, 2019 at 7:46:13 AM UTC-4, David Brown wrote:
> 
> A little look suggests that it is for parsing binary JSON formats (like
> UBJSON), rather than standard text-based JSON.  But is still not
> necessary, and I doubt if the code's algorithm is more efficient than
> the obvious endian independent method.

An "obvious endian independent method"? For decoding and encoding CBOR, 
Message Pack, UBJSON, and BSON into native types? What are you referring to?

CBOR encodes 8, 16, 32, and 64 bit signed and unsigned ints as big endian, and 
16, 32, and 64 bit floats as big endian. "CBOR for typed arrays" accommodates 
typed arrays of 8, 16, 32, and 64 bit signed and unsigned ints, and 16, 32, 
64, 128 bit floating point numbers, in either little endian or big endian. 
Could you explain the "obvious endian independent method" that will encode and 
decode that?

Thanks,
Daniel 

[toc] | [prev] | [next] | [standalone]

#68955

Daniel <danielaparker@gmail.com> writes:
>On Thursday, October 3, 2019 at 7:46:13 AM UTC-4, David Brown wrote:
>> 
>> A little look suggests that it is for parsing binary JSON formats (like
>> UBJSON), rather than standard text-based JSON.  But is still not
>> necessary, and I doubt if the code's algorithm is more efficient than
>> the obvious endian independent method.
>
>An "obvious endian independent method"? For decoding and encoding CBOR, 
>Message Pack, UBJSON, and BSON into native types? What are you referring to?
>
>CBOR encodes 8, 16, 32, and 64 bit signed and unsigned ints as big endian, and 
>16, 32, and 64 bit floats as big endian. "CBOR for typed arrays" accommodates 
>typed arrays of 8, 16, 32, and 64 bit signed and unsigned ints, and 16, 32, 
>64, 128 bit floating point numbers, in either little endian or big endian. 
>Could you explain the "obvious endian independent method" that will encode and 
>decode that?

If the incoming big-endian data is treated as a sequence of bytes, you
can easily reconstruct a binary value simply shifting the data into the
correct sized container a byte at a time.

  uint64_t  value = 0ul;
  char *bp = &buffer[first_byte_of_uint64_big_endian_data];

  for (size_t i = 0; i < sizeof(uint64_t); i++) {
      value <<= 8;
      value |= *bp++;
  }

This works regardless of the host endianness.

[toc] | [prev] | [next] | [standalone]

#68961

On 03/10/2019 15:17, Scott Lurndal wrote:
> Daniel <danielaparker@gmail.com> writes:
>> On Thursday, October 3, 2019 at 7:46:13 AM UTC-4, David Brown wrote:
>>>
>>> A little look suggests that it is for parsing binary JSON formats (like
>>> UBJSON), rather than standard text-based JSON.  But is still not
>>> necessary, and I doubt if the code's algorithm is more efficient than
>>> the obvious endian independent method.
>>
>> An "obvious endian independent method"? For decoding and encoding CBOR, 
>> Message Pack, UBJSON, and BSON into native types? What are you referring to?
>>
>> CBOR encodes 8, 16, 32, and 64 bit signed and unsigned ints as big endian, and 
>> 16, 32, and 64 bit floats as big endian. "CBOR for typed arrays" accommodates 
>> typed arrays of 8, 16, 32, and 64 bit signed and unsigned ints, and 16, 32, 
>> 64, 128 bit floating point numbers, in either little endian or big endian. 
>> Could you explain the "obvious endian independent method" that will encode and 
>> decode that?
> 
> If the incoming big-endian data is treated as a sequence of bytes, you
> can easily reconstruct a binary value simply shifting the data into the
> correct sized container a byte at a time.
> 
>   uint64_t  value = 0ul;
>   char *bp = &buffer[first_byte_of_uint64_big_endian_data];
> 
>   for (size_t i = 0; i < sizeof(uint64_t); i++) {
>       value <<= 8;
>       value |= *bp++;
>   }
> 
> This works regardless of the host endianness.
> 

You want "uint8_t" (or, for even more portability, uint_least8_t)
instead of "char" for your pointer.  Unpleasant things will happen if
"char" is signed.

Apart from that, that is the "obvious" method I meant.

Some compilers are smart enough to optimise this kind of code into a
single load (if the cpu supports unaligned accesses) and a byte-swap (if
the cpu is little-endian).  Clang manages it for 16-bit and 32-bit
versions of this code, but not the 64-bit version.  gcc won't do it at
all in this case (I've seen it do such optimisation with other code).

[toc] | [prev] | [next] | [standalone]

#68966

On Thursday, October 3, 2019 at 10:03:52 AM UTC-4, David Brown wrote:
> On 03/10/2019 15:17, Scott Lurndal wrote:
> > 
> > If the incoming big-endian data is treated as a sequence of bytes, you
> > can easily reconstruct a binary value simply shifting the data into the
> > correct sized container a byte at a time.
> > 
> >   uint64_t  value = 0ul;
> >   char *bp = &buffer[first_byte_of_uint64_big_endian_data];
> > 
> >   for (size_t i = 0; i < sizeof(uint64_t); i++) {
> >       value <<= 8;
> >       value |= *bp++;
> >   }
> > 
> > This works regardless of the host endianness.
> > 
> 
> You want "uint8_t" (or, for even more portability, uint_least8_t)
> instead of "char" for your pointer.  Unpleasant things will happen if
> "char" is signed.
> 
> Apart from that, that is the "obvious" method I meant.
> 
Thanks, Scott and David, obvious, perhaps, but I didn't know about it.

Daniel

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.c++

csiph-web