Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #12593 > unrolled thread

signed int64 pack/unpack

Back to article view | Back to comp.lang.php

Contents

  signed int64 pack/unpack cameron7@gmail.com - 2014-01-11 18:18 -0800
    Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-11 21:35 -0500
      Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-11 18:46 -0800
        Re: signed int64 pack/unpack The Natural Philosopher <tnp@invalid.invalid> - 2014-01-12 02:59 +0000
          Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-11 22:03 -0500
          Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-11 19:10 -0800
        Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-11 22:04 -0500
    Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 02:42 +0000
      Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-11 22:01 -0500
        Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-11 19:08 -0800
        Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 03:11 +0000
          Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-11 22:15 -0500
            Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 12:37 +0000
              Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-12 10:51 -0500
                Re: signed int64 pack/unpack Christoph Michael Becker <cmbecker69@arcor.de> - 2014-01-12 17:00 +0100
                  Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-12 16:09 -0500
                    Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 22:35 +0000
                      Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-12 17:59 -0500
                        Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-13 01:59 +0000
                Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 22:30 +0000
      Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-11 21:54 -0800
        Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-12 07:25 -0800
          Re: signed int64 pack/unpack Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-12 11:11 -0500
          Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-12 23:27 +0000
            Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-12 16:03 -0800
            Re: signed int64 pack/unpack cameron7@gmail.com - 2014-01-12 16:22 -0800
              Re: signed int64 pack/unpack Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-13 01:20 +0000
    Re: signed int64 pack/unpack Christoph Michael Becker <cmbecker69@arcor.de> - 2014-01-12 17:37 +0100

Page 2 of 2 — ← Prev page 1 [2]

#12606

Ok, so here's some code where we can start from:

This is giving me:

OUTPUT:
------------
PACKING: 8223372036854775807
UNPACKED: 8223372036854775807

However, when we're dealing with negative numbers:

OUTPUT:
------------
PACKING: -8223372036854775807
UNPACKED: 10223372036854776028

This likely will simply require some experimentation, for example I'm using "unsigned" flags to unpack, but it's not obvious, since when I go with a "signed" flag for negative values (ie. 'l' for signed long) I'm getting the same.  If someone on the list has experience hacking binary protocols in PHP would love to know if there's a better way...

<?php

$i64 = 8223372036854775807;

echo "PACKING: ".$i64.PHP_EOL;

$binary = getBinaryString($i64, 64);

list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary,  0, 32), 2, 10)));
list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));

$i64c = getBinaryString($i64a, 32);
$i64d = getBinaryString($i64b, 32);

echo "UNPACKED: ".base_convert($i64c.$i64d,2,10).PHP_EOL;

function getBinaryString($packed,$bits){
    $a = 0x01;
    $binary = '';
    for($x=0; $x<$bits; $x++){
        $binary .= (int)(bool)($a & $packed);
        $a = $a << 1;
    }
    return strrev($binary);
}

[toc] | [prev] | [next] | [standalone]

#12608

I guess I'll reply with my own solution.  Turns out there are 2 separate issues here.

1)  the "signed" bit needs to be flipped back to 0 before converting to int.  This is because it appears the conversion from base 2 to base 10 will always assume the end value is positive.
2)  Because of this, the rest of the bits need to be flipped.

So I added 2 arguments to the function, one will tell us if we're supposed to be checking for a "signed bit", and the other will tell us if we need to flip bits.

As you'll see I'm still off by 1 in the end result, but I'm sure that's just in the details.


I hope you'll all agree, this is really ugly, but seems to work  :)

<?php

$i64 = -8223372036854775807;

echo "PACKING: ".$i64.PHP_EOL;

list(,$binary) = getBinaryString($i64, 64);

list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary,  0, 32), 2, 10)));
list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));

list($flag,$i64c)= getBinaryString($i64a, 32, true, true);
list(,$i64d) = getBinaryString($i64b, 32, false, true);

$unpack = base_convert($i64c.$i64d,2,10);
$unpack *= $flag ? -1 : 1;

echo "UNPACKED: ".$unpack.PHP_EOL;

function getBinaryString($packed,$bits,$check_signed = false, $flip = false){
    $a = 0x01;
    $binary = '';
    for($x=0; $x<$bits; $x++){
        $binary .= (int)(bool)($a & $packed);
        $a = $a << 1;
    }
    $signed_flag = false;
    if($check_signed && substr($binary,$bits-1,1) === '1'){
        $signed_flag = true;
    }
    if($flip){
        for($x=0;$x<strlen($binary);$x++){
            $binary[$x] = $binary[$x] === '1' ? '0' : '1';
        }
    }
    if($signed_flag){
        $binary = substr($binary,0,$bits-1).'0';
    }
    return array($signed_flag, strrev($binary));
}

[toc] | [prev] | [next] | [standalone]

#12611

On 1/12/2014 10:25 AM, cameron7@gmail.com wrote:
> I guess I'll reply with my own solution.  Turns out there are 2 separate issues here.
>
> 1)  the "signed" bit needs to be flipped back to 0 before converting to int.  This is because it appears the conversion from base 2 to base 10 will always assume the end value is positive.
> 2)  Because of this, the rest of the bits need to be flipped.
>

True.  base_convert

> So I added 2 arguments to the function, one will tell us if we're supposed to be checking for a "signed bit", and the other will tell us if we need to flip bits.
>
> As you'll see I'm still off by 1 in the end result, but I'm sure that's just in the details.
>

That's because when you use 2's compliment, you need to flip the bits 
and add one.

>
> I hope you'll all agree, this is really ugly, but seems to work  :)
>
> <?php
>
> $i64 = -8223372036854775807;
>
> echo "PACKING: ".$i64.PHP_EOL;
>
> list(,$binary) = getBinaryString($i64, 64);
>
> list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary,  0, 32), 2, 10)));
> list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));
>
> list($flag,$i64c)= getBinaryString($i64a, 32, true, true);
> list(,$i64d) = getBinaryString($i64b, 32, false, true);
>
> $unpack = base_convert($i64c.$i64d,2,10);
> $unpack *= $flag ? -1 : 1;
>
> echo "UNPACKED: ".$unpack.PHP_EOL;
>
> function getBinaryString($packed,$bits,$check_signed = false, $flip = false){
>      $a = 0x01;
>      $binary = '';
>      for($x=0; $x<$bits; $x++){
>          $binary .= (int)(bool)($a & $packed);
>          $a = $a << 1;
>      }
>      $signed_flag = false;
>      if($check_signed && substr($binary,$bits-1,1) === '1'){
>          $signed_flag = true;
>      }
>      if($flip){
>          for($x=0;$x<strlen($binary);$x++){
>              $binary[$x] = $binary[$x] === '1' ? '0' : '1';
>          }
>      }
>      if($signed_flag){
>          $binary = substr($binary,0,$bits-1).'0';
>      }
>      return array($signed_flag, strrev($binary));
> }
>

I've seen worse :)

However, you might want to check some other functions.  The gmp_xxx 
functions will work on longer numbers; so will the BCD functions. 
There are some suggestions and links under base_convert() in the online 
documentation at http://us1.php.net/manual/en/function.base-convert.php.

Of course, if you're on a shared host and can't get them to install the 
libraries for you, you're out of luck.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#12630

cameron7@gmail.com writes:

> I guess I'll reply with my own solution.  Turns out there are 2
> separate issues here.

Here's mine.  You might have to alter the order of the two 32-bit
components to match your external format.  I've used the same names you
might be able to drop it into your tests.

echo "PACKING: ".$i64.PHP_EOL;

$ps = pack('N2', $i64 >> 32, $i64 & 0xffffffff);

list(, $hi, $lo) = unpack('N2', $ps);
$unpack = ($hi << 32) + $lo;

echo "UNPACKED: ".$unpack.PHP_EOL;

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#12631

Ben, This is exactly what I was after.  Thank you.

[toc] | [prev] | [next] | [standalone]

#12632

Although it does lead me to the question, how will we do unsigned int64 :)

For another day I guess...

[toc] | [prev] | [next] | [standalone]

#12633

cameron7@gmail.com writes:

> Although it does lead me to the question, how will we do unsigned
> int64 :)

:) indeed!  PHP does not support unsigned integers so it's a genuine
problem.  In languages without unsigned integers, you can get by only
when the larges unsigned quantity you every use is <= INT_MAX.

> For another day I guess...

The answer will depend on what the code must do with the results.
Keeping them as signed might work for a lot of what gets done with it.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#12613

cameron7@gmail.com wrote:

> I'm having a bit of trouble figuring out a straightforward way to
> "pack" / "unpack" a signed int64.
> 
> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
> 4-byte length.  And of course even though double comes back with 8
> length it's a floating point and thus loses precision.
> 
> Any thoughts?  Any help would be greatly appreciated.
> 
> BTW, unsigned int64 seems relatively straightforward, but signed
> int64 has been unusually difficult to figure out.

It seems to me that it should be possible to pack 64bit numbers by
splitting them into two 32bit numbers and packing the "halves", as Ben
already noticed.  Unpacking should be possible by doing the reverse
operations.  It shouldn't really matter in this case if you're dealing
with signed or unsigned values, if your platform supports the respective
64bit values.

I can't test with 64bit Integers as I'm on Windows with
PHP_INT_MAX==2147483647.  However, I've written the following functions
for working with 32bit integers by relying on the 'n' format of
pack()/unpack().  You may try to adjust the functions for 64bit by using
the 'N' format and replacing the literal "16" with "32".

  function packInt($number)
  {
    $high = $number >> 16;
    $low = $number & ((1 << 16) - 1);
    return pack('n2', $high, $low);
  }

  function unpackInt($string)
  {
    $strings = unpack('n2', $string);
    return ($strings[1] << 16) | $strings[2];
  }

-- 
Christoph M. Becker

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.php

csiph-web