Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.sys.apple2.programmer > #2702 > unrolled thread

packBytes and unpackBytes

Back to article view | Back to comp.sys.apple2.programmer

Contents

  packBytes and unpackBytes markpmlim@yahoo.com.sg - 2016-04-28 01:04 -0700
    Re: packBytes and unpackBytes awanderin <awanderin@gmail.com> - 2016-04-30 20:44 -0600
    Re: packBytes and unpackBytes Bill Buckels <bbuckels@escape.ca> - 2016-05-14 11:13 -0700
      Re: packBytes and unpackBytes Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-05-14 13:27 -0700
        Re: packBytes and unpackBytes Bill Buckels <bbuckels@escape.ca> - 2016-05-15 02:36 -0700
        Re: packBytes and unpackBytes Bill Buckels <bbuckels@escape.ca> - 2016-05-15 02:43 -0700
          Re: packBytes and unpackBytes Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-05-15 15:19 -0700
      Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2016-05-15 18:27 -0700
        Re: packBytes and unpackBytes wssimms@gmail.com - 2016-05-15 21:29 -0700
          Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2016-05-16 18:42 -0700
            Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2016-05-18 04:52 -0700
              Re: packBytes and unpackBytes Bill Buckels <bbuckels@escape.ca> - 2016-07-17 18:21 -0700
                Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2016-07-18 14:23 -0700
              Re: Load and display an Apple IIGS SHR graphic markpmlim@yahoo.com.sg - 2018-09-18 18:19 -0700
    Re: packBytes and unpackBytes burgerbecky@gmail.com - 2016-07-27 17:52 -0700
      Re: packBytes and unpackBytes burgerbecky@gmail.com - 2016-07-27 17:53 -0700
      Re: packBytes and unpackBytes Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-07-27 19:37 -0700
    Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2018-09-18 18:29 -0700
      Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2018-12-05 17:23 -0800
        Re: packBytes and unpackBytes markpmlim@yahoo.com.sg - 2018-12-07 17:46 -0800

#2702 — packBytes and unpackBytes

Using resources available on the Internet, the equivalent of the Apple IIgs misc tool calls packBytes and unpackBytes have been coded in Swift. 

The Swift playground project demonstrates how to encode and decode files using the 2 methods packBytes and unpackBytes. If you wish to encode/decode  Apple Preferred Format (APF) IIgs graphic files, more work needs to be done besides passing the scanlines as parameters to the 2 methods.

The project can be downloaded or previewed at

https://github.com/markpmlim/PackBytesAndUnpackBytes

A major part of the credit should go to the authors of packBytes and unpackBytes who had made their source  code available.

Although Xcode 7.01 was used, it is possible to convert the code to a lower version.

Regards.

Mark

[toc] | [next] | [standalone]

#2704

markpmlim@yahoo.com.sg writes:

> Using resources available on the Internet, the equivalent of the Apple
> IIgs misc tool calls packBytes and unpackBytes have been coded in
> Swift.
>
> The Swift playground project demonstrates how to encode and decode
> files using the 2 methods packBytes and unpackBytes. If you wish to
> encode/decode Apple Preferred Format (APF) IIgs graphic files, more
> work needs to be done besides passing the scanlines as parameters to
> the 2 methods.
>
> The project can be downloaded or previewed at
>
> https://github.com/markpmlim/PackBytesAndUnpackBytes
>
> A major part of the credit should go to the authors of packBytes and
> unpackBytes who had made their source code available.
>
> Although Xcode 7.01 was used, it is possible to convert the code to a
> lower version.
>
> Regards.
>
> Mark


Interesting.   FWIW, here's unpackBytes in Python that I wrote several
years back:


def unpackBytes(data):
    """
    data is a bytearray

    PackBytes is a IIgs Toolset method.  It is used to compress
    various formats of Super Hi-res pictures.

     00xxxxxx  - 1 to 64 bytes follow - all different
       bits 0-5 are the count - 1
     01xxxxxx  - 3, 5, 6, or 7 repeats of next byte
       bits 0-5 are 2, 4, 5, or 6.  Add one to get count.
       [I've read some other stuff stating that these can have other
        values, but the Apple documentation does state these limitations.]
     10xxxxxx  - 1 to 64 repeats of next 4 bytes
       bits 0-5 are count - 1.
     11xxxxxx  - 1 to 64 repeats of next byte taken as 4 bytes
       bits 0-5 are count - 1
           In other words, this repeats the byte 4 - 256 times

     See: http://www.fadden.com/techmisc/hdc/lesson02.htm
          http://web.pdx.edu/~heiss/technotes/iigs/tn.iigs.094.html
    """
    out = bytearray()
    L = len(data)
    i = 0

    while i < L:
        b = data[i]
        i += 1

        op = (b & 0xc0) >> 6
        count = (b & 0x3f) + 1

        if op == 0:
            out += data[i:i+count]
            i += count
        elif op == 1:
            #if count not in [3, 5, 6, 7]:
            #    sys.stderr.write('COUNT %d not in [3,5,6,7] '
            #                      'at offset %d (0x%x)\n'
            #                      % (count, i-1, i-1))
            #    # assert count in [3, 5, 6, 7]
            out += data[i:i + 1] * count
            i += 1
        elif op == 2:
            out += data[i:i + 4] * count
            i += 4
        elif op == 3:
            out += data[i:i + 1] * (4 * count)
            i += 1
        else:
            assert False
    return out


--
Jerry    awanderin at gmail dot com

[toc] | [prev] | [next] | [standalone]

#2725

On Thursday, 28 April 2016 03:04:48 UTC-5, mark...@yahoo.com.sg  wrote:
> Using resources available on the Internet, the equivalent of the Apple IIgs misc tool calls packBytes and unpackBytes have been coded in Swift. 
> 
> The Swift playground project demonstrates how to encode and decode files using the 2 methods packBytes and unpackBytes. If you wish to encode/decode  Apple Preferred Format (APF) IIgs graphic files, more work needs to be done besides passing the scanlines as parameters to the 2 methods.
> 
> The project can be downloaded or previewed at
> 
> https://github.com/markpmlim/PackBytesAndUnpackBytes
> 
> A major part of the credit should go to the authors of packBytes and unpackBytes who had made their source  code available.
> 
> Although Xcode 7.01 was used, it is possible to convert the code to a lower version.
> 
> Regards.
> 
> Mark

Hi Mark,

There are better methods than the original to encode APF and that problem was solved in C in 2014... you may want to reference the following PDF...

http://www.appleoldies.ca/graphics/shr/PortableHackBytesMay2014.pdf

Decoding APF is trivial. Andy does so in CiderPress. 

Also Sheldon Simms provides a packbytes encoder in his tohgr utility...

Here's my orginal post from May 2014... I can't say that the original authors deserve any credit at all... they don't. Their encoding method is inneficient compared to my equivalent.

Read about it here:
http://www.appleoldies.ca/graphics/shr/PortableHackBytesMay2014.pdf

Download it here:
http://www.appleoldies.ca/graphics/shr/P2P.zip

Complete with MinGW (gcc) Win32 executable, source, test images and
documentation. Open Source. See licence for details.

This article provides a working program called "p2p" as a practical and
portable C language programming example of using Apple IIgs "PackBytes"
Run-Length Encoding (RLE).

Also provided in this article is a relatively detailed description of
Portable HackBytes. This is the PackBytes encoder that I have written, and
that I demonstrate in the p2p program and explain in this document.

Portable HackBytes achieves slightly better compression than Apple Computer's
PackBytes encoder by using a simple list processor and simple logic. Future
optimizations have been identified but have not been implemented for the
first release (this bundle). Whether or not I take this further as a command
line utility will depend largely on the degree of interest in more efficient
PackBytes data compression.

HackBytes and Apple Computer's less robust PackBytes encoded SHR images both
unpack the same way. But not all encoders are created equal. Besides the
smaller encoded size provided by HackBytes it is certainly much faster to
pack or unpack SHR images on a modern computer.

The RLE list processor included in this release is adaptable to other RLE
schemes of the same vintage as well.

P2P(C) Version 1.0 Copyright Bill Buckels 2014
All Rights Reserved.
Portable HackBytes(C) Copyright Bill Buckels 2014
All Rights Reserved.
UnPackBytes(c) Copyright (c) 2007, FaddenSoft, LLC
All rights reserved.
Raw format to Portable HackBytes Encoded format:
Default:
Usage: "p2p file.shr" or "p2p file.sh3"
Output: file.pnt
Option -p (Packed Screen)
"p2p file.shr -p" or "p2p file.sh3 -p"
Output: file.pak or file.pa3
Option -u (UnPack)
PackBytes Encoded format to Raw format:
Usage: "p2p file.pnt -u" or "p2p file.pak -u"
Output: file.shr
"p2p file.pnt -u" or "p2p file.pa3 -u"
Output: file.sh3
Option -t (CiderPress File Attribute Preservation Tags)
Option -s (-s1 to -s4 Singleton Thresholds)
Option -q disable HackQuads (for demo purposes only)
See Source Code and Documentation for More Information!

Anything you read on PackBytes would probably lead you to believe that 4
different StereoTypes of mutually exclusive encoding exist. Common sense and
HackBytes say that PackBytes has only two types of encoding  and not 4!

"Classic" run length encoding schemes are somewhat simpler than PackBytes
encoding but the idea is still the same; encode Singletons and Repeats
separately.

I am also very pleased to retract my initial thoughts of some less robust
PackBytes encoder that I had originally written. This is all covered in my
article.

Unfortunately I am forced to withdraw my first wish to release this under
Creative Commons. That just isn't possible in its present form and I am
inclined to leave this part as is since my own purposes are not affected.

I realize this is a rather trivial effort, something like putting printf
into a loop to test the performance of a C compiler, but this is something I
think needed to be done in order for me to move-on with some more advanced
efforts for working with SHR images that I will be releasing for Win32
shortly

Note Also: Due to lack of interest I will no longer be providing binaries
from my websites for Linux or OSX. Unless of course I see a better effort.
After all, it takes much less time for a Linux or OSX user to run a compiler
than it does to write a program. I will also not accept binaries from
anonymous sources. That's just about being careful and not damaging anyone's
computer.

This decision of mine would simplify the release of my proposed Apple II
Image Studio. I initially intended to provide desktop binaries for use in
Linux and OSX. But I already have much of the code written for a simpler
Win32 version in MicroSoft C which is much easier for me. Since I doubt if I
will port my studio to other platforms when complete due to lack of support
from Linux and OSX users in providing me with binaries of my utilities for
those respective platforms, in some small way this studio of mine becomes
like AppleWin and CiderPress, and accessible to the majority of computer
users who are of course Windows users, but not available otherwise.

That's really too bad from my view but facts is facts. There seems little
interest despite the fun that can be had from any of this so a guy just
needs to cut his losses.

Best Regards,

Bill
 

[toc] | [prev] | [next] | [standalone]

#2726

On Saturday, May 14, 2016 at 11:13:34 AM UTC-7, Bill Buckels wrote:
> > https://github.com/markpmlim/PackBytesAndUnpackBytes

Nice writeup!  

Ignoring your taste in beer, although forgiven with the ladies, is that a bug or feature that the Budweiser Girls picture is "x flipped" on page 28? :-)

I would have also liked to see an Appendix A, with a list of filenames, and a link to the original source art, for *ahem* independent verification. :-)

[toc] | [prev] | [next] | [standalone]

#2727

On Saturday, 14 May 2016 15:27:56 UTC-5, Michael Pohoreski  wrote:
> On Saturday, May 14, 2016 at 11:13:34 AM UTC-7, Bill Buckels wrote:
> > > https://github.com/markpmlim/PackBytesAndUnpackBytes
> 
> Nice writeup!  
> 
> Ignoring your taste in beer, although forgiven with the ladies, is that a bug or feature that the Budweiser Girls picture is "x flipped" on page 28? :-)
> 
> I would have also liked to see an Appendix A, with a list of filenames, and a link to the original source art, for *ahem* independent verification. :-)

Blame the proof-reading on either the glaucoma or the pot... your choice. The source "art" is here along with hard drives filled with conversions. Most of this died due to lack of interest. When one continues to provide better everything than anyone else people lose interest. You know the feeling :)

So since no interest, no uploads to my websites... no corrections or revisions to my documents... my work continues in my fortress of solitude with nothing left to prove :)

My taste in beer tends toward Craft Beer like IPA and Stout... so continue to ignore my taste in beer ads.

Over and Under,

Bill

[toc] | [prev] | [next] | [standalone]

#2728

On Saturday, 14 May 2016 15:27:56 UTC-5, Michael Pohoreski  wrote:
> Ignoring your taste in beer, although forgiven with the ladies, is that a bug or feature that the Budweiser Girls picture is "x flipped" on page 28? :-)

I wanted to say that this was deliberate to avoid copyright infringement issues, but no I longer have the energy to BS for fun so didn't lie... I have some conversions under way of old comic book heroes that require any spare energy left over from the pursuit of life and reality :) No point in wasting energy...

Bill

[toc] | [prev] | [next] | [standalone]

#2729

On Sunday, May 15, 2016 at 2:43:23 AM UTC-7, Bill Buckels wrote:
> I wanted to say that this was deliberate to avoid copyright infringement issues, but no I longer have the energy to BS for fun so didn't lie... I have some conversions under way of old comic book heroes that require any spare energy left over from the pursuit of life and reality :) No point in wasting energy...
> 
> Bill

Looking forward to your next "publication" !

[toc] | [prev] | [next] | [standalone]

#2730

Hi Bill,

> Also Sheldon Simms provides a packbytes encoder in his tohgr utility...

The Swift version of packBytes was ported from the source code of this utility.

> Their encoding method is inneficient compared to my equivalent.

A few weeks ago, I had written a simple utility called APFConverter (in Objective-C) that could batch convert SHR and Brooks 3200 graphic files to APF by incorporating the packBytes function from the tohgr program.

This morning I incorporated enough code from your p2p source code into APFConverter to test the HackBytes function. 

Here are some statistics:

                 HackBytes  PackBytes
ASTRONUT.APF     22 273     27 379
BUTTERFLY.APF    11 476     12 119
CLOWN.APF        24 564     25 076
FLOWER.APF       18 306     19 473
OWL.APF          23 645     23 733

BTW, I had completed QuickLookSHR so I could view the APFs on the Mac OS X desktop. The following suffixes are used by QuickLookSHR to determine how to decode an Apple IIgs graphic and present it on the Mac desktop.

PWF, PAK, DG, APF, 3201, 3200, SHR

>  Future optimizations have been identified

I look forward to that.

> Besides the smaller encoded size provided by HackBytes it is certainly much
> faster to or unpack SHR images on a modern computer. 

Modern microprocessors uses multiple threads so the perceived difference in speed to encode or decode a file is much greater.


Regards,
 
Mark

[toc] | [prev] | [next] | [standalone]

#2731

On Monday, May 16, 2016 at 10:27:08 AM UTC+9, mark...@yahoo.com.sg wrote:
> Hi Bill,
> 
> > Also Sheldon Simms provides a packbytes encoder in his tohgr utility...
> 
> The Swift version of packBytes was ported from the source code of this utility.

I feel flattered. I had forgotten that there was a packbytes function in there until Bill mentioned it and the whole thing is more of a crufty hack than an engineered solution.

[toc] | [prev] | [next] | [standalone]

#2732

 
> I feel flattered. I had forgotten that there was a packbytes function in there until Bill mentioned it and the whole thing is more of a crufty hack than an engineered solution.

As far as I am concerned, you deserve credit for putting in effort.

It would make my day if someone could come out with the source code of an encoder for Dream Graphics.

BTW, MacOSX 10.11 has a compression library that supports LZ4 compression.

Regards.

Mark

[toc] | [prev] | [next] | [standalone]

#2733

> It would make my day if someone could come out with the source code of an encoder for Dream Graphics.

Never mind. I managed to write an encoder that can generate Dream Graphics compressed files by porting the GIF encoder that I had written in Swift


Mark

[toc] | [prev] | [next] | [standalone]

#2827

On Wednesday, 18 May 2016 06:52:33 UTC-5, mark...@yahoo.com.sg  wrote:
> > It would make my day if someone could come out with the source code of an encoder for Dream Graphics.
> 
> Never mind. I managed to write an encoder that can generate Dream Graphics compressed files by porting the GIF encoder that I had written in Swift
> 
> 
> Mark

Hi Mark,

Sorry I've been working night and day until almost just now so no time to respond. Glad everyone is still plugging away :)

I have a pile of work to post, but so many loose ends and major computer crashes, so I won't be coming-up for air for awhile yet.

Your exercise in packbytes and hackbytes was much appreciated... Sheldon is not the only one who feels flattered :)

Regards,

Bill   

[toc] | [prev] | [next] | [standalone]

#2834

Hi Bill,

> Your exercise in packbytes and hackbytes was much appreciated... Sheldon is not the only one who feels flattered :)

You are welcomed.

I have just finished writing another RLE encoder 2 days ago. The corresponding decoder had been written for QuickLookSHR so that compressed Apple IIgs graphics with the suffix 3201 could be viewed on the Mac OS X desktop. This kind of files have a header with the following bytes: 0xCE, 0xD2, 0xCC, 0x00 (NRL with msb on). 

Now, we have 5 encoders viz. packBytes, hackBytes, nrlPack, lzwPack and, of course, LZ4_compress_HC. A program could be written to batch compress $C1/$0000 and $C1/$0002 graphic images to any of the following formats: PWF, APF, 3201, DG, PAK and LZ4.

Regards.

Mark

[toc] | [prev] | [next] | [standalone]

#4778 — Re: Load and display an Apple IIGS SHR graphic

Hi,

The PackBytesAndUnpackBytes project has been upgraded to Swift version 3.x; XCode 8.x or later is required to run this program.

At the same time, I have uploaded a simple Swift Playground demo that can display unpacked Apple IIGS Picture files ($C1/$0000) with a resolution of 320x200.

https://github.com/markpmlim/GraphicConverterIIGS

With a little extra effort, a programmer could use the source code of the above demo with that in the PackBytesAndUnpackBytes demo to produce a Swift Playground (or full-fledged program) to load and display a packed Apple IIGS packed 320 x 200 graphics ($C0/$0001).

With a little modification to the converter code, 3 200 Apple IIGS graphics ($C0/$0002) could be loaded and displayed. Note: the entries of each of the 200 color tables are saved in reverse order.

Naturally, more would need to be done to support "Apple Preferred Format" graphics ($C0/$0002). In particular, the code must take into account the width and height of such graphics may not be standard.

An experienced programmer should have no problems adding support for 640x200 or 3 200 Apple IIGS graphics. Or even those in DreamGrafix format ($C0/$8005).

[toc] | [prev] | [next] | [standalone]

#2847

Here's the source of a MACE decoder I did years ago.

https://github.com/Olde-Skuul/burgerlib/blob/master/source/audio/brmace.cpp

[toc] | [prev] | [next] | [standalone]

#2848

On Wednesday, July 27, 2016 at 5:53:00 PM UTC-7, burge...@gmail.com wrote:
> Here's the source of a MACE decoder I did years ago.
> 
> https://github.com/Olde-Skuul/burgerlib/blob/master/source/audio/brmace.cpp

And Unpackbytes https://github.com/Olde-Skuul/burgerlib/blob/master/source/compression/brunpackbytes.cpp

[toc] | [prev] | [next] | [standalone]

#2849

On Wednesday, July 27, 2016 at 5:53:00 PM UTC-7, burge...@gmail.com wrote:
> Here's the source of a MACE decoder I did years ago.
> 
> https://github.com/Olde-Skuul/burgerlib/blob/master/source/audio/brmace.cpp

Thanks for the free code !

Oh wow ... old school game dev! Always nice to meet another one!   o/ o7

> I'm responsible for games such as The Bard's Tale III: The Thief of Fate, Dragon Wars, Tass Times in ToneTown, Borrowed Time, Mindshadow, The Tracer Sanction, Out of This World SNES and dozens of Macintosh and Apple IIgs conversions.
> In 1983, I've helped found Interplay Productions ...

[toc] | [prev] | [next] | [standalone]

#4779

Hi,

The PackBytesAndUnpackBytes project has been upgraded to Swift version 3.x; XCode 8.x or later is required to run this program.

At the same time, I have uploaded a simple Playground demo that can display unpacked Apple IIGS Picture files ($C1/$0000) with a resolution of 320x200.

https://github.com/markpmlim/GraphicConverterIIGS

With a little extra effort, a programmer could use the source code of the above demo with that in the PackBytesAndUnpackBytes demo to produce a Swift Playground (or fully-fledged program) to load and display a packed Apple IIGS packed 320 x 200 graphics ($C0/$0001).

With a little modification to the converter code, 3 200 Apple IIGS graphics ($C1/$0002) could be loaded and displayed. Note: the entries of each of the 200 color tables are saved in reverse order.

Mark

[toc] | [prev] | [next] | [standalone]

#4920

Finally, I managed to write a metal kernel function (compute shader) that converts an unpacked Apple IIGS graphic (format $C1/$0000) and write it to an instance of MTLTexture. The calling code in the Swift playground demo calls a pair of vertex-fragment functions to render the graphic as a textured rectangle.

Currently, I am working on the documentation of this demo and would appreciate if someone out there could review the details of this documentation to ensure it is a) accurate and b) clear.

The Swift playground demo will be posted on github when the documentation done.

Thanks.

[toc] | [prev] | [next] | [standalone]

#4924

The Swift playground demo named MetalConverter320 is available at:

https://github.com/markpmlim/MetalConverter320

PM

[toc] | [prev] | [standalone]

Back to top | Article view | comp.sys.apple2.programmer

csiph-web