Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #905 > unrolled thread

New lossless photo codec

Back to article view | Back to comp.compression

Contents

  New lossless photo codec Noob <root@127.0.0.1> - 2012-01-31 14:30 +0100
    Re: New lossless photo codec Thomas Richter <thor@math.tu-berlin.de> - 2012-01-31 18:15 +0100
      Re: New lossless photo codec Jim Leonard <mobygamer@gmail.com> - 2012-02-01 10:35 -0800
        Re: New lossless photo codec Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-02-07 15:28 +0200
          Re: New lossless photo codec Jim Leonard <mobygamer@gmail.com> - 2012-02-07 07:05 -0800
            Re: New lossless photo codec Thomas Richter <thor@math.tu-berlin.de> - 2012-02-07 16:43 +0100
          Re: New lossless photo codec BGB <cr88192@hotmail.com> - 2012-02-13 14:51 -0700
            Re: New lossless photo codec Thomas Richter <thor@math.tu-berlin.de> - 2012-02-17 18:22 +0100
              Re: New lossless photo codec BGB <cr88192@hotmail.com> - 2012-02-17 15:48 -0700
                Re: New lossless photo codec BGB <cr88192@hotmail.com> - 2012-02-17 17:12 -0700

#905 — New lossless photo codec

Saw this on LWN, some of you may be interested.

https://kdepepo.wordpress.com/2012/01/30/fast-lossless-color-image-compression/

  via http://lwn.net/Articles/478045/

Cheers.

[toc] | [next] | [standalone]

#906

Am 31.01.2012 14:30, schrieb Noob:
> Saw this on LWN, some of you may be interested.
>
> https://kdepepo.wordpress.com/2012/01/30/fast-lossless-color-image-compression/
>
>    via http://lwn.net/Articles/478045/

Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a 
lot faster than PNG if implemented properly, and not at all slower. And 
the HP patent they mention is available under a royality free licence, 
as with all patents that cover JPEG standards. Just for the records, 
Weinberger - at HP labs - invented LS, and yes, they granted free access 
to the license.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#926

On Jan 31, 11:15 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
> Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
> lot faster than PNG if implemented properly, and not at all slower.

And it's really easy to outperform PNG's compression performance too.
I didn't see the point in creating a new compression method.  It
sounds like he had a knee-jerk reaction to a bad JPEG-LS
implementation.

[toc] | [prev] | [next] | [standalone]

#993

Jim Leonard <mobygamer@gmail.com> writes:
> On Jan 31, 11:15 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
> > Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
> > lot faster than PNG if implemented properly, and not at all slower.
> 
> And it's really easy to outperform PNG's compression performance too.
> I didn't see the point in creating a new compression method.  It
> sounds like he had a knee-jerk reaction to a bad JPEG-LS
> implementation.

Well, the problem was that there were N different incompatible 
losslessly compressed image file formats...

Phil
-- 
Unix is simple. It just takes a genius to understand its simplicity
-- Dennis Ritchie (1941-2011), Unix Co-Creator

[toc] | [prev] | [next] | [standalone]

#995

On Feb 7, 7:28 am, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:
> > And it's really easy to outperform PNG's compression performance too.
> > I didn't see the point in creating a new compression method.  It
> > sounds like he had a knee-jerk reaction to a bad JPEG-LS
> > implementation.
>
> Well, the problem was that there were N different incompatible
> losslessly compressed image file formats...

Obligatory xkcd:  http://xkcd.com/927/

[toc] | [prev] | [next] | [standalone]

#996

On 07.02.2012 16:05, Jim Leonard wrote:

>> Well, the problem was that there were N different incompatible
>> losslessly compressed image file formats...
>
> Obligatory xkcd:  http://xkcd.com/927/

Oh, I actually like standards. There are so many to pick from. (-;

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#1086

On 2/7/2012 6:28 AM, Phil Carmody wrote:
> Jim Leonard<mobygamer@gmail.com>  writes:
>> On Jan 31, 11:15 am, Thomas Richter<t...@math.tu-berlin.de>  wrote:
>>> Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
>>> lot faster than PNG if implemented properly, and not at all slower.
>>
>> And it's really easy to outperform PNG's compression performance too.
>> I didn't see the point in creating a new compression method.  It
>> sounds like he had a knee-jerk reaction to a bad JPEG-LS
>> implementation.
>
> Well, the problem was that there were N different incompatible
> losslessly compressed image file formats...
>

I have mostly just been using PNG and calling it "good enough".
even if one has a new one, and even if it works better, it won't be 
supported by typical graphics apps, ..., sort of defeating the point.


what I might care more about is if there was a standard lossless 
graphics format which supported layers and animation and was supported 
by graphics programs.

there are APNG and MNG, but these fall in the territory of "well, the 
web-browser theoretically supports them, but hardly anything else does...".

practically, this leaves one using either PNG for single-layer static 
images, or dealing with program-specific formats (for having layers and 
so on).


and, in my case, I had before created my own animated PNG like format, 
which differs mostly in that it added basic delta-filtering and 
motion-compensation, but didn't turn out to produce results 
significantly smaller than individually-encoded PNGs in my tests (maybe 
because I partly discovered that apparently block delta filtering and 
Paeth don't mix very well, and would likely require devising some sort 
of "better" predictor, which could utilize 2 frames worth of pixels).

so, I didn't really bother using it, and in my case just stuck with 
using a bunch of individual PNGs with a text file to describe how to use 
them as animation frames (and so on...).


it is possibly the case that even if such a format were created, no one 
could really agree for how the contents of such a format should be 
interpreted and used.

one person then is like:
"ok, I want animation frames and movements";
another is all like "and I want layers and blending";
and another is like "and I want normal and specular maps";
yet another is like "I want 48-bit color depth and ICC";
...

then it is a mess, and if any apps end up supporting it, it is only 
likely to be partial:
app A ignores the existence of layers and normal+specular maps;
app B ignores the existence of animation;
most every one else pretends the format doesn't exist;
...

and nothing really ends up being all that much better off.


or such...

[toc] | [prev] | [next] | [standalone]

#1088

Am 13.02.2012 22:51, schrieb BGB:
> On 2/7/2012 6:28 AM, Phil Carmody wrote:
>> Jim Leonard<mobygamer@gmail.com> writes:
>>> On Jan 31, 11:15 am, Thomas Richter<t...@math.tu-berlin.de> wrote:
>>>> Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
>>>> lot faster than PNG if implemented properly, and not at all slower.
>>>
>>> And it's really easy to outperform PNG's compression performance too.
>>> I didn't see the point in creating a new compression method. It
>>> sounds like he had a knee-jerk reaction to a bad JPEG-LS
>>> implementation.
>>
>> Well, the problem was that there were N different incompatible
>> losslessly compressed image file formats...
>>
>
> I have mostly just been using PNG and calling it "good enough".
> even if one has a new one, and even if it works better, it won't be
> supported by typical graphics apps, ..., sort of defeating the point.
>
>
> what I might care more about is if there was a standard lossless
> graphics format which supported layers and animation and was supported
> by graphics programs.
>
> there are APNG and MNG, but these fall in the territory of "well, the
> web-browser theoretically supports them, but hardly anything else does...".
>
> practically, this leaves one using either PNG for single-layer static
> images, or dealing with program-specific formats (for having layers and
> so on).

The problem is *not* that of the format. The format is there. The 
problem is rather the web-browsers, and this issue cannot be addressed 
by a format.

>
> one person then is like:
> "ok, I want animation frames and movements";
> another is all like "and I want layers and blending";
> and another is like "and I want normal and specular maps";
> yet another is like "I want 48-bit color depth and ICC";
> ...
>
> then it is a mess, and if any apps end up supporting it, it is only
> likely to be partial:
> app A ignores the existence of layers and normal+specular maps;
> app B ignores the existence of animation;
> most every one else pretends the format doesn't exist;

This format does exist. It is called JPEG 2000. But it is not JPEG's 
fault that the browsers do not implement it. Yes, layers, animation 
48bpp color depth, lossless, ICC is all there...

[toc] | [prev] | [next] | [standalone]

#1089

On 2/17/2012 10:22 AM, Thomas Richter wrote:
> Am 13.02.2012 22:51, schrieb BGB:
>> On 2/7/2012 6:28 AM, Phil Carmody wrote:
>>> Jim Leonard<mobygamer@gmail.com> writes:
>>>> On Jan 31, 11:15 am, Thomas Richter<t...@math.tu-berlin.de> wrote:
>>>>> Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
>>>>> lot faster than PNG if implemented properly, and not at all slower.
>>>>
>>>> And it's really easy to outperform PNG's compression performance too.
>>>> I didn't see the point in creating a new compression method. It
>>>> sounds like he had a knee-jerk reaction to a bad JPEG-LS
>>>> implementation.
>>>
>>> Well, the problem was that there were N different incompatible
>>> losslessly compressed image file formats...
>>>
>>
>> I have mostly just been using PNG and calling it "good enough".
>> even if one has a new one, and even if it works better, it won't be
>> supported by typical graphics apps, ..., sort of defeating the point.
>>
>>
>> what I might care more about is if there was a standard lossless
>> graphics format which supported layers and animation and was supported
>> by graphics programs.
>>
>> there are APNG and MNG, but these fall in the territory of "well, the
>> web-browser theoretically supports them, but hardly anything else
>> does...".
>>
>> practically, this leaves one using either PNG for single-layer static
>> images, or dealing with program-specific formats (for having layers and
>> so on).
>
> The problem is *not* that of the format. The format is there. The
> problem is rather the web-browsers, and this issue cannot be addressed
> by a format.
>
>>
>> one person then is like:
>> "ok, I want animation frames and movements";
>> another is all like "and I want layers and blending";
>> and another is like "and I want normal and specular maps";
>> yet another is like "I want 48-bit color depth and ICC";
>> ...
>>
>> then it is a mess, and if any apps end up supporting it, it is only
>> likely to be partial:
>> app A ignores the existence of layers and normal+specular maps;
>> app B ignores the existence of animation;
>> most every one else pretends the format doesn't exist;
>
> This format does exist. It is called JPEG 2000. But it is not JPEG's
> fault that the browsers do not implement it. Yes, layers, animation
> 48bpp color depth, lossless, ICC is all there...
>

one also needs application support though, from things like photo 
editing apps, ...

for example, Paint.NET doesn't have JPEG 2000 support last I checked, ...

but, as noted, people "pretending the format doesn't exist" doesn't mean 
that it "doesn't actually exist", rather "most people don't bother with 
supporting it".


similarly, neither Paint.NET's PDN format, nor GIMP's XCF format, can be 
used by the other app. the annoyance is mostly that each has abilities 
in areas the other is poor at (GIMP is much better for airbrushing, 
Paint.NET is much better for pixel-editing, ...), leading one to often 
shuffle images back and forth between them in the form of PNGs.


OTOH, my 3D engine mostly just supports PNG, JPEG, BMP, TGA, PCX, ... 
(more often it makes sense to compose multi-layer images from multiple 
flat images, using a text-based format to put it all together).

then there are also formats like VTX and DDS, which none of my stuff 
supports, nor do they have graphics-editor support.

a minor annoyance of PNG is partly the time-costs of decompressing them 
(mostly the filtering), which impedes loading times (still faster than 
JPEG though, and TGA takes up too much HD space and may slow down 
loading-times with intensive disk IO...). I mostly ended up opting with 
PNG for having the generally best tradeoffs for my uses (speed vs ratio 
vs quality), and having an alpha channel is a big one (lack of alpha 
blending is a major drawback for JPEG, or at least for the JFIF variant).

also, bicubic spline interpolation can be slow, but is only needed in 
cases where the image size is not a power of 2 (typically, for creating 
textures it is important to have a power-of-2 size). the merit of 
splines though is that they can look much better than linear 
interpolation without being drastically slower.


likewise for "ideal" use in a modern 3D engine, one would likely need 
around 12 channels:
Base: Red, Green, Blue, Alpha;
Normal: X, Y, Z, Height;
Specular: R, G, B, Gloss (exponent).

in practice, one needs 3 or more PNGs to store this (5 is also common, 
with height and gloss as mono images, which typically end up being 
shoved into the alpha components on load).

with JPEGs, this amounts to about 6 images:
RGB, Alpha, Normal XYZ, Height, Specular RGB, Gloss.

it would be interesting if a free graphics-editor would allow images 
with all of these components (as-is, one needs a plug-in just to get 
GIMP to be able to generate normal maps), but one can't get their hopes up.


also even more cool would be if there were some sort of camera which 
could capture all of this.

this would probably need multiple CCDs and the ability to capture 
multiple images close together (say, twin CCD, to help capture depth and 
normal information), and probably using rapid flashes of multiple pure 
colors (from LEDs or similar) to help calculate specular properties (and 
help separate the base-color from the ambient lighting, maybe storing 
the ambient lighting as another RGB triple, or maybe even it could help 
isolate a translucent surface from the background or from reflections, ...).

sadly, if built, such a camera would likely be very expensive (still 
leaving the present situation of dealing with most of this stuff in a 
graphics program, which will still fail to handle all of this in a 
non-clumsy way...).


or such...

[toc] | [prev] | [next] | [standalone]

#1090

On 2/17/2012 3:48 PM, BGB wrote:
> On 2/17/2012 10:22 AM, Thomas Richter wrote:
>> Am 13.02.2012 22:51, schrieb BGB:
>>> On 2/7/2012 6:28 AM, Phil Carmody wrote:
>>>> Jim Leonard<mobygamer@gmail.com> writes:
>>>>> On Jan 31, 11:15 am, Thomas Richter<t...@math.tu-berlin.de> wrote:
>>>>>> Thanks, though I wonder what they are doing. JPEG-LS is *certainly* a
>>>>>> lot faster than PNG if implemented properly, and not at all slower.
>>>>>
>>>>> And it's really easy to outperform PNG's compression performance too.
>>>>> I didn't see the point in creating a new compression method. It
>>>>> sounds like he had a knee-jerk reaction to a bad JPEG-LS
>>>>> implementation.
>>>>
>>>> Well, the problem was that there were N different incompatible
>>>> losslessly compressed image file formats...
>>>>
>>>
>>> I have mostly just been using PNG and calling it "good enough".
>>> even if one has a new one, and even if it works better, it won't be
>>> supported by typical graphics apps, ..., sort of defeating the point.
>>>
>>>
>>> what I might care more about is if there was a standard lossless
>>> graphics format which supported layers and animation and was supported
>>> by graphics programs.
>>>
>>> there are APNG and MNG, but these fall in the territory of "well, the
>>> web-browser theoretically supports them, but hardly anything else
>>> does...".
>>>
>>> practically, this leaves one using either PNG for single-layer static
>>> images, or dealing with program-specific formats (for having layers and
>>> so on).
>>
>> The problem is *not* that of the format. The format is there. The
>> problem is rather the web-browsers, and this issue cannot be addressed
>> by a format.
>>
>>>
>>> one person then is like:
>>> "ok, I want animation frames and movements";
>>> another is all like "and I want layers and blending";
>>> and another is like "and I want normal and specular maps";
>>> yet another is like "I want 48-bit color depth and ICC";
>>> ...
>>>
>>> then it is a mess, and if any apps end up supporting it, it is only
>>> likely to be partial:
>>> app A ignores the existence of layers and normal+specular maps;
>>> app B ignores the existence of animation;
>>> most every one else pretends the format doesn't exist;
>>
>> This format does exist. It is called JPEG 2000. But it is not JPEG's
>> fault that the browsers do not implement it. Yes, layers, animation
>> 48bpp color depth, lossless, ICC is all there...
>>
>
> one also needs application support though, from things like photo
> editing apps, ...
>
> for example, Paint.NET doesn't have JPEG 2000 support last I checked, ...
>
> but, as noted, people "pretending the format doesn't exist" doesn't mean
> that it "doesn't actually exist", rather "most people don't bother with
> supporting it".
>
>
> similarly, neither Paint.NET's PDN format, nor GIMP's XCF format, can be
> used by the other app. the annoyance is mostly that each has abilities
> in areas the other is poor at (GIMP is much better for airbrushing,
> Paint.NET is much better for pixel-editing, ...), leading one to often
> shuffle images back and forth between them in the form of PNGs.
>
>
> OTOH, my 3D engine mostly just supports PNG, JPEG, BMP, TGA, PCX, ...
> (more often it makes sense to compose multi-layer images from multiple
> flat images, using a text-based format to put it all together).
>
> then there are also formats like VTX and DDS, which none of my stuff
> supports, nor do they have graphics-editor support.
>
> a minor annoyance of PNG is partly the time-costs of decompressing them
> (mostly the filtering), which impedes loading times (still faster than
> JPEG though, and TGA takes up too much HD space and may slow down
> loading-times with intensive disk IO...). I mostly ended up opting with
> PNG for having the generally best tradeoffs for my uses (speed vs ratio
> vs quality), and having an alpha channel is a big one (lack of alpha
> blending is a major drawback for JPEG, or at least for the JFIF variant).
>
> also, bicubic spline interpolation can be slow, but is only needed in
> cases where the image size is not a power of 2 (typically, for creating
> textures it is important to have a power-of-2 size). the merit of
> splines though is that they can look much better than linear
> interpolation without being drastically slower.
>
>
> likewise for "ideal" use in a modern 3D engine, one would likely need
> around 12 channels:
> Base: Red, Green, Blue, Alpha;
> Normal: X, Y, Z, Height;
> Specular: R, G, B, Gloss (exponent).
>
> in practice, one needs 3 or more PNGs to store this (5 is also common,
> with height and gloss as mono images, which typically end up being
> shoved into the alpha components on load).
>
> with JPEGs, this amounts to about 6 images:
> RGB, Alpha, Normal XYZ, Height, Specular RGB, Gloss.
>
> it would be interesting if a free graphics-editor would allow images
> with all of these components (as-is, one needs a plug-in just to get
> GIMP to be able to generate normal maps), but one can't get their hopes up.
>
>
> also even more cool would be if there were some sort of camera which
> could capture all of this.
>
> this would probably need multiple CCDs and the ability to capture
> multiple images close together (say, twin CCD, to help capture depth and
> normal information), and probably using rapid flashes of multiple pure
> colors (from LEDs or similar) to help calculate specular properties (and
> help separate the base-color from the ambient lighting, maybe storing
> the ambient lighting as another RGB triple, or maybe even it could help
> isolate a translucent surface from the background or from reflections,
> ...).
>
> sadly, if built, such a camera would likely be very expensive (still
> leaving the present situation of dealing with most of this stuff in a
> graphics program, which will still fail to handle all of this in a
> non-clumsy way...).
>

add (going on a tangent about the above idea):

3D cameras do exist, the main difference would be adding a flash which 
can emit 3 different pure colors (and probably also emit white light), 
and the ability to program the camera to take multiple shots in a row.

example:
first shot, no flash;
second shot, red light;
third shot, green light;
fourth shot, blue light;
fifth shot, white light (all LEDs on).

the rest would then be done by filtering in software.

say:
triangulation to calculate distance;
colored flashes and depth used to separate ambient light vs base color;
delta between flashes and modeled estimates (calculated from depth), and 
the ambient light, can be used to help calculate specular exponents and 
normals (may also produce ambient-light direction vectors), can also 
help better infer specular-color vs base-color.

basically, once one has inferred base-colors, and distance, and one 
knows the brightness of the flash, it should be possible to calculate 
what the image would look like if it were matte and being lit by the 
flash (using idealized normals). if this and ambient light can be 
subtracted out, then the difference should theoretically be due to 
specular and surface normals (normalized color giving the specular 
color, and the value gives the intensity).

these can be used to further separate the specular color from the base 
color, by assuming that the real base color is the estimated base color 
with the specular component subtracted out.

...


more filtering and a secondary pass should allow further isolating out 
light which may have been reflected onto the surface, or may have been 
visible from behind the surface (if translucent), and maybe also help 
separate translucency vs reflection.

the net result of all of this would be "rendering down" the results into 
a new set of images which could hopefully reasonably-accurately mimic 
the original surface in something like a game or similar...

or such...

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compression

csiph-web