Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1482 > unrolled thread

Is there a differential archiving format which works well with random access

Back to article view | Back to comp.compression

Contents

  Is there a differential archiving format which works well with random access w <mcha226@gmail.com> - 2012-08-07 07:38 -0700
    Re: Is there a differential archiving format which works well with random access glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-08-07 15:41 +0000
      Re: Is there a differential archiving format which works well with random access w <mcha226@gmail.com> - 2012-08-08 05:26 -0700
    Re: Is there a differential archiving format which works well with random access Thomas Richter <thor@math.tu-berlin.de> - 2012-08-08 15:37 +0200
      Re: Is there a differential archiving format which works well with random access w <mcha226@gmail.com> - 2012-08-09 09:02 -0700
        Re: Is there a differential archiving format which works well with random access Thomas Richter <thor@math.tu-berlin.de> - 2012-08-11 07:01 +0200

#1482 — Is there a differential archiving format which works well with random access

I am looking for a format to archive multiple images. The images have a special property in that: often series of images will be very similar to each other. The whole image set will have several of these groups (but each group will be quite different to each other).

I am looking for an archiving format which takes advantage of differential storage, but also have a good random access rate.

My conceptual format works like this. Say there are three files, A, A' and A''. They are quite similar to each other. The archive will store A, and the difference A-A', and the difference A-A''.

When accessing A'', it will access A, and then access the difference A-A''. It will output the A merged with that difference.

Since it requires random access, the archive will store the file offset information in an index section.

I am wondering if there already exists a format with similar capabilities.

[toc] | [next] | [standalone]

#1483

w <mcha226@gmail.com> wrote:

> I am looking for a format to archive multiple images. 
> The images have a special property in that: often series 
> of images will be very similar to each other. 
> The whole image set will have several of these groups 
> (but each group will be quite different to each other).

At some point, that is what MPEG does.

It considers a movie as groups of images that are similar and,
for each group, includes one copy of the whole image and then
differences, including various translations (in space).

While MPEG might not be right for you, it might be a good
starting point.

-- glen

[toc] | [prev] | [next] | [standalone]

#1484

glen herrmannsfeldt於 2012年8月7日星期二UTC+8下午11時41分29秒寫道：
> At some point, that is what MPEG does.
> It considers a movie as groups of images that are similar and,
> for each group, includes one copy of the whole image and then
> differences, including various translations (in space).
> While MPEG might not be right for you, it might be a good
> starting point.
> -- glen

I think you are quite right. They are quite similar. After reading the wikipedia, I think the differences are that the differentials are based on adjacent frames but not the key frame, and it is not lossless.

I guess my dream format is waiting to be invented.

[toc] | [prev] | [next] | [standalone]

#1485

Am 07.08.2012 16:38, schrieb w:
> I am looking for a format to archive multiple images. The images have a special property in that: often series of images will be very similar to each other. The whole image set will have several of these groups (but each group will be quite different to each other).
>
> I am looking for an archiving format which takes advantage of differential storage, but also have a good random access rate.
>
> My conceptual format works like this. Say there are three files, A, A' and A''. They are quite similar to each other. The archive will store A, and the difference A-A', and the difference A-A''.
>
> When accessing A'', it will access A, and then access the difference A-A''. It will output the A merged with that difference.
>
> Since it requires random access, the archive will store the file offset information in an index section.
>
> I am wondering if there already exists a format with similar capabilities.

Certainly. JPEG 2000 part 2 with a wavelet transformation or a 
differential color transformation in the third direction would easily do 
that for you. Not that I'm aware of many implementations (I have one you 
can get from Accusoft) but it can do pretty much of what you describe.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#1486

Thomas Richter於 2012年8月8日星期三UTC+8下午9時37分11秒寫道：
> Am 07.08.2012 16:38, schrieb w:
> 
> > I am looking for a format to archive multiple images. The images have a special property in that: often series of images will be very similar to each other. The whole image set will have several of these groups (but each group will be quite different to each other).
> 
> >
> 
> > I am looking for an archiving format which takes advantage of differential storage, but also have a good random access rate.
> 
> >
> 
> > My conceptual format works like this. Say there are three files, A, A' and A''. They are quite similar to each other. The archive will store A, and the difference A-A', and the difference A-A''.
> 
> >
> 
> > When accessing A'', it will access A, and then access the difference A-A''. It will output the A merged with that difference.
> 
> >
> 
> > Since it requires random access, the archive will store the file offset information in an index section.
> 
> >
> 
> > I am wondering if there already exists a format with similar capabilities.
> 
> 
> 
> Certainly. JPEG 2000 part 2 with a wavelet transformation or a 
> 
> differential color transformation in the third direction would easily do 
> 
> that for you. Not that I'm aware of many implementations (I have one you 
> 
> can get from Accusoft) but it can do pretty much of what you describe.
> 
> 
> 
> Greetings,
> 
> 	Thomas

I have just had a look. JPEG2000 part 6 Compound jpm format (http://www.digitalpreservation.gov/formats/fdd/fdd000144.shtml) seems like the one which fits my condition. But you mentioned part 2, which seems to be a single image format. Does it allows for two set of images, each set containing similar images, but two sets are very different? Thanks.

[toc] | [prev] | [next] | [standalone]

#1487

	Thomas
>
> I have just had a look. JPEG2000 part 6 Compound jpm format (http://www.digitalpreservation.gov/formats/fdd/fdd000144.shtml) seems like the one which fits my condition. But you mentioned part 2, which seems to be a single image format. Does it allows for two set of images, each set containing similar images, but two sets are very different? Thanks.

Please format your postings properly, thanks.

Yes, part 2 allows that, too. It may contain multiple images (there 
called compositing layers), but it does not allow images mixed with 
bi-level graphics (in a reasonable way at least) as does part-6.

BTW, I wouldn't expect a high advantage of running an additional 
redundancy reduction over he images. The high-passes are already pretty 
much decorrelated, and the single low-pass that is left is the only part 
of the image that can profit from such a decorrelation transformation. 
But this is the smallest part of the image, so there is not too much gain.

If the images are correlated like in a movie/video, then you probably 
migh want to look into H.264/HEVC.

Greetings,
	Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compression

csiph-web