Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1371

Re: video compression idea

Show all headers | View raw

On 6/29/2012 5:35 AM, Alex Mizrahi wrote:
> Break all video frames into blocks, say, 4x4 ones, as usual.
> Run a clustering on those blocks (lots of them!) using, say, Frobenius
> norm for distance.
>
> Find mean or median for each such cluster.
>
> For each block of each frame we need to find and encode cluster
> identifier. Our first video approximation will be a mean image of that
> cluster.
>
> Cluster ids can be encoded efficiently by taking into account position
> in image (say, blocks on top are likely to be sky) and previous frame.
> (In many cases cluster id doesn't change).
>
> This crude image approximation can be further refined by calculating
> difference between actual block and cluster mean and encoding this
> difference using KLT. KLT will be different for each cluster.
>
> ...
>
> Now tell me why it won't work :). I know that modern video compression
> algorithms use some clever frame decorrelation and psychovisual models,
> but would something like I described above work at least as some crappy
> video codec?
>

it "could" work, but whether or not it would get good performance, 
compress well, and not look like crap, is a bit more debatable.


there is a reason that most major video codecs use at least vaguely 
JPEG-like and MPEG-like strategies (8x8 DCT blocks, color decorrelation, 
and motion-compensation).

partly as they actually compress reasonably well and are not too costly 
to decode in real-time.


meanwhile:
I had recently imagined (once again) doing something similar to PNG but 
for video;
basically, using scanline filtering and motion prediction rather than 
block-based (note that for x-offset, likely the scanline would wrap 
around or similar);
my current motivation was more related to codec decode speed though 
(mostly, since multiple videos will be streamed in at the same time).

I hadn't really gotten to it, mostly as for now things work "well 
enough" with MJPEG (in a somewhat hacked/extended form), whereas with a 
custom codec, no one would be able to see the videos apart from needing 
a special player or similar.


IOW: each scanline has:
1 or 2 bytes for the filter;
1/2 byte x-offset, 1/2 byte y-offset (VLI);
filtered scanline pixels.

the filters would likely differ from those in PNG, and likely include 7 
sample points:
A,B,C,- (like in PNG)
E,F,G,H (likewise, but prior frame).

the 2nd filter byte would likely be for color-filtering modes (possibly 
including some lossy options).

back-end storage would likely use a deflate-like compression scheme 
(likely adapted from my current network protocol, it uses LZ77 and a 
64kB sliding window, but differs mostly in that it uses markers and 
disjoint Huffman tables more like JPEG, albeit still using a more 
Deflate-like table representation, albeit using rice-coding in place of 
Huffman coding for the tables).

possibly, AVI would be used as the container format.


I tried a vaguely similar idea before, but this prior version was not 
significantly smaller than a collection of individually-coded PNG frames.

MJPEG later won out as it worked and it was at least "sort of" common, 
but the JPEG decoding is itself a little costly.

PNG decoding itself also has a bottleneck (namely, given the Paeth 
filter has several conditional jumps, it itself can eat a lot of 
clock-cycles, itself becoming actually the main time-waster in PNG 
decoding vs even inflating the data, so probably a simpler "(A+B)-C" or 
similar filter would be used instead).

so, it is more a tradeoff of either going this route (considering a 
custom/non-standard codec), or seeing about trying to optimize my MJPEG 
decoder more (eventually) and staying with this.


> ...
>
> Rationale: with scheme described above drawing actual pixels on screen
> is incredibly simple and can easily be accelerated even with old
> versions of OpenGL: for each block you draw a bunch of textured rectangles.
>

not actually likely to perform better than simply loading the video 
frames into a texture and drawing said texture.

this is typically how I do video-streaming onto surfaces, and it 
actually seems to perform moderately well if one knows what they are 
doing (note: no mipmaps or texture compression, say, using GL_NEAREST 
for the minimization filter).


> It is also easily scalable: if hardware capabilities or bandwidth is
> lacking, just skip low-energy components of KLT which encode block
> difference. So you'll never see lagging video with this scheme, just
> loss of details.
>

theoretically could also be done with a DCT-based codec.

in practice, this is unlikely, and either way it is likely that the 
codec will display a full-resolution framebuffer.


> Motivation: I'm kinda shocked that I cannot comfortably watch youtube
> videos on my 1.6 GHz laptop. Sometimes it lags, sometimes I need to
> switch quality, sometimes I have to run it in a small window. Ugh...
> I want to make a proof-of-concept codec which would provide smooth video
> playback under any conditions.
>

this is likely because people keep going for H.264 and similar, which 
tends to be fairly costly to decode.

a lot of this isn't due to the "basic" compression technology, but due 
to some of the added things they put onto it (improve compression and 
perceptual quality, such as CABAC and mandatory deblocking filters and 
so on).


if it were MPEG-1, there probably wouldn't be nearly so much of an issue 
here regarding decoder performance.

Back to comp.compression | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

video compression idea Alex Mizrahi <alex.mizrahi@gmail.com> - 2012-06-29 13:35 +0300
  Re: video compression idea BGB <cr88192@hotmail.com> - 2012-06-29 12:04 -0500
    Re: video compression idea Alex Mizrahi <alex.mizrahi@gmail.com> - 2012-07-01 14:55 +0300
      Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-01 12:22 -0500
      Re: video compression idea Jim Leonard <mobygamer@gmail.com> - 2012-07-02 12:04 -0700
        Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-02 18:38 -0500
          Re: video compression idea Jim Leonard <mobygamer@gmail.com> - 2012-07-10 12:01 -0700
            Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-10 17:00 -0500
              Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-10 22:54 -0500
    Re: video compression idea Industrial One <industrial_one@hotmail.com> - 2012-07-02 04:32 -0700
      Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-02 10:38 -0500
      Re: video compression idea Alex Mizrahi <alex.mizrahi@gmail.com> - 2012-07-04 10:23 +0300
        Re: video compression idea BGB <cr88192@hotmail.com> - 2012-07-04 11:56 -0500
          Re: video compression idea "George Johnson" <matrix29@charter.net> - 2012-07-04 22:52 -0400
            Re: video compression idea Industrial One <industrial_one@hotmail.com> - 2012-07-11 09:03 -0700
  Re: video compression idea SG <s.gesemann@gmail.com> - 2012-07-02 01:44 -0700
  Re: video compression idea Noob <root@127.0.0.1> - 2012-07-02 12:52 +0200

csiph-web