Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #203 > unrolled thread

removing predictable crap from poor quality jpegs

Back to article view | Back to comp.compression

Contents

  removing predictable crap from poor quality jpegs Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-05-03 13:32 +0300
    Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-03 13:53 +0200
      Re: removing predictable crap from poor quality jpegs Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-05-03 16:09 +0100
        Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-03 18:10 +0200
          Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-03 18:32 +0200
    Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-03 06:22 -0700
      Re: removing predictable crap from poor quality jpegs Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-05-03 18:03 +0300
        Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-05 04:18 -0700
          Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-05 13:35 +0200
            Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-06 00:23 -0700
              Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-06 14:24 +0200
                Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-06 20:33 +0200
                  Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-08 09:52 +0200
                    Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-08 15:27 +0200
                      Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-08 17:55 +0200
                    Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-08 07:56 -0700
                      Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-08 18:44 +0200
                        Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-08 10:21 -0700
                          Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-09 10:51 +0200
                            Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 11:49 +0200
                              Re: removing predictable crap from poor quality jpegs human <noreply@elfavo.com> - 2011-05-09 12:06 +0200
                                Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 15:29 +0200
                                  Re: removing predictable crap from poor quality jpegs Guido Vollbeding <gv@uc.ag> - 2011-05-09 16:03 +0200
                                  Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-09 22:01 +0200
                                    Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 23:24 +0200
                                      Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-10 16:45 +0200
                            Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-09 04:15 -0700
                              Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-09 14:26 +0200
                                Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 15:51 +0200
                                  Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-09 21:09 +0200
                                    Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 22:28 +0200
                                      Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-09 23:06 +0200
                                        Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-10 17:34 +0200
                                          Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-12 15:01 +0200
                                Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-09 09:40 -0700
                                  Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-09 20:23 +0200
                                    Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-09 22:37 +0200
                                    Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-10 02:27 -0700
                                      Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-10 05:01 -0700
                                      Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-12 15:51 +0200
                                        Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-13 05:07 -0700
                                          Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-13 14:39 +0200
                                          Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-13 15:39 +0200
                                          Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-13 16:36 +0200
                                            Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-13 08:52 -0700
                                              Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-13 19:00 +0200
                                                Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-13 10:07 -0700
                                          Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-13 17:25 +0200
                                            Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-13 10:06 -0700
                                            Re: removing predictable crap from poor quality jpegs Pete Fraser <pfraser@covad.net> - 2011-05-13 18:17 -0700
                                              Re: removing predictable crap from poor quality jpegs Guido Vollbeding <guido@jpegclub.org> - 2011-05-14 09:16 +0200
                                              Re: removing predictable crap from poor quality jpegs Guido Vollbeding <guido@jpegclub.org> - 2011-05-14 10:07 +0200
                                              Re: removing predictable crap from poor quality jpegs Guido Vollbeding <guido@jpegclub.org> - 2011-05-14 19:52 +0200
                                              Re: removing predictable crap from poor quality jpegs Guido Vollbeding <guido@jpegclub.org> - 2011-05-14 21:08 +0200
                                              Re: removing predictable crap from poor quality jpegs Guido Vollbeding <guido@jpegclub.org> - 2011-05-16 08:43 +0200
                                              Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-16 05:15 -0700
                                                Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-16 17:00 +0200
                                                  Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-16 13:02 -0700
                                                    Re: removing predictable crap from poor quality jpegs Thomas Richter <thor@math.tu-berlin.de> - 2011-05-17 09:43 +0200
                                                  Re: removing predictable crap from poor quality jpegs Sebastian <s.gesemann@gmail.com> - 2011-05-19 02:55 -0700
                                                Re: removing predictable crap from poor quality jpegs Noob <root@127.0.0.1> - 2011-05-20 13:23 +0200
                                                  Re: removing predictable crap from poor quality jpegs Pete Fraser <pfraser@covad.net> - 2011-05-20 05:38 -0700
                                      Re: removing predictable crap from poor quality jpegs human <noreplay@elfavo.com> - 2011-05-12 20:15 +0200

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#229

Thomas Richter wrote:
> 
> No, you aren't. You are, at best, responsible for a specific
> implementation. The folks responsible for the standard are we. (-;

You perhaps haven't noticed that your standard is obsolete.
It lags behind the reality, and bringing that in sync would
be one of the tasks we would have to talk about.
That way you could retain your responsibility at least for
the paper - currently you have lost even that, because, as
said, you are busy with too many mistakes.

ciao
human

[toc] | [prev] | [next] | [standalone]

#232

Am 09.05.2011 12:06, schrieb human:
> Thomas Richter wrote:
>>
>> No, you aren't. You are, at best, responsible for a specific
>> implementation. The folks responsible for the standard are we. (-;
>
> You perhaps haven't noticed that your standard is obsolete.
> It lags behind the reality, and bringing that in sync would
> be one of the tasks we would have to talk about.

What do you think why I'm approaching you? Look, I *know* that JPEG-1 
requires maintaince, and not only me. And, as I may add, you - as quite 
often - only know half of what you need to know and should know. And yet 
again I'm again trying to fix this. Instead, you seem to prefer to react 
as a sociopath and prefer to insult everone not sharing your opinion. 
But as always, things aren't as "simple" or easy as they seem...

I'm really trying to get something moving, I can show you the path, I 
can give you the hand, but really, this is as far as I can go. It's up 
to you to make the next step. Or as you say "you can lead a horse to the 
water, but you cannot make it drink". Now, your turn.

> That way you could retain your responsibility at least for
> the paper - currently you have lost even that, because, as
> said, you are busy with too many mistakes.

No Guido, not at all. You are full of prejustice, and so much proud of 
yourself that you did never consider to look left or right. For example, 
I did mention the paper - this is how things work and should be done. 
I'm investigating algorithms like the one Dennis proposed (which is 
really pretty good), and I'm looking at things and testing things. I'm 
contacting people that may have to say something. Or to put it in a 
different way, to be sure whether something is a mistake or not one has 
to try a lot of options. Which you don't seem to do. And old JPEG did 
make a lot of compromises you wouldn't do that way today for reasons you 
probably are not yet aware of.

Anyhow, enough of that. My email is still valid. And empty. Pick your 
choice, and pick it careful. I'm trying to get things going, and again, 
I'm not your enemy. My interest is solely to get JPEG moving forward. 
Not at all costs, though.

[toc] | [prev] | [next] | [standalone]

#234

Thomas Richter wrote:
> 
> What do you think why I'm approaching you? Look, I *know* that JPEG-1
> requires maintaince, and not only me.

Nice, the first thing you will have to learn is that there is no
such thing as "JPEG-1" in reality, that is again a term created
only in your speculation domain, I will not use it.  There is the
original JPEG standard paper, and in reality, there is currently
JPEG 8.  Nothing else.
Reality comes first, and papers are not the reality, and not
primary, they are secondary.  That's why I can leave the
responsibility for the paper to you.

> I'm really trying to get something moving, I can show you the path, I
> can give you the hand, but really, this is as far as I can go. It's up
> to you to make the next step. Or as you say "you can lead a horse to the
> water, but you cannot make it drink". Now, your turn.

Yes, I understand and appreciate that, thank you.
But you see my position, we will see how it can work.
I will have to talk to my people.

> I'm investigating algorithms like the one Dennis proposed (which is
> really pretty good), and I'm looking at things and testing things. I'm
> contacting people that may have to say something. Or to put it in a
> different way, to be sure whether something is a mistake or not one has
> to try a lot of options.

Oh my God, that is your usual way of "investigation" which is
considerably deficient.  I have not seen anything worth to look
at from this Dennis, this will sure be another mistake.
Oh oh oh, that's why my doubt, but we may try anyway...

ciao
human

[toc] | [prev] | [next] | [standalone]

#239

Thomas Richter wrote:
> 
> And old JPEG did make a lot of compromises you wouldn't do that
> way today for reasons you probably are not yet aware of.

Of course, I don't create the problems as your unconscious mind does.

I say that that "old JPEG" is a useful basis for further development.
We are now at JPEG 8, which has unique features which cannot be found
anywhere else, and there is much more potential.
The only problem is that you don't understand this.

By the way, even half of the original JPEG standard is obsolete
(the lossless and hierarchical modes), because the standard
editors obviously didn't understand the unique DCT properties.
Both features can be provided by our extensions, the core is
available.

For example, v8 is capable for lossless decode.
8c introduced the option to allow lossless encode (-block 1).
8d (next year) will add -rgb option so that conventional
lossless coding can be applied.  I'm currently using cjpeg
-block 1 -rgb -arithmetic for lossless image encodes in JPEG
format - can be decoded since v8.  Of course, the compression
is not the highest due to the simple DC coding model, but it
is very useful to have that feature in the standard JPEG codec.
And if necessary, there is potential to extend the coding model.
That I mean with "everything can be done".  And I insist that
it can and should be done in the given JPEG framework for
compatibility, there is no need to introduce incompatible
formats.

ciao
human

[toc] | [prev] | [next] | [standalone]

#244

On 09.05.2011 22:01, human wrote:
> Thomas Richter wrote:
>>
>> And old JPEG did make a lot of compromises you wouldn't do that
>> way today for reasons you probably are not yet aware of.
>
> By the way, even half of the original JPEG standard is obsolete
> (the lossless and hierarchical modes),

Hierarchical is not only for scaling, you know. Of course, as you're 
wearing your "block glasses" again, you cannot see that. And please, 
don't tell me what I "supposed to not know" in your eyes. You know a 
damn little of what I know, in fact.

> For example, v8 is capable for lossless decode.
> 8c introduced the option to allow lossless encode (-block 1).
> 8d (next year) will add -rgb option so that conventional
> lossless coding can be applied.  I'm currently using cjpeg
> -block 1 -rgb -arithmetic for lossless image encodes in JPEG
> format - can be decoded since v8.  Of course, the compression
> is not the highest due to the simple DC coding model, but it
> is very useful to have that feature in the standard JPEG codec.
> And if necessary, there is potential to extend the coding model.
> That I mean with "everything can be done".  And I insist that
> it can and should be done in the given JPEG framework for
> compatibility, there is no need to introduce incompatible
> formats.

I don't care much about a specific implementation, you should know this. 
The problem with "extending the coding model" is not so much as that I 
could extend it. The problem is to extend it in such a way that it 
integrates and plays nicely with the existing toolchain. I've probably 
said this before, but I want to stress this. That is, anything that 
breaks existing codecs I don't care about at this point.

I'm not sure what you mean by "lossless decode", or at least this term 
admits several interpretations. You could either mean that the 
encoding-decoding chain is idempotent (which requires basically a 
lifting-type integer based DCT, which is possible), or you mean that the 
encoder/decoder chain *as such* is lossless, which requires either the 
lossless chain of JPEG (which I don't care about), or it requires 
preserving additional bits in the coefficients, which is *likely* not 
compatible to the way JPEG-1 defined the quantizer (but I need to 
compute a bit more on this), again unless you twiddle the bits into the 
stream somewhere else, which should also be possible.

In either case, yes, lossless DCT is possible, sure (and as such not 
new, it's in the literature).

Again, I'm not such much into extending the coding model *except* if the 
additional bits can be placed in such a way into the codestream that 
existing codecs don't care, e.g. if the data goes into the APP markers.

And, I neither care about arithmetic, simply on the basis that it never 
became popular, so it breaks programs (like it or not). But highest 
compression performance is not at all the goal here in first place, I 
might also have said this.

If things *cannot* be decoded with anything JPEG compliant (say, v6), I 
don't care. If decoding with v6 creates additional loss compared to if I 
would decode with v8 - that would be OK. Additional loss can be 
tolerated as long as the image is acceptable. What is not tolerated is 
anything that cannot be decoded "reasonably" by a JPEG compliant 
decoder. And JPEG compliant decoder means exactly that: The old fart 
standard. Not anything you made up yourself.

Thus, retrofitting bits into the old codestream such that a compliant 
decoder doesn't complain, may possibly not get full quality.

Understood?

[toc] | [prev] | [next] | [standalone]

#256

Thomas Richter wrote:
> 
> And, I neither care about arithmetic, simply on the basis that it never
> became popular, so it breaks programs (like it or not).

Some people, myself included, find it useful because they use modern
programs for this purpose...
In this particular case cjpeg -block 1 -rgb -arithmetic for lossless
image coding, the arithmetic option does not add another
incompatibility, because the block option requires a SmartScale
capable decoder anyway (>= v8) which then also includes arithmetic
coding.  That's why, whenever one uses a SmartScale feature
(jpegtran -scale or cjpeg -block), one can also give the
arithmetic option without further problem.

There is another reason: In the current SmartScale implementation,
for block sizes < 8, in the Huffman case the optimization is turned
on inside the library, which raises the memory usage considerably -
with block size 1, for every image sample a complete 8x8 DCT
coefficient block would be allocated.  That is horror if you know
that, but it was just the straightforward implementation.
So, because I know that, I avoid that by switch to arithmetic which
is always single scan adaptive and does not have that memory abuse...

ciao
human

[toc] | [prev] | [next] | [standalone]

#230

On 9 Mai, 10:51, human wrote:
> [...]
> See the situation: You and Thomas are great intellectual speculators,
> but you are not responsible for the actual JPEG code and standard
> development.
> [...]

What does this have to do with anything? I said something (*) which is
trivially true and all you do is trying to turn this into a personal
pissing contest without actually disagreeing with me.

(* filtering in an LTI sense requires interaction across blocks if you
want to avoid blocking artefacts )

--
SG

[toc] | [prev] | [next] | [standalone]

#231

Sebastian

In fact I don't dispute that you seem to be a keen individual.
I just want you to understand that there are deeper levels of
knowledge than the usual intellect (logical mind), and as long
as you don't accept this, we will have a difficult communication.
I don't want to argue on your level of knowledge, because I know
it is deficient.  My source of knowledge is a different domain,
and difficult to communicate to someone who doesn't accept this.

> (* filtering in an LTI sense requires interaction across blocks
> if you want to avoid blocking artefacts )

I have a great problem with your notion of "filtering".
I consider it a great source of evil in signal processing, as it
is currently done.  My experience tells me that it is often done
wrong.  And, as said, I have no problem with blocks, as I have
no problem with pixels of digital images.

To place a practical example, which might lead a bit off-topic
but is to the point in my opinion:
What do you think of the digital images taken by conventional
digital cameras, say 99% of them (no matter if JPEG coded or not)?
Your answer to this question would be quite revealing.
If you somehow accept those results, as the majority of people
which are clueless do, then this tells everything and we could
stop to argue.
My answer to this question is: All these digital images and
digital cameras are crap, unacceptable. The production of that
whole industry is just junk, trash, worthless.
And the primary reason for this trash is the filtering of the
crippled mosaic sensors.
There is only one single camera manufacturer in the world which
does not use that crappy filtering technique and produces true
digital images, that is Sigma with non-filtering Foveon sensors.
Everyone else uses this crap and produces fake digital images.
Why do they use this crap?  Have you ever asked?  Why don't you
go there and tell them that what they do is bullshit, instead
of telling me what filtering may be good for?
Look, that's why I don't argue with you about filtering.
As long as you accept that people make such horror with
filtering, I cannot take you seriously.
I cannot take anyone seriously who accepts such nonsense.

ciao
human

[toc] | [prev] | [next] | [standalone]

#233

>> (* filtering in an LTI sense requires interaction across blocks
>> if you want to avoid blocking artefacts )
>
> I have a great problem with your notion of "filtering".
> I consider it a great source of evil in signal processing, as it
> is currently done.  My experience tells me that it is often done
> wrong.  And, as said, I have no problem with blocks, as I have
> no problem with pixels of digital images.

*Sigh* The question was not whether filtering is good or bad. The 
question to begin with was: Given that I need a filter, what is the best 
way to implement it?

Please note that these are two different questions.

The answer to the question of the OP was *not* to do filtering, and this 
was appreciated, and I mentioned it several times. Neither Sebastian nor 
I are stupid. If you know the cause of the error, fix it right there, 
and not in any other way. The amusing fact was that you
seem to indicate that filtering by DCTs is the "ideal" way of 
implementing a filter, which is outright incorrect.

So please, don't shift the discussion.

> To place a practical example, which might lead a bit off-topic
> but is to the point in my opinion:
> What do you think of the digital images taken by conventional
> digital cameras, say 99% of them (no matter if JPEG coded or not)?
> Your answer to this question would be quite revealing.
> If you somehow accept those results, as the majority of people
> which are clueless do, then this tells everything and we could
> stop to argue.
> My answer to this question is: All these digital images and
> digital cameras are crap, unacceptable. The production of that
> whole industry is just junk, trash, worthless.
> And the primary reason for this trash is the filtering of the
> crippled mosaic sensors.

*Sigh*. As everything, it is a compromize. Alternatives with vertically 
stacked sensors have problems of their own, as reduced sensibility and 
poor color calibration. I've seen such pictures as well, and I'm not so 
pleased. There is no silver bullet, and there is no silver bullet in 
sensor design either. Problems with modern cameras are manifold, and the 
primary problem is not the sensor setup, but

a) the "megapixel" insanity, driven by the industry to sell cameras 
under a single quality label called "pixels", and the consumer buying 
this. (We *could* address this deficency by the JPEG by working on 
quality indices that are closer to human perception than "megapixels").

a) causes noisy images and cameras that are unusable except under direct 
exposure of sunlight. Not appreciated. Things seem to have changed a 
little bit in the recent years.

b) Vendor-lock-in by camera vendors selling cameras taping in "raw" 
format, only usable by tools from the vendor itself. Believe it or not, 
but compression is a secondary side effect today, and otherwise 
irrelevant. Lossy JPEG is a bad compromize itself. At least in the 
professional and semi-professional market, consumers do not want any 
compression, but prefer to work as close to the sensor as possible. Old 
JPEG can only inadequately address these needs, which requires higher 
bit depths, encoding of the sensor layout, ...

b) is also a problem for digital archives, libraries and so on because 
it means that images taken today are likely unusable in a couple of 
years because any vendor-specific software able to decypher these 
formats is going to be obsolete and unsupported. Consumers do not yet 
realise this, but will soon learn the problem. JPEG can only partially 
address this, see above.

The problem with b) is that you need the vendors in the boat, which is a 
*very* tough problem because any standardization works against their
market and binding consumers to their products. (And yes, we have spoken 
to these people). It also means that you never have the right people in 
the meeting to discuss this. A chicken and egg problem.

Thus, the market needs to be opened somehow. My current approach is to 
focus more on the consumer side of the professional market where this 
might work.

> There is only one single camera manufacturer in the world which
> does not use that crappy filtering technique and produces true
> digital images, that is Sigma with non-filtering Foveon sensors.

...which also provides crappy images. Yes, I know. I've seen them. We 
have a couple of them in our collection. Mostly noisy, and requires also 
heavy postprocessing. So, please, no advertisment. It is another 
compromize taken, an interesting compromize taken for sure, but no 
solution for the problem at hand either. I believe a) (sensors being too 
small) is much more of a problem than their actual layout. Of course, 
the two are related, but not quite the same.

For example, one could ask the question how much additional sensitivity 
you need for stacked sensors compared to how much sensitivity you need 
for smaller sensors, and *then* compute a bit.

Thus, once again, not everything is "bullshit" because you don't like 
it. You may simply not have enough background to have a broader picture 
on the situation. That's the true problem. And as always, I *could* help 
if you want.

[toc] | [prev] | [next] | [standalone]

#238

Thomas Richter wrote:
> 
> b) is also a problem for digital archives, libraries and so on because
> it means that images taken today are likely unusable in a couple of
> years because any vendor-specific software able to decypher these
> formats is going to be obsolete and unsupported. Consumers do not yet
> realise this, but will soon learn the problem. JPEG can only partially
> address this, see above.
> 
> The problem with b) is that you need the vendors in the boat, which is a
> *very* tough problem because any standardization works against their
> market and binding consumers to their products. (And yes, we have spoken
> to these people). It also means that you never have the right people in
> the meeting to discuss this. A chicken and egg problem.

Such kind of problem is exactly what requires a higher perspective
to be solved.
And that particular problem is created by yourself, the JPEG committee,
by adding confusion and introducing mistakes.
With more understanding you would see that in reality this problem
does not exist - there is already a perfect standard digital image
format, which only needs to be further developed in the right
direction.  IJG has this important statement in the README file:

  In any case, our decoder will remain capable of reading existing
  JPEG image files indefinitely.

So we are committed to retain compatibility for all times.
But everything what the JPEG committee did works against this
promise, by introducing one incompatible mistake after another.
Don't you see that?  And from your statements it appears that
you are not going to stop this insane game.
So you create problems everywhere (of course, you say the blocks
are the problem, so you must find something different, etc. etc.),
and then you complain about the created confusion.
This is exactly the way how the unconscious mind works: It can
only create more problems, not solve anything.  The only way to
get out of this vicious circle is to gain consciousness - and
unfortunately you have no methods to do that, you have never
learned that.
With more understanding you would see a simple solution:
My understanding tells me that everything can be achieved by the
proper further development of the original JPEG technology.

And by the way, the RAW formats are crap anyway, as mentioned,
except those from Sigma/Foveon cameras (.X3F).  One could look
there if these could be replaced by a potential JPEG update.
So the situation is simple with appropriate understanding.

ciao
human

[toc] | [prev] | [next] | [standalone]

#240

On 09.05.2011 21:09, human wrote:
> Thomas Richter wrote:
>>
>> b) is also a problem for digital archives, libraries and so on because
>> it means that images taken today are likely unusable in a couple of
>> years because any vendor-specific software able to decypher these
>> formats is going to be obsolete and unsupported. Consumers do not yet
>> realise this, but will soon learn the problem. JPEG can only partially
>> address this, see above.
>>
>> The problem with b) is that you need the vendors in the boat, which is a
>> *very* tough problem because any standardization works against their
>> market and binding consumers to their products. (And yes, we have spoken
>> to these people). It also means that you never have the right people in
>> the meeting to discuss this. A chicken and egg problem.
>
> Such kind of problem is exactly what requires a higher perspective
> to be solved.
> And that particular problem is created by yourself, the JPEG committee,
> by adding confusion and introducing mistakes.

No "mistakes", the problem is the "good enough" phenomenon. JPEG is used 
as "last resort" nowadays, and "good enough" for most people.

> With more understanding you would see that in reality this problem
> does not exist - there is already a perfect standard digital image
> format, which only needs to be further developed in the right
> direction.  IJG has this important statement in the README file:

Look, as already indicated, JPEG does *not* address the problem.
JPEG is lossy, or to be precise, the only part that has found wide 
adoption in the photography market is lossy. The lossless part is absent 
and hasn't been adopted there at all (neither in your software, btw). In 
the medical market it has been superceeded by JPEG 2000 a while ago 
since this market required higher quality and better loss control. So in 
the digital movie market. All not end user markets, of course, but in 
the professional area.

However, as you might have noticed, the higher ends of the photography 
consumer ("prosumer") market are moving away from JPEG, and use its 
"loss" as an argument for pushing proprietary formats. Addressing this 
market by a lossy(!) IJG code is not the right approach to begin with.

>    In any case, our decoder will remain capable of reading existing
>    JPEG image files indefinitely.

I don't think it is capable of reading even parts of the existing JPEG 
images today, to be precise. It doesn't support the lossless branch (at 
least not the versions I have), and the versions I know require a 
compile time decision for 12bpp, leave alone that it doesn't support 
JPEG files encoded in 15444 or 29199, which are also JPEG files. Or 
JPEG-LS, for that matter, which is just another standard. So well, this 
"promise" is already not true, and never has been true.

> So we are committed to retain compatibility for all times.

I'm not against it, but this compatibility is, in a sense, in the way as 
well. If you check what happened now that all the AC coding patents have 
run out, then you would realize that nothing of that stuff, even though 
standardized, has found a way into the market. Neither have any of the 
extensions (JPEG-3) found adoption, and even found much less adoption 
than any of the later standards (15444-x for example). Thus, any 
possible extension that breaks backwards compatibility is very unlikely 
to be acccepted at this point.

So what has to be done is the following, and this is I think the only 
doable perspective that *might* have a chance: It is necessary to 
retro-fit a lossless version into the existing standard such that it 
remains backwards compatible at one hand, while allows more and higher 
bitdepths on the other. This will most likely require giving up the 
existing 12bpp representation (rarely used anyhow in photography) but 
find a different way of representing such bits in a hybrid approach.

Of course, a couple of "poor" solutions to this problem have been found, 
with vendors writing the "raw" data into the application tags of JPEG-1 
and similar tricks. While possible, this is neither vendor-neutral nor 
completely future-proof.

> But everything what the JPEG committee did works against this
> promise, by introducing one incompatible mistake after another.
> Don't you see that?  And from your statements it appears that
> you are not going to stop this insane game.

You apparently don't understand how standards work. Standards are an 
option people can pick from, they represent a framework to work in. This 
is a different game from science. As it happens to be, the original 
JPEG-1 framework is too tight to work in anymore, and the parts that 
would open up a perspective haven't found wide applications, so no go in 
this area as well.

Thus, I don't think it was unreasonable to start from scratch and 
rebuild something new (which was 15444) which has also found adoption. 
*Except* in photography, basically because people here - vendors and 
customers - depend on the existing toolchain. Extending or even using 
the standard in a way that breaks this toolchain will no longer find 
adoption, but on the other hand, something has to be done or JPEG will 
sooner or later be replaced by a bunch of "raw" formats nobody can 
manage anymore.

Thus, a challenge seeking for a solution.

> So you create problems everywhere (of course, you say the blocks
> are the problem, so you must find something different, etc. etc.),
> and then you complain about the created confusion.

I don't complain about any confusion because I don't have any. You 
probably fail to understand me and interpret this as confusion.

> This is exactly the way how the unconscious mind works: It can
> only create more problems, not solve anything.  The only way to
> get out of this vicious circle is to gain consciousness - and
> unfortunately you have no methods to do that, you have never
> learned that.

Very nice. Ever tried to apply this to yourself? Just for a moment? I 
mean, look, what you claim me to do here is *exactly not* what I'm 
doing, but quite the opposite: Your argument is backwards in the sense 
that you always want to find reasons to preserve what you already know 
(JPEG-1). So which results do you expect to begin with? With this type 
of mindset, you're in the comfortable position of never having to think 
about anything beyond what you know - though you never get there either.

You need to start from the other end. What might be the reasons to go 
beyond it in a way?

> With more understanding you would see a simple solution:
> My understanding tells me that everything can be achieved by the
> proper further development of the original JPEG technology.

The technology had been further developed, that's not the point. Again, 
you argue backwards: *Because* you want to preserve it (this is you 
starting point), it must be preserved. This is not a good argument.

Start from the problem, and what can we do? The problem is, as said, not 
at all anymore compression. Buy a bigger disk, buy a larger card. 
Problem solved, with minimal investment of money. Much less investment I 
need to do for investing into a new toolchain.

The problem is an interchange file format, a representation of 
photographic images that is
a) backwards compatible, and
b) capable of expressing the varity of photographic images in such a way 
that they are *as close as possible* to the sensor, while preserving as 
much as possible of the toolchain.

Parts of these goals are not compatible to each other. Finding a 
compromize is the job of the JPEG.

> And by the way, the RAW formats are crap anyway, as mentioned,
> except those from Sigma/Foveon cameras (.X3F).

Oh my... Again, you start backwards. You like the sensors (for reasons 
beyond me, but let it be as it is), and from that starting point you 
argue that it is a good format. *This is not a scientific argument*.

The argument must start from the problem. And the problem is that we 
have various sensors, not only the Foveon types. And, believe me or not, 
nobody will be accepting a format that is based on such sensor types 
alone, or on a format defined by a single vendor. It's simply not going 
to work this way. Broaden your mind. What else is there? What else do we 
need to express? Sensor layouts may not even be available to begin with, 
they are locked up. So, all not so easy - can we find a way to preserve 
a good amount of data without requiring vendors giving too much IP out 
of their hands?

> One could look
> there if these could be replaced by a potential JPEG update.

Wrong way around. You cannot look at a single vendor and decide from 
there. Where is the problem? Lossless representation (not even 
compression, though that would be a nice add-on) of images in a 
vendor-neutral way preserving the tool-chain.

> So the situation is simple with appropriate understanding.

Oh yeah. Question is, what is "appropriate". From a technological point 
of view, the answer is clear. There are smarter and better ways to 
compress images. However, that is an answer to a question that I don't 
have at this point right now (wearing my JPEG hat, not my scientist 
hat). Thus, the question should rather be: How can we address market 
needs. And *this* question has a different answer from the first question.

As far as I can see, this is not your way of "understanding" because you 
believe you already have the ideal solution. But that stops you from 
searching. You don't. You have *parts* of the solution, but an 
incomplete picture.

Let it be what it is. I think different opinions are necessary in this 
world, and this is why I believe why standards committees work in the 
end, because people have to arrange to find compromises.

[toc] | [prev] | [next] | [standalone]

#242

Thomas Richter wrote:
> 
> Look, as already indicated, JPEG does *not* address the problem.
> JPEG is lossy, or to be precise, the only part that has found wide
> adoption in the photography market is lossy. The lossless part is absent
> and hasn't been adopted there at all (neither in your software, btw).

See my other post, it is ok that it hasn't been adopted because it is
obsolete.  We do have a seamlessly integrated lossless mode since v8.

> In the medical market it has been superceeded by JPEG 2000 a while ago
> since this market required higher quality and better loss control. So in
> the digital movie market. All not end user markets, of course, but in
> the professional area.

What you call "professional area" here is a speculative domain,
so that fits perfectly.  They use unreal technologies, because they
themselves are unreal.  You are healed by nature, or by consciousness,
while the medical market is just a fake for clueless people, so again,
this fits perfectly.

> However, as you might have noticed, the higher ends of the photography
> consumer ("prosumer") market are moving away from JPEG, and use its
> "loss" as an argument for pushing proprietary formats. Addressing this
> market by a lossy(!) IJG code is not the right approach to begin with.

Speculative domain, unreal, nice fit.

> >    In any case, our decoder will remain capable of reading existing
> >    JPEG image files indefinitely.
> 
> I don't think it is capable of reading even parts of the existing JPEG
> images today, to be precise. It doesn't support the lossless branch (at
> least not the versions I have), and the versions I know require a
> compile time decision for 12bpp, leave alone that it doesn't support
> JPEG files encoded in 15444 or 29199, which are also JPEG files. Or
> JPEG-LS, for that matter, which is just another standard. So well, this
> "promise" is already not true, and never has been true.

You have not understood.
With "existing JPEG images" it means files created with prior JPEG
versions in real use.  Again: Your standard papers are not the reality.
The lossless branch is obsolete, as mentioned, and "JPEG-LS", for
that matter, is also obsolete.

ciao
human

[toc] | [prev] | [next] | [standalone]

#252

Am 09.05.2011 23:06, schrieb human:

> You have not understood.
> With "existing JPEG images" it means files created with prior JPEG
> versions in real use.

Oh, there is real use for lossless JPEG, just not in the photography market.

> Again: Your standard papers are not the reality.
> The lossless branch is obsolete, as mentioned, and "JPEG-LS", for
> that matter, is also obsolete.

Actually, have you understood which market LS tries to address? For 
*this* specific market, even JPEG is too heavy-weight.

Remember the Mars Rover? There's been JPEG-LS on it (or rather, LOCO, 
it's precursor before it has been standardized). You don't want to have 
DCTs on this hardware. LS is *a damn lot* simpler than this.

Nevermind. I posted a suggestion, if you want to respond, respond by mail.

[toc] | [prev] | [next] | [standalone]

#257

Thomas Richter wrote:
> 
> You don't want to have DCTs on this hardware.
> LS is *a damn lot* simpler than this.

There is hardly something more simple than a 1-point DCT...
That's how it works in our case - if you only wanted the lossless
coding, you could write a stripped-down implementation to do just
that, without any DCT (well, as said, consider a 1-point DCT).

> Nevermind. I posted a suggestion, if you want to respond,
> respond by mail.

I will.
Shortly.

ciao
human

[toc] | [prev] | [next] | [standalone]

#235

On 9 Mai, 14:26, Guido Vollbeding wrote:
>
> I don't want to argue on your level of knowledge,

I don't think you were even able to because this would require you to
understand what I was actually saying. As it seems, you did not
understand.

> > (* filtering in an LTI sense requires interaction across blocks
> > if you want to avoid blocking artefacts )
>
> I have a great problem with your notion of "filtering".

I would say that this is a standard notion in the DSP realm. You see,
my math/DSP background is quite strong. My experience is not limited
to image processing. It also covers a great deal audio processing and
digital filter design with applications in noise shaping, critically-
sampled filterbanks and more. Applied math and DSP is my bread and
butter. When I use the words "filter" and "filtering" I am almost
always talking about some LTI system. If I talk about filtering
digital images, I mean spatial convolution of the intensity signal
with some filter's impulse response. I am neither talking about
nonlinear operations (i.e. "median filter") nor am I talking about
some kind of color filtering (i.e. "Bayer filter"). It seems you were
not aware of that.

SG

[toc] | [prev] | [next] | [standalone]

#237

Hi Sebastian

That is fine for you if you have such kind of knowledge,
but it is different from the kind of knowledge that I am
talking about.
And certainly you and other folks have not understood the
DCT properly, so that part is lacking in your knowledge,
and it's an essential part.
I know something about the DCT that apparently nobody
else knows.
Now I told you the problem:  The essential part of that
knowledge is apparently not on your level of understanding.
That requires different methods than you have learned at
your schools and universities.
I would have to raise your understanding to a higher level,
and with your background this would probably be a long
and difficult process, even if you would be prepared.
Staying on that higher level of consciousness is essential
for me, I currently don't have the resources to work out
a "theory" which would better fit your level, sorry.
That may be possible sometime later, but as said I can't
do it at the moment, because I have important things to
do for which I am responsible.

ciao
human

[toc] | [prev] | [next] | [standalone]

#241

On 09.05.2011 20:23, human wrote:
> Hi Sebastian
>
> That is fine for you if you have such kind of knowledge,
> but it is different from the kind of knowledge that I am
> talking about.
> And certainly you and other folks have not understood the
> DCT properly, so that part is lacking in your knowledge,
> and it's an essential part.
> I know something about the DCT that apparently nobody
> else knows.

And your "apparently" is wrong. I told you already, many times. But this 
is simply because you don't have enough overview, and you argue from the 
wrong end. It's signal processing 101 we talk about.

Again, there's nothing wrong with the DCT. But advocating it as the best 
invention since sliced bread is neither the truth. You need to know a 
bit more about what else is around in signal processing to decide and 
gain a larger perspective. It is understood that you cannot see it from 
your point yet, and the sad part is that this keeps you from looking. If 
I would only knew a way how to trigger your curiosity without damaging 
your ego. You're really giving me a very hard job, you know...

But again, this is a very *different* argument from the JPEG standards 
argument. If I would wear my Science hat, I would tell you that there is 
quite a damn lot wrong with DCT (well, maybe not "wrong", but not ideal 
for various reasons, block size and the induced inability to collect 
correlations beyond it being one example). But I'm not this time. I'm 
wearing my standards hat, and I'm fine with preserving it. But for other 
reasons. For market reasons.

Understood?

[toc] | [prev] | [next] | [standalone]

#247

On 9 Mai, 20:23, human wrote:
> [...]
> And certainly you and other folks have not understood the
> DCT properly, so that part is lacking in your knowledge,
> and it's an essential part.

You don't know that. You're just assuming.

> I know something about the DCT that apparently nobody
> else knows.
> Now I told you the problem: The essential part of that
> knowledge is apparently not on your level of understanding.
> That requires different methods than you have learned at
> your schools and universities.
> [...]
> I currently don't have the resources to work out
> a "theory" which would better fit your level, sorry.
> [...]

I'm afraid, this sounds like something a crackpot would say to defend
his lack of classical training. If you have something to share, put a
little effort into writing it down. In case you've already done that,
please post a link to it. If this explanation requires defining new
things/methods/properties, so be it. Anything else seems like lazyness
on your part. Don't make it sound like others don't have the mental
capacity to understand what you have to say just because you're unable
and/or too lazy to explain yourself clearly. It takes two to tango.

Anyhow, I still don't see what all this has to do with what I said
earlier. If you're not sure about what I actually meant by "filtering
requires interaction across blocks", feel free to ask. You may have
misunderstood something on account of you not being familiar with DSP
basics. As I already mentioned, by "filtering digital images" I mean
spatial convolution and by "blocks" I mean a non-overlapping partition
of the image and by "requires interaction across blocks" I mean the
opposite of "treating blocks independently from each other". This
should really be a no-brainer, considering that convolution of an
image with some impulse response is equivalent to a superposition of
scaled and shifted images. Obviously, you need information from
surrounding blocks to generate a block of a shifted image. Example in
1D:

original:  ...c d|e f g h|i j...
shifted:      ...|d e f g|...       (where | is a block boundary)

This simple "pixel shift" operation _is_ a filter in the DSP sense
since it's linear and shift-invariant. As you can see, the new block
contains new information (d) from a surrounding block of the original
image that was not present in the old block.  q.e.d.

I suggest you spend some more time on learning DSP basics. For
example, I can recommend dspguide.com as a start. You will gain a new
level of understanding. The title of the other paper I referred to
earlier was actually "A Pixel Is Not A Little Square, A Pixel Is Not A
Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a
Little Cube)" by Alvy Ray Smith. If you think "hey, this does not
apply to real sensors!" you're wrong. It still does. You can think of
a pixel as a point measurement *after* spatial convolution. And this
spatial convolution would be simply part of the anti-alias filter.

SG

[toc] | [prev] | [next] | [standalone]

#249

On 10 Mai, 11:27, Sebastian wrote:
> [...] This
> should really be a no-brainer, considering that convolution of an
> image with some impulse response is equivalent to a superposition of
> scaled and shifted images. Obviously, you need information from
> surrounding blocks to generate a block of a shifted image.

To avoid confusion: by "scaled" I mean scaled sample values. I don't
mean any kind of resampling/resizing of the image.

SG

[toc] | [prev] | [next] | [standalone]

#258

Sebastian wrote:
> 
> Don't make it sound like others don't have the mental
> capacity to understand what you have to say ...

Actually, your mental structure is too heavy, so that it completely
covers up your consciousness, which is finer and different domain.
You are completely identified with your mind, YOU are not present.
Of course, you don't understand what I'm talking about, that's why
a communication is useless.
Even if I would tell you the discovered DCT properties, you would
not be able to grasp it with your rational mind, similarly as one
cannot represent an irrational or transcendent number by a rational
fraction, and yet it is real.
I have done that, trying to explain the fundamental DCT properties
to various kinds of people, and the result is interesting:
The more intellectually educated (=conditioned) the person is, the
less capable to understand.  Their mind is conditioned with lots
of abstract, dead, theory, therefore they are unable to see a
simple real phenomenon.  The less conditioned the person, the more
capable to understand.  Either you see it directly, or you don't,
there is nothing to argue about.
Look, at least the first fundamental DCT property is published.
You can find it.  But you are not even familiar with the IJG code,
as you have said, where you could find the pointer.  That is your
kind of ignorance, you just overlook it, you don't see it.
I have seen your kind of publications, but these are all
useless for me.
Ask Thomas Richter, he should know better, but he also ignores it,
he doesn't look at it, he overlooks it or he doesn't understand it.
So why should I manifest the second and further fundamental
properties, if you are not even able to get the first?  That would
be useless, and I can't waste my time with useless attempts.

The mind is always conditioned.
Consciousness is BEYOND the mind, therefore unconditioned.
You don't know what this is.  Yes, you know the word, you have
heard that word consciousness, but you have no experience of it.
This is a distinct science, with its own methods, and you are
not aware of it.

ciao
human

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.compression

csiph-web