Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #2160 > unrolled thread

Bug on palette

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-18 03:32 -0700
    Re: Bug on palette Ethan A Merritt <merritt@u.washington.edu> - 2013-10-18 17:18 -0700
      Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-19 01:28 -0700
        Re: Bug on palette Olaf Schultz <o.schultz@enhydralutris.de> - 2013-10-19 13:28 +0200
        Re: Bug on palette Olaf Schultz <o.schultz@enhydralutris.de> - 2013-10-19 13:37 +0200
        Re: Bug on palette Ethan A Merritt <EAMerritt@gmail.com> - 2013-10-19 10:34 -0700
          Re: Bug on palette Ethan A Merritt <EAMerritt@gmail.com> - 2013-10-19 11:28 -0700
            Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-20 05:27 -0700
            Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-20 08:59 -0700
            Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-20 09:37 -0700
              Re: Bug on palette sfeam <sfeam@users.sourceforge.net> - 2013-10-20 12:05 -0700
                Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-21 00:28 -0700
                  Re: Bug on palette sfeam <sfeam@users.sourceforge.net> - 2013-10-22 14:20 -0700
                    Re: Bug on palette Vasilis <vvlachoudis@gmail.com> - 2013-10-23 00:42 -0700

#2160 — Bug on palette

Dear all,

this was a bug I've reported some time ago on version 4.3 but it  reappeared on version 4.6.x

set pm3d map explicit corners2color c1
set palette defined ( 0 0 0 0, 0.25 0 0 1, 0.50 0 1 0, 0.75 1 0 0, 1 1 1 1 )
set palette maxcolors 30
set logscale cb
splot x*y

At the "join" locations of the palette some strange brown color appears
See the result on the plot
http://pceet075.cern.ch/bnv/gnuplot_palette_bug.png

Thanks in advance
Vasilis

[toc] | [next] | [standalone]

#2164

Vasilis wrote:

> Dear all,
> 
> this was a bug I've reported some time ago on version 4.3 but it 
> reappeared on version 4.6.x
> 
> set pm3d map explicit corners2color c1
> set palette defined ( 0 0 0 0, 0.25 0 0 1, 0.50 0 1 0, 0.75 1 0 0, 1 1 1 1
> ) set palette maxcolors 30
> set logscale cb
> splot x*y
> 
> At the "join" locations of the palette some strange brown color appears
> See the result on the plot
> http://pceet075.cern.ch/bnv/gnuplot_palette_bug.png

There is an inherent contradiction between requesting a gradient-based
palette and requesting a fixed number of colors.
I suppose it is possible to add more special-case tests for when
the gradient boundaries don't line up with the quantized palette 
boundaries. There will always be combinations where the result is
not pleasing.

I am curious why would you ever want to set maxcolors to 30?
What use case requires setting maxcolors at all?
In the days of 6-color pen plotters it made some sense, but what
real-world 21st century device is limited to exactly 30 colors?

And for those cases where you really do want a small fixed number
of colors N, I think palette coloring is no longer the best way to
create a plot.  I would recommend instead to define N separate linetype
colors and plot with "linecolor variable".  Then you have perfect 
control over the color that is used.

	Ethan

> 
> Thanks in advance
> Vasilis

[toc] | [prev] | [next] | [standalone]

#2165

Dear Ethan,

thank you for your reply, but i think it is a problem that could be fixed. In any case it was working back in 4.2.

Setting a low number of maxcolor is extremely important when we make colorplots with dose (output of monte carlo physics programs). Setting a low number of maxcolors allows you to perfectly distinguish visually various regions of interest in your plot. If you use a smooth palette, you get a pleasing result for the eye but it is very hard to read it (e.g get the value-range of a pixel). We wonder is this color or the next one? Almost all scientific programs that I know (monte carlo, molecular dynamics, finite elements etc) they use the same approach they define a limited number of colors (16-30). Further more since we plot always in 'log cb' we try to align the colors with the decades e.g. 4-5 colors per decade.

Maybe as you proposed using "N separate linetype colors and plot with linecolor variable"  could be a solution for us.
Could you please give me an example.

BTW I've tried to make also a contour plot of our data with

reset
set contour
set dgrid3d 40,40
unset surface
splot 'enedep.dat' w l

and I get the error
>>>>      Gridding of the color column is not implemented
I don't have any "color" column

If I give
splot 'enedep.dat' u 1:2:3 w l
it makes the plot

our data are blocks of
x1 y1 z err
x2 y1 z err
...

x1 y2 z err
x1 y2 z err
...

Vasilis

[toc] | [prev] | [next] | [standalone]

#2166

Vasilis wrote:
> Dear Ethan,
> 
> thank you for your reply, but i think it is a problem that could be fixed. In any case it was working back in 4.2.
> 
> Setting a low number of maxcolor is extremely important when we make colorplots with dose (output of monte carlo physics programs). Setting a low number of maxcolors allows you to perfectly distinguish visually various regions of interest in your plot. If you use a smooth palette, you get a pleasing result for the eye but it is very hard to read it (e.g get the value-range of a pixel). We wonder is this color or the next one? Almost all scientific programs that I know (monte carlo, molecular dynamics, finite elements etc) they use the same approach they define a limited number of colors (16-30). Further more since we plot always in 'log cb' we try to align the colors with the decades e.g. 4-5 colors per decade.
> 
> Maybe as you proposed using "N separate linetype colors and plot with linecolor variable"  could be a solution for us.
> Could you please give me an example.
> 
> BTW I've tried to make also a contour plot of our data with
> 
> reset
> set contour
> set dgrid3d 40,40
> unset surface
> splot 'enedep.dat' w l
> 
> and I get the error
>>>>>      Gridding of the color column is not implemented
> I don't have any "color" column
> 
> If I give
> splot 'enedep.dat' u 1:2:3 w l
> it makes the plot
> 
> our data are blocks of
> x1 y1 z err
> x2 y1 z err
> ....
> 
> x1 y2 z err
> x1 y2 z err
> ....
> 
> Vasilis

[toc] | [prev] | [next] | [standalone]

#2167

Sorry for first empty, wrong key for line-wrapping.

Vasilis wrote:
> Dear Ethan,
> 
> thank you for your reply, but i think it is a problem that could be
> fixed. In any case it was working back in 4.2.
> 
> Setting a low number of maxcolor is extremely important when we make
> colorplots with dose (output of monte carlo physics programs).
> Setting a low number of maxcolors allows you to perfectly distinguish
> visually various regions of interest in your plot. If you use a
> smooth palette, you get a pleasing result for the eye but it is very
> hard to read it (e.g get the value-range of a pixel).

Exact my point. I also like the maxcolor (10 or so). Sometimes to be
adjusted with the cbrange-tics... so that the cbrange-tics are at the
color-switch.

 We wonder is
> this color or the next one? Almost all scientific programs that I
> know (monte carlo, molecular dynamics, finite elements etc)

[x] MSC Patran/Nastran has 12/16 colors by default. If you use a good
interpretable hot-cold-mapping sometimes it becomes difficult to read
the right color in the printout.

Olaf

[toc] | [prev] | [next] | [standalone]

#2168

Vasilis wrote:

> Dear Ethan,
> 
> thank you for your reply, but i think it is a problem that could be
> fixed. In any case it was working back in 4.2.

Your particular case was working in 4.2, but there are many other
cases that were not.  In particular the code in 4.2 assumed that the
width of each color interval was identical, and produced poor
results when they were not.

I should clarify that the problem you encountered arises specifically
from mixing the "maxcolors" option with a palette containing multiple
interpolation ranges.  The data z value is mapped onto a range [0:1]
and then truncated to the nearest interval (1./maxcolors).  But if the
palette is defined by several distinct interpolation ranges, as in 
your case, this truncation can cause the value to shift from one
interpolation range to the next.  If this happens, the color is 
chosen from the "wrong" sub-palette.  This won't happen if the 
palette is defined as a single continuous interval.  It also should
not happen (or happen only rarely due to precision errors)
if maxcolors is evenly divisible by the number of palette intervals.
In your case there are 4 equally spaced palette intervals, so setting
maxcolors to 28 or 30 or any multiple of 4 would avoid the problem.

As I said, this problem case is detectable. But I am not convinced
there is a unique resolution that is correct in the general case.

> Setting a low number of maxcolor is extremely important when we make
> colorplots with dose (output of monte carlo physics programs). Setting
> a low number of maxcolors allows you to perfectly distinguish visually
> various regions of interest in your plot. If you use a smooth palette,
> you get a pleasing result for the eye but it is very hard to read it
> (e.g get the value-range of a pixel). We wonder is this color or the
> next one? Almost all scientific programs that I know (monte carlo,
> molecular dynamics, finite elements etc) they use the same approach
> they define a limited number of colors (16-30). Further more since we
> plot always in 'log cb' we try to align the colors with the decades
> e.g. 4-5 colors per decade.
> 
> Maybe as you proposed using "N separate linetype colors and plot with
> linecolor variable"  could be a solution for us. Could you please give
> me an example.

Sounds like an interesting assignment :-)  I'll look into it.

By the way, the "test palette" command is useful for checking your
palette definition without doing an actual plot.

	Ethan

[toc] | [prev] | [next] | [standalone]

#2169

Ethan A Merritt wrote:

> In your case there are 4 equally spaced palette intervals, so setting
> maxcolors to 28 or 30 or any multiple of 4 would avoid the problem.

Bleah.  Make that "28 or 32 or any multiple of 4".

	Ethan

[toc] | [prev] | [next] | [standalone]

#2171

On Saturday, October 19, 2013 8:28:10 PM UTC+2, Ethan A Merritt wrote:
> Ethan A Merritt wrote:
> 
> 
> 
> > In your case there are 4 equally spaced palette intervals, so setting
> 
> > maxcolors to 28 or 30 or any multiple of 4 would avoid the problem.
> 
> 
> 
> Bleah.  Make that "28 or 32 or any multiple of 4".
> 
> 
> 
> 	Ethan


Hi Ethan,

Combining maxcolors with a multiple gradient palette is a very nice feature, and should work properly. I am pretty sure that it can be corrected.

The problem is visible even using the defined palette from gnuplot try
set palette defined
set palette maxcolors 16
test palette

Actually what ever combination I chose for maxcolors 26,28,30,40,50 it always shows some colors wrongly.

Vasilis

[toc] | [prev] | [next] | [standalone]

#2172

Actually it is a bug, in the function getcolor.c:double 
quantize_gray( double gray )

if you comment out the block of code

if (sm_palette.colorMode == SMPAL_COLOR_MODE_GRADIENT) {
..
}

which for some reason changes the gray value to the middle of the segment everything works Ok.

Actually I changed the 'if' to the following:

    if (sm_palette.colorMode == SMPAL_COLOR_MODE_GRADIENT &&
        !sm_palette.use_maxcolors) {
..
}

and all test cases I've run worked fine.

I hope you can do the necessary corrections also to the 4.6.x versions

Best Regards
Vasilis

[toc] | [prev] | [next] | [standalone]

#2173

Firefox decided to die before my last post, so here it comes again.

After looking this function I have the feeling that it makes no-sense to alter inside the if() the quantized gray value, to the mid-value of the matching interpolation range. Why? The two things are not related to my mind. 

However I am sure you have a good reason for that, so in this case a more correct way would be:
		if (degray < sm_palette.gradient[j].pos)
		    degray = sm_palette.gradient[j].pos;
		else if (degray > sm_palette.gradient[j+1].pos)
		    degray = sm_palette.gradient[j+1].pos;

Last thing, doing the check linear for every plotted pixel is a real CPU-waste. Why not doing it with bisect as in the interpolate_color_from_gray()

Cheers
Vasilis

[toc] | [prev] | [next] | [standalone]

#2174

Vasilis wrote:

> Firefox decided to die before my last post, so here it comes again.
> 
> After looking this function I have the feeling that it makes no-sense
> to alter inside the if() the quantized gray value, to the mid-value of
> the matching interpolation range. Why? The two things are not related
> to my mind.

I agree that the current code works badly for your case and should
be fixed.  But I don't know of a perfect fix.  Everything I have come
up with fails for some class of reasonable palette definitions.
I welcome your suggestions.

As to why it switches to the midpoint of an adjacent range, that was
added to handle the case where 
1) The palette contains multiple segments (each treated as a gradient)
2) One or more segments are small compared to the others.

An example is a log scale palette with intervals
   0 1 2 4 8 16 32 64 ...
You might think maxcolors=8 would be reasonable, but without the
extra check all ranges below 4 would be treated as equal to 0.

> However I am sure you have a good reason for that, so in this case a
> more correct way would be: if (degray < sm_palette.gradient[j].pos)
> degray = sm_palette.gradient[j].pos;
> else if (degray > sm_palette.gradient[j+1].pos)
> degray = sm_palette.gradient[j+1].pos;

That code still has the effect of skipping small color ranges entirely.
 
> Last thing, doing the check linear for every plotted pixel is a real
> CPU-waste. Why not doing it with bisect as in the
> interpolate_color_from_gray()

It is linear in the number of palette segments, which is usually small.

... 

After thinking about this some more, I am trying out a variant that
works as follows:

1) If the range of the smallest palette segment is still larger than
   (total_range/maxcolors) then we don't have to worry about missing
   that segment due to truncation.  This returns to the version 4.2
   behavior for palettes with equal-width gradient segments and 
   maxcolors >= number of segments.  This condition need only be checked
   once at the time the palette is defined.

2) If condition (1) is not satisfied, then we check to see if the
   current gray value lies in one of the too-small gradient intervals.
   If so, set it to the segment midpoint; 
   if not, let the truncated value stand.

This certainly handles your current problem case "perfectly", because
it is covered by condition (1).  It also does a good but not perfect
job with the binary log scale example abouve. But it is not hard to come
up with examples where it behaves badly. In particular it produces odd
results if the truncated gray value happens to lie inside one of the
small palette segments, but the true gray value does not.
Here is an extreme case example:

   set palette defined (0 "white", 0.49 "blue", \
                        0.49 "black", 0.51 "black", \
                        0.51 "white", 1.0 "red")
   set palette maxcolors 9
   test palette

The intent is clearly to have several low-range segments distinguished
by blue saturation and several high-range segments distinguished by
red saturation, with black indicating some mid-point condition.
And that's exactly what you get for even-valued of maxcolors.
But for odd-valued maxcolor the supposedly narrow range of black
values becomes disproportionately wide.

This example case is also handled poorly by the current code, so
the revised algorithm is clearly an improvement.  But I'd like to do
even better if possible.

	Ethan

> Cheers
> Vasilis

[toc] | [prev] | [next] | [standalone]

#2176

I agree with you that in your example there is a contradiction when
someone sets palette segments smaller than the interval. Its like
asking for a fixed binning histogram (1/maxcolor) and a variable-
binning histogram at the same time!

My approach would be the same like yours #1. Issue a warning
message since the palette is ill-defined. There is no logic setting
a fix interval (1Jmaxcolors) and asking for color-ranges smaller
than this interval.

Another approach will be to make some-kind of color-antialiasing,
color-blending. When someone sets the maxcolors, at init when you
create the palette, set the average ncolor> for each interval
(1/maxcolors), instead of the color at a specific gray-value.

for each i interval range 0..maxcolors
<color[i]> = integral(for x in i/maxcolors to (i+1)/maxcolors of
color_components_from_gray(x)) * maxcolors

One more aesthetic correction is that the quantizengray()
calculates the degray to the lower-limit of the interval

double degray = floor(gray * sm palette.use maxcolors)
          / (smnpalette.usenmaxcolors-1);

instead of the middle value

double degray = ((double)(int)(gray.sm_palette.use_maxcolors)+0.5)
          / sm_palette.use_maxcolors;

Honestly the effect is minor and hardly visible.

I would appreciate if you could issue a quick fix on 4.6 quite
soon. In my program (www.fluka.org/flair) with 3000+ users, the
most important information are the color-plots output of the
particle simulation code and it is annoying to ask people to
downgrade to 4.2 to get them right.

Cheers

[toc] | [prev] | [next] | [standalone]

#2178

Vasilis wrote:
> 
> I would appreciate if you could issue a quick fix on 4.6 quite
> soon. In my program (www.fluka.org/flair) with 3000+ users, the
> most important information are the color-plots output of the
> particle simulation code and it is annoying to ask people to
> downgrade to 4.2 to get them right.

Modified code is now in the source repositories for both 4.6 and 4.7.  

Of course, that won't end up in distributed binary packages for quite
a while, so probably you should look at work-arounds that your users
can adopt immediately. 

One work-around already mentioned is to use "set palette maxcolors 4*N".
That avoids the problem for the specific palette you gave as an example.

Another option is to specify the palette itself in terms of continuous
functions.  That is, "set palette func ..." 
rather than "set palette defined ..."

The palette in your example can be described using commands:

  palred(gray)   =  4*gray-2
  palgreen(gray) = (gray<.75) ? 1-4*abs(0.5-gray) : 4*gray-3
  palblue(gray)  = (gray<.75) ? 1-4*abs(.25-gray) : 4*gray-3
  set palette func palred(gray), palgreen(gray), palblue(gray)
  set palette maxcolors 30

That's more than you'd want to type interactively, but a program
that produces gnuplot output could easily emit these commands 
instead of the "set palette defined ..." variant.

	Ethan

[toc] | [prev] | [next] | [standalone]

#2179

Thank you Ethan,

the palette(s) that I am using have more color/regions than what I've send. Plus I have multiple palettes as a user choice. The one I've sent was just an example to find the problem. Probably as you say I would have to make the palette my self, probably setting one-color-range per 1/maxcolor interval.

Vasilis

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web