Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #3528 > unrolled thread

plotting an adaptive mesh

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  plotting an adaptive mesh maierd83@gmail.com - 2017-01-17 01:54 -0800
    Re: plotting an adaptive mesh Karl Ratzsch <mail.kfr@gmx.net> - 2017-01-17 11:18 +0100
      Re: plotting an adaptive mesh maierd83@gmail.com - 2017-01-17 04:49 -0800
        Re: plotting an adaptive mesh Karl Ratzsch <mail.kfr@gmx.net> - 2017-01-17 14:11 +0100
          Re: plotting an adaptive mesh maierd83@gmail.com - 2017-01-17 06:05 -0800
            Re: plotting an adaptive mesh Karl Ratzsch <mail.kfr@gmx.net> - 2017-01-17 15:25 +0100
              Re: plotting an adaptive mesh maierd83@gmail.com - 2017-01-17 07:25 -0800
                svg with bitmap image graph? (was: Re: plotting an adaptive mesh) Karl Ratzsch <mail.kfr@gmx.net> - 2017-01-17 21:36 +0100
                  Re: svg with bitmap image graph? (was: Re: plotting an adaptive mesh) Ethan A Merritt <sfeam@users.sourceforge.net> - 2017-01-17 14:41 -0800
    Re: plotting an adaptive mesh maierd83@gmail.com - 2017-01-17 06:07 -0800

#3528 — plotting an adaptive mesh

Hello,

I am doing some calculations on the basis of an adaptive mesh. So, my data look like:

|----|----|---------|---------|
|  1 | 6  |         |         |
|----|----|    4    |    9    |
|  2 | 9  |         |         |
|----|----|----|----|---------|
|         |  1 | 8  |         |
|    5    |----|----|    2    |
|         |  8 | 5  |         |
|---------|----|----|----|----|
|         |         |  8 | 2  |
|    1    |    4    |----|----|
|         |         |  3 | 7  |
|---------|---------|----|----|

I want to plot these data with an 2D heat map.

To do so, I am expanding the fine pitched mesh to the hole data set 
|----|----|---------|---------|
|  1 | 6  |  4 | 4  |  9 | 9  |
|----|----|----|----|----|----|
|  2 | 9  |  4 | 4  |  9 | 9  |
|----|----|----|----|---------|
|  5 | 5  |  1 | 8  |  2 | 2  |
|----|----|----|----|----|----|
|  5 | 5  |  8 | 5  |  2 | 2  |
|---------|----|----|----|----|
|  1 | 1  |  4 | 4  |  8 | 2  |
|----|----|----|----|----|----|
|  1 | 1  |  4 | 4  |  3 | 7  |
|---------|---------|----|----|
and used "plot "data.dat" with image pixels".

This works, but it generates big data set (30MB) and also a big image (my used output is svg -> ~60MB) and is against the idea of an adaptive mesh.

In stackoverflow, I have found the idea to use set pm3d corners2color:
http://stackoverflow.com/questions/19294342/heatmap-with-gnuplot-on-a-non-uniform-grid

But this doesn't help in my case as the fine pitched pixels are not in one column but span over a diagonal.


A new idea would be to plot each rectangle by myself and filling it with a color according to its data value. I remember something like a statistics command in gnuplot, but I can't remember the details. Furthermore I am wondering if I have to print the rectangle command for each rectangle I want to draw (in principle I can do that because I produce the data by myself but printing some repetitive text ~1000 times will blow up again the data file) or if I can use a command that reads the values written in the data file.

Or this is all not a good idea and maybe there is an easy way to plot an adaptive mesh in gnuplot?

Thanks for any help.

[toc] | [next] | [standalone]

#3529

Am 17.01.2017 um 10:54 schrieb maierd83@gmail.com:
> Hello,
> 
> I am doing some calculations on the basis of an adaptive mesh. So, my data look like:
> 
> |----|----|---------|---------|
> |  1 | 6  |         |         |
> |----|----|    4    |    9    |
> |  2 | 9  |         |         |
> |----|----|----|----|---------|
> |         |  1 | 8  |         |
> |    5    |----|----|    2    |
> |         |  8 | 5  |         |
> |---------|----|----|----|----|
> |         |         |  8 | 2  |
> |    1    |    4    |----|----|
> |         |         |  3 | 7  |
> |---------|---------|----|----|
> 
> I want to plot these data with an 2D heat map.
> 
> To do so, I am expanding the fine pitched mesh to the hole data set 
> and used "plot "data.dat" with image pixels".
> 
> This works, but it generates big data set (30MB) and also a big image (my used output is svg -> ~60MB) and is against the idea of an adaptive mesh.
> 

For a bitmap, you should use a bitmap output (e.g. png). A vector
graphic that consists of single pixels is simply a large waste of disk
space.

As for the plotting, "set dgrid3d" will not lead to exactly the same
result as filling each larger mesh with the same value, but perhaps
it'll suit your needs. Try out the different kernels.

Otherwise, you might write a program that expands the dataset (i.e.
fills out the larger meshes) online, and pipe the output into gnuplot.

  plot '< expand dataset.dat' with image

That way you don't have those large, redundant files lying around.

  Karl

[toc] | [prev] | [next] | [standalone]

#3532

Thanks for your answer Karl.

My idea is to minimize the size of the data file and the image file. Furthermore, I want to reduce the calculation time of my program. At the moment the expanding of the adaptive mesh to a regular mesh takes longer than the calculation of the adaptive mesh data.

So, I want definitely get rid of the idea of expansion.

As adaptive meshing seems not to be supported by gnuplot directly, I want to use some gnuplot plotting utilities to make it by myself. My idea:
1) Finding with stats the min and max value of my data.
2) Make a color palette according to the min and max values.
3) Plot each data point as a rectangle with the fillcolor equal to the 
   color defined according to the color palette

The finer pitched data points (smaller rectangles) shall then be plotted on top of the bigger rectangles.

I hope I can do point 1) and 2) by myself, but I see problems on point 3):
how can I plot a rectangle with a fillcolor defined by the index of a colorbar? Something like (not working):
set obj rect cen 4,5 size 1,3 fc colorbar(5.3)


At the moment I am playing with this example:

reset
set terminal svg size 800,600
set output "./test.svg"

set style rect fc lt -1 fs noborder

#stats 'data.dat' using 3 prefix "A"
set xrange [2 : 8]
set yrange [2 : 8]

set pm3d
set palette defined (0 "black", 100 "blue", 200 "red", 300 "orange", 500 "white")

set obj rect cen 4,5 size 1,3 fc rgb "#5FD8F0"
set obj rect cen 5,4 size 1,1 fc rgb "#5FD800"
set obj rect cen 5,5 size 1,1 fc rgb "#9FD900"
set obj rect cen 5,6 size 1,1 fc rgb "#CFDA00"
set obj rect cen 6,5 size 1,3 fc rgb "#FFDB00"

plot x



I don't know if this is a reasonable way to go....

[toc] | [prev] | [next] | [standalone]

#3533

Am 17.01.2017 um 13:49 schrieb maierd83@gmail.com:

> My idea is to minimize the size of the data file and the image file. Furthermore, I want to reduce the calculation time of my program. At the moment the expanding of the adaptive mesh to a regular mesh takes longer than the calculation of the adaptive mesh data.

svg is the _wrong_ format to plot a bitmap. It takes up 30 or 40 bytes
_for every pixel_! Use jpeg or png. This will already speed up the task.

> So, I want definitely get rid of the idea of expansion.
> 
> 3) Plot each data point as a rectangle with the fillcolor equal to the 
>    color defined according to the color palette
> 
> The finer pitched data points (smaller rectangles) shall then be plotted on top of the bigger rectangles.

I don't think you use "set object" to print hundreds of thousands of
rectangles. I have no idea how to make gnuplot do that, except by
writing an additional program that translates your dataset into a
gnuplot script, and either way it will be horribly slow.

With huge datasets, the parsing of text data can be quite slow. Can you
output the expanded dataset in a binary (bitmap) format? How big (width
in x and y) is your dataset, anyway?


> set terminal svg size 800,600

see above.

[toc] | [prev] | [next] | [standalone]

#3534

> svg is the _wrong_ format to plot a bitmap. It takes up 30 or 40 bytes
> _for every pixel_! Use jpeg or png. This will already speed up the task.

Ok. I tried (my new idea, see below) with png and indeed the output is smaller. I am not sure if I understand this. I thought that svg saves a rectangle with position of two diagonal sides, and some parameters like fillstyle, fillcolor.... while pixel graphics save each pixel with position and color and if the rectangle consists of 1000 pixels it saves this 1000-times. Obviously png does not use such a raw format.

Anyway, the good news is that I might have found an easy way to do what I want:

1) I plot the data with the largest pitch.
2) I plot each sub-pixel on top of the existing image

The example looks like this:

set terminal png
set output "./test.png"

end = 2

plot for [i=0:2] "test.dat" index i with image pixels


with test.dat:
1 1 1
1 4 2
1 7 3

4 1 8
4 4 9
4 7 4

7 1 5 
7 4 2
7 7 3


3 3 8
3 4 9
3 5 7

4 3 6
4 4 9
4 5 6

5 3 9
5 4 8
5 5 5


6 6 1
6 7 2
6 8 3

7 6 2
7 7 3
7 8 2

8 6 1
8 7 1
8 8 2


Now, I (only?) have to find a way how to define the end-variable. I guess my c-program will print this value in the first line of the data file and gnuplot reads this value.

Thanks for any help. 
I'll check out how png works... ;-)

[toc] | [prev] | [next] | [standalone]

#3536

Am 17.01.2017 um 15:05 schrieb maierd83@gmail.com:
>> svg is the _wrong_ format to plot a bitmap. It takes up 30 or 40
>> bytes _for every pixel_! Use jpeg or png. This will already speed
>> up the task.
> 
> Ok. I tried (my new idea, see below) with png and indeed the output
> is smaller. I am not sure if I understand this. I thought that svg
> saves a rectangle with position of two diagonal sides, and some
> parameters like fillstyle, fillcolor.... while pixel graphics save
> each pixel with position and color and if the rectangle consists of
> 1000 pixels it saves this 1000-times. Obviously png does not use such
> a raw format.
> 

png is a compressed bitmap stream. It simply saves the dimensions of the
whole picture (height x width), and then the colour information of each
successive point, left to right, top to bottom. The position of each
pixel is totally redundant and not saved. In truecolor, before
compression, that's three bytes per pixel.

Look once into a svg file, this is a text file, meant for vector
graphics. Every pixel (=a four corner polygon) is a whole line of text:

<polygon fill = 'rgb(126,   4, 255)' points = '507.0,340.6 529.9,340.6
529.9,337.1 507.0,337.1 '/>



If your data comes from a self-written C program, why don't you just let
it write a binary data stream of the expanded picture, and pipe that
into gnuplot? Check "help binary matrix".

[toc] | [prev] | [next] | [standalone]

#3537

For the moment I am more than happy with the overplotting method.
It works very fast and my data sizes are reduced from:
        old             new
data:  ~18MB        ->  41  kB
image: ~33MB (svg)  ->  209 kB (svg) and 7kB (png)

Unfortunately the png has the problem with the blurry lables for the axes...

> If your data comes from a self-written C program, why don't you just let
> it write a binary data stream of the expanded picture, and pipe that
> into gnuplot? Check "help binary matrix".

I'll check that option later. Sounds like I could use this for many things...

Thanks again,
Daniel

[toc] | [prev] | [next] | [standalone]

#3538 — svg with bitmap image graph? (was: Re: plotting an adaptive mesh)

Am 17.01.2017 um 16:25 schrieb maierd83@gmail.com:
> For the moment I am more than happy with the overplotting method.
> It works very fast and my data sizes are reduced from:
>         old             new
> data:  ~18MB        ->  41  kB
> image: ~33MB (svg)  ->  209 kB (svg) and 7kB (png)
> 
> Unfortunately the png has the problem with the blurry lables for the axes...
> 

Try pngcairo, it does antialiasing, which the ligd-based png terminal
does not (I think). Should look a bit better.

I don't know how to combine an svg graph with png data. There is some
mention about it in "help svg" (next-to-last paragraph), but I have no
idea how that would work.

[toc] | [prev] | [next] | [standalone]

#3539 — Re: svg with bitmap image graph? (was: Re: plotting an adaptive mesh)

Karl Ratzsch wrote:

> Am 17.01.2017 um 16:25 schrieb maierd83@gmail.com:
>> For the moment I am more than happy with the overplotting method.
>> It works very fast and my data sizes are reduced from:
>>         old             new
>> data:  ~18MB        ->  41  kB
>> image: ~33MB (svg)  ->  209 kB (svg) and 7kB (png)
>> 
>> Unfortunately the png has the problem with the blurry lables for the
>> axes...
>> 
> 
> Try pngcairo, it does antialiasing, which the ligd-based png terminal
> does not (I think). Should look a bit better.
> 
> I don't know how to combine an svg graph with png data. There is some
> mention about it in "help svg" (next-to-last paragraph), but I have no
> idea how that would work.


You have this kind of backwards.

By default the svg terminal represents an image by creating a png
bitmap. The png bitmap is saved into a separate *.png file and
referenced by name in the *.svg file.   

This mechanism gives you the advantage of compact image representation,
at the cost of extra bookkeeping because you have to track both the 
*.svg and *.png files together and make sure the reference-by-name in
the *.svg continues to work if you move the files.

Current CVS (5.1) allows you to instead embed the png image bitmap
in the *.svg itself so there is no separate file.  The keyword for
this is "standalone".  This should probably become the default but
as of now it isn't.

However...
   the original question here asked about plotting "with image pixels".
The extra keyword "pixels" tells gnuplot _not_ to represent the image in
a compact bitmap array;  instead it does what you described, drawing a
filled rectangle for each pixel.   Sometimes that has advantages, other
times it does not.  The big disadvantage is the extra space required.
The potential benefit is that when svg renders a bitmap it does 
color-smoothing, which may or may not be acceptable.  The "pixels" mode 
guarantees that each pixel will have the expected color.
See the online demo 

   http://gnuplot.sourceforge.net/demo_svg_5.0/heatmaps.html

[toc] | [prev] | [next] | [standalone]

#3535

edit: the plot command should be:
plot for [i=0:end] "test.dat" index i with image pixels

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web