Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #1888 > unrolled thread

Canvas precision

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 07:13 -0700
    Re: Canvas precision Karl <mail.kfr@gmx.net> - 2013-06-14 17:08 +0200
      Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 12:38 -0700
    Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 12:48 -0700
      Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-15 00:28 -0700

#1888 — Canvas precision

Before digging deep into gnuplot internals, I would like to know if it's normal that I get a resolution of only 3 digits after the comma when using canvas 2D plots?

I have this data file:

25785.0000998 -68.8343006527
25785.0001028012 -80.5616032527
25785.0001058025 -87.8913075527
25785.0001088037 -87.6995350527
25785.000111805 -87.0571553527
25785.0001148063 -86.7244984527
25785.0001178075 -84.5539128527
...

it has ~18000 entries. For those who are interested, the firs columns are frequencies (MHz), the seconds ones are gains (dB).

Now when zooming deep into the canvas plot the points are drawn only to precision of 0.001 - which means that several points are drawn on the same x-position.
This precision seems to change depending on the number of points I use.

I am using gnuplot 4.6.

Anybody an idea?

thx

[toc] | [next] | [standalone]

#1889

Am 14.06.2013 16:13, schrieb tombert:
> Before digging deep into gnuplot internals, I would like to know if it's normal that I get a resolution of only 3 digits after the comma when using canvas 2D plots?
> 
> I have this data file:
> 
> 25785.0000998 -68.8343006527
> 25785.0001028012 -80.5616032527
> 25785.0001058025 -87.8913075527
> 25785.0001088037 -87.6995350527
> 25785.000111805 -87.0571553527
> 25785.0001148063 -86.7244984527
> 25785.0001178075 -84.5539128527
> ...
> 
> it has ~18000 entries. For those who are interested, the firs columns are frequencies (MHz), the seconds ones are gains (dB).
> 
> Now when zooming deep into the canvas plot the points are drawn only to precision of 0.001 - which means that several points are drawn on the same x-position.
> This precision seems to change depending on the number of points I use.
> 
> I am using gnuplot 4.6.
> 
> Anybody an idea?
> 
> thx
> 
Have you tried substracting 25785 from the frequencies? Your numbers
differ in the 9-10th position, i´d say that is 32 bit precision.

- Karl

[toc] | [prev] | [next] | [standalone]

#1890

I think I found the problem. 

With the following data:
25785.0001 -68.997037
25785.00010027 -69.1994067
25785.00010054 -69.5961962
25785.00010081 -70.1889926
25785.00010108 -70.9806719
25785.00010135 -71.9761248
... till ...
25785.99586 -62.0013247
25785.99668 -61.3820468
25785.9975 -60.851285
25785.99832 -60.4307604
25785.99914 -60.136098
25785.99996 -59.9775515
... 6000 points


gnuplot creates 6000 path points:
M(941,2093);
L(941,2107);
L(941,2133);
L(941,2172);
L(941,2225);
L(941,2292);
... till ...
L(10459,1627);
L(10467,1586);
L(10475,1550);
L(10483,1522);
L(10491,1503);
L(10499,1492);

One sees that the first data points all got translated into 941. This seems to be dependent on the total number of points.

When I delete the last 5000 points, then gnuplot translates into:
M(940,627);
L(944,651);
L(947,697);
L(951,766);
L(955,858);
L(958,974);
...

So I would like to know how to increase the value range for these translations?

thx

[toc] | [prev] | [next] | [standalone]

#1891

And probably here is the solution:

from canvas.trm:

#define CANVAS_OVERSAMPLE	10.
#define CANVAS_XMAX		(600 * CANVAS_OVERSAMPLE) 

If I increase the sampling would this solve my problem? Would it be somehow nice if this value could be changed via gnuplot command?

thx

[toc] | [prev] | [next] | [standalone]

#1892

Solved here:
https://sourceforge.net/p/gnuplot/feature-requests/366/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web