Groups > comp.graphics.apps.gnuplot > #1888

Canvas precision

Newsgroups	comp.graphics.apps.gnuplot
Date	2013-06-14 07:13 -0700
Message-ID	<cfd808bd-3e83-462c-99bb-b738568e96c0@googlegroups.com> (permalink)
Subject	Canvas precision
From	tombert <tombert.groups@live.at>

Show all headers | View raw

Before digging deep into gnuplot internals, I would like to know if it's normal that I get a resolution of only 3 digits after the comma when using canvas 2D plots?

I have this data file:

25785.0000998 -68.8343006527
25785.0001028012 -80.5616032527
25785.0001058025 -87.8913075527
25785.0001088037 -87.6995350527
25785.000111805 -87.0571553527
25785.0001148063 -86.7244984527
25785.0001178075 -84.5539128527
...

it has ~18000 entries. For those who are interested, the firs columns are frequencies (MHz), the seconds ones are gains (dB).

Now when zooming deep into the canvas plot the points are drawn only to precision of 0.001 - which means that several points are drawn on the same x-position.
This precision seems to change depending on the number of points I use.

I am using gnuplot 4.6.

Anybody an idea?

thx

Back to comp.graphics.apps.gnuplot | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 07:13 -0700
  Re: Canvas precision Karl <mail.kfr@gmx.net> - 2013-06-14 17:08 +0200
    Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 12:38 -0700
  Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-14 12:48 -0700
    Re: Canvas precision tombert <tombert.groups@live.at> - 2013-06-15 00:28 -0700

csiph-web