Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #4193

Re: Why does plotting with point labels make plot generation extremely slow?

Show all headers | View raw

On Tuesday, April 30, 2019 at 8:06:45 PM UTC+1, Ethan Merritt wrote:
> On Tue, 30 Apr 2019 01:17:21 -0700, ciro.santilli wrote:
> 
> [snip]
> 
> >> > My use case is described in detail at:
> >> > https://stats.stackexchange.com/questions/376361/how-to-find-the-
> sample-points-that-have-statistically-meaningful-large-outlier-r
> >> > 
> >> > Basically, I have 10 million multidimensional points, and I do a XY
> >> > scatter plot on the two main dimensions of interest.
> >> 
> >> OK.  Upstream development source (version 5.3) now reduces the label
> >> insertion overhead to zero by tracking both the head and tail of the
> >> list.
> >> Timings before and after:
> >> 
> >>      Labels             old (sec)       new (sec)
> >>     -----------------------------------------
> >>        10^4             1.5             1.0 
> >>        10^5             160             7.3
> >>        10^6             hopeless         66
> >> 
> >> One million hypertext labels took 66 seconds with the wxt terminal,
> >> slightly longer with qt (hard to compare because qt uses two processes
> >> running in parallel).  That's to draw the initial plot, however.
> >> With that many points the time to respond to mouse-over may be a
> >> substantial fraction of the original draw time.  To help with that,
> >> the upstream source now short-cuts hypertext generation to skip
> >> production of a hypertext tag if the text is empty.  So if you can
> >> manage to provide text for only the "potentially interesting" points
> >> you may find the mousing speed to be adequate to identify outliers.
> >> 
> >> The labels are still profligate in consuming memory, however, as there
> >> has never any particular pressure to reduce the size of the structure
> >> that describes a generic label.
> >> According to "top" my 10^6 point trial pinned 2.6 Gbyte of memory.
> >> I am dubious about the practicality of 10^7 points.
> >> 
> >> Let us know if you try, and it works!
> >> 
> >> 
> > Awesome, thanks!!
> > 
> > I tested out on master at a59021c8e97e20d00cb9e99d6392fe6bc5b56a07 from
> > GitHub, and it handled my 1m points just fine with wxt, opens in
> > reasonable time, and zoom / hover works. Is your computer particularly
> > old? That is a good thing for a dev actually, as it means that things
> > will run smoothly in any computer ;-)
> 
> Heh. I prioritize silence over power, and mostly work off a fanless
> nano-box sitting on my desk.
> 
> > Oh, and the compile time for this software is just amazing, so fast.
> > 
> > Me: Lenovo ThinkPad P51 laptop with CPU: Intel Core i7-7820HQ CPU (4
> > cores / 8 threads), RAM: 2x Samsung M471A2K43BB1-CRC (2x 16GiB), SSD:
> > Samsung MZVLB512HAJQ-000L7 (3,000 MB/s), NVIDIA Quadro M1200 4GB GDDR5
> > GPU.
> > 
> > 10m was however took about 16Gb memory and was too slow to be usable in
> > practice. So ultimately I'll have to stick to VisIt / Paraview for this
> > one (also they have that amazing point selection that stays on screen),
> > but I'm sure new applications have been opened by this patch :-)
> > 
> > It might be easy however to implement a functionality that dumps hovered
> > labels to stdout on wxt / debug console on HTML. Otherwise I have to
> > remember and copy 6 digit IDs around manually. This would make gnuplot
> > almost good enough for this use case.
> 
> Interesting idea. I am not sure how many of our supported platforms could
> provide this, but at least for wxt + linux + X11 it would be easy to have 
> a mouse click send the current hypertext to the X clipboard.
> 

I would recommend writing to stdout instead of clipboard if possible, as it would allow to quickly click a bunch of points, and then process all at once more easily by copying terminal / redirecting.

> > I noticed one thing though: when I tried 1m points conversion to HTML
> > the output seemed broken even without labels, but it works on Ubuntu
> > packaged gnuplot. I configured just with: ./configure --with-wx
> 
> I don't know what you mean "conversion to HTML".
> Are you referring plotting with "set term canvas" rather than "set term 
> wxt"?
> 

Yes, I was referring to set term canvas, which produces an HTML file.

>   Ethan

Back to comp.graphics.apps.gnuplot | Previous | Next — Previous in thread | Find similar

Thread

Why does plotting with point labels make plot generation extremely slow? ciro.santilli@gmail.com - 2019-04-28 02:36 -0700
  Re: Why does plotting with point labels make plot generation extremely slow? Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2019-04-28 19:11 +0200
  Re: Why does plotting with point labels make plot generation extremely slow? Ethan Merritt <eamerritt@gmail.com> - 2019-04-29 06:52 +0000
    Re: Why does plotting with point labels make plot generation extremely slow? ciro.santilli@gmail.com - 2019-04-29 01:01 -0700
      Re: Why does plotting with point labels make plot generation extremely slow? Ethan Merritt <eamerritt@gmail.com> - 2019-04-30 01:49 +0000
        Re: Why does plotting with point labels make plot generation extremely slow? ciro.santilli@gmail.com - 2019-04-30 01:17 -0700
          Re: Why does plotting with point labels make plot generation extremely slow? Ethan Merritt <sfeam@users.sf.net> - 2019-04-30 19:06 +0000
            Re: Why does plotting with point labels make plot generation extremely slow? ciro.santilli@gmail.com - 2019-04-30 14:26 -0700

csiph-web