Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #31068 > unrolled thread

matrix visualization

Back to article view | Back to comp.lang.java.programmer

Contents

  matrix visualization Philipp Kraus <philipp.kraus@flashpixx.de> - 2015-02-07 12:44 +0100
    Re: matrix visualization Marcel Mueller <news.5.maazl@spamgourmet.org> - 2015-02-07 12:57 +0100
    Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-07 14:49 -0500
      Re: matrix visualization Philipp Kraus <philipp.kraus@flashpixx.de> - 2015-02-07 21:16 +0100
        Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-07 17:33 -0500
          Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-07 21:17 -0500
            Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-09 03:43 -0500
              Re: matrix visualization Martin Gregorie <martin@address-in-sig.invalid> - 2015-02-09 13:10 +0000
                Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-09 08:31 -0500
                  Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-09 08:44 -0500
                    Re: matrix visualization Jeff Higgins <jeff@invalid.invalid> - 2015-02-09 10:01 -0500
                      Re: matrix visualization Philipp Kraus <philipp.kraus@flashpixx.de> - 2015-02-11 08:50 +0100
                Re: matrix visualization Philipp Kraus <philipp.kraus@flashpixx.de> - 2015-02-11 08:48 +0100
              Re: matrix visualization Philipp Kraus <philipp.kraus@flashpixx.de> - 2015-02-11 08:47 +0100

#31068 — matrix visualization

Hello,

I would like to create a large table view of a matrix. I use a 
OD-Matrix (origin destination matrix), which is a quadratic matrix wil 
numeric values.
My current system can create a matrix with 40.000 columns (and 40.000 
rows), but a lot of elements are zero. But I would like to create larger
datasets.

My first idea is to use a JTable structure to visualize the data, but 
I'm a little bit unsure if it works well. On my simulation I can create 
at each simulation
step this matrix, so there can be a lot of changes during runtime.

Did you have got any good ideas for realization?

Thx

[toc] | [next] | [standalone]

#31069

On 07.02.15 12.44, Philipp Kraus wrote:
> I would like to create a large table view of a matrix. I use a OD-Matrix
> (origin destination matrix), which is a quadratic matrix wil numeric
> values.
> My current system can create a matrix with 40.000 columns (and 40.000
> rows), but a lot of elements are zero. But I would like to create larger
> datasets.
>
> My first idea is to use a JTable structure to visualize the data, but
> I'm a little bit unsure if it works well.

No UI on will survive to display 1.6E9 matrix elements directly. No 
human will be able to find something within this set. So the basic 
question is what you intend to visualize.

You mentioned a sparse matrix. Maybe it is helpful to create an image 
with pixels that represent the values by their color. But this is still 
too much. Maybe you can combine 10x10 or more matrix elements to one 
pixel and provide a zoom function for the details. It is quite easy to 
find the nonzero elements this way.


Marcel

[toc] | [prev] | [next] | [standalone]

#31074

On 02/07/2015 06:44 AM, Philipp Kraus wrote:
> Hello,
>
> I would like to create a large table view of a matrix. I use a OD-Matrix
> (origin destination matrix), which is a quadratic matrix wil numeric
> values.
> My current system can create a matrix with 40.000 columns (and 40.000
> rows), but a lot of elements are zero. But I would like to create larger
> datasets.
>
> My first idea is to use a JTable structure to visualize the data, but
> I'm a little bit unsure if it works well. On my simulation I can create
> at each simulation
> step this matrix, so there can be a lot of changes during runtime.
>
> Did you have got any good ideas for realization?
>
Since you haven't mentioned your search results:
<http://lmgtfy.com/?q=java+large+dataset+visualization>

[toc] | [prev] | [next] | [standalone]

#31076

[Multipart message — attachments visible in raw view] — view raw

On 2015-02-07 19:49:41 +0000, Jeff Higgins said:

> On 02/07/2015 06:44 AM, Philipp Kraus wrote:
>> Hello,
>> 
>> I would like to create a large table view of a matrix. I use a OD-Matrix
>> (origin destination matrix), which is a quadratic matrix wil numeric
>> values.
>> My current system can create a matrix with 40.000 columns (and 40.000
>> rows), but a lot of elements are zero. But I would like to create larger
>> datasets.
>> 
>> My first idea is to use a JTable structure to visualize the data, but
>> I'm a little bit unsure if it works well. On my simulation I can create
>> at each simulation
>> step this matrix, so there can be a lot of changes during runtime.
>> 
>> Did you have got any good ideas for realization?
>> 
> Since you haven't mentioned your search results:
> <http://lmgtfy.com/?q=java+large+dataset+visualization>

Sorry, I know system like multidimensional scaling or 
self-organizing-maps (which comes from big data analysis),
but at the moment I diskuss this 
http://en.wikipedia.org/wiki/Trip_distribution structure, so a plain 
numeric matrix is used.

I'm searching for some ideas, which can be used within Java to create a 
good useability structure for this large datasets.

So I hope for some ideas :-)

Phil

[toc] | [prev] | [next] | [standalone]

#31078

On 02/07/2015 03:16 PM, Philipp Kraus wrote:

> Did you have got any good ideas for realization?
>
>
> Since you haven't mentioned your search results:
>
> <http://lmgtfy.com/?q=java+large+dataset+visualization>
>
>
> Sorry, I know system like multidimensional scaling or
> self-organizing-maps (which comes from big data analysis),
>
> but at the moment I diskuss this
> http://en.wikipedia.org/wiki/Trip_distribution structure, so a plain
> numeric matrix is used.
>
>
> I'm searching for some ideas, which can be used within Java to create a
> good useability structure for this large datasets.
>
>
> So I hope for some ideas :-)
>
Marcel Mueller mentions zoomable pixels - sounds good for clumps.
Another comes from the search mentioned above - closer to your dimensions:
<http://flowingdata.com/2009/04/21/visual-representation-of-tabular-information-how-to-fix-the-uncommunicative-table/>
Good Luck.

[toc] | [prev] | [next] | [standalone]

#31079

On 02/07/2015 05:33 PM, Jeff Higgins wrote:

Well, Circos.

I was going to point to:
<http://www.cse.ust.hk/~huamin/zeng_eurovis13.pdf>
but then ran across this:
<http://en.wikibooks.org/wiki/Fundamentals_of_Transportation/Destination_Choice>
and conclude you are not looking to visualize results
but for a massive spreadsheet.

[toc] | [prev] | [next] | [standalone]

#31093

On 02/07/2015 09:17 PM, Jeff Higgins wrote:
> and conclude you are not looking to visualize results
> but for a massive spreadsheet.

So you'll likely want to tile your matrix.
SWT table model has "virtual" tables built into the api.
<https://eclipse.org/articles/Article-SWT-Virtual/Virtual-in-SWT.html>
But I don't know how (well) it works, I haven't used tried to use.

When I open LibreOffice Calc in a maximised window I see
fifteen columns and forty-five rows. Perhaps that is a good
tile size. So keep a nine by nine block of tiles from your
matrix as your "virtual" table viewport and move the viewport
about as your user scrolls.
?

[toc] | [prev] | [next] | [standalone]

#31096

On Mon, 09 Feb 2015 03:43:38 -0500, Jeff Higgins wrote:

> On 02/07/2015 09:17 PM, Jeff Higgins wrote:
>> and conclude you are not looking to visualize results but for a massive
>> spreadsheet.
> 
> So you'll likely want to tile your matrix.
> SWT table model has "virtual" tables built into the api.
> <https://eclipse.org/articles/Article-SWT-Virtual/Virtual-in-SWT.html>
> But I don't know how (well) it works, I haven't used tried to use.
> 
> When I open LibreOffice Calc in a maximised window I see fifteen columns
> and forty-five rows. Perhaps that is a good tile size. So keep a nine by
> nine block of tiles from your matrix as your "virtual" table viewport
> and move the viewport about as your user scrolls.
> ?

This may not be exactly what the OP wants, but should at least be fairly 
easy to implement. In Swing term, here's my suggestion Use the zooming 
image visualiser as the outer layer, and (probably minimising the 
minimising the number of zoom layers by using a large image in a 
JScrollPane with co-ordinates etc used as the frame's title. When zoomed 
in close enough, use a to JTable to show the actual content, populating 
the JTable model with the relevant patch copied from the master data 
matrix. The first column and the header row would be used to show matrix 
co-ordinates.

I've recently been using a JScrollPane containing a set of JPanes managed 
by the CardLayout layout manager to navigate a set of related panels, 
some of which are too big to fit my screen. This worked well once I'd got 
my head round how you're meant to use CardLayouts (which is not well 
explained in the Oracle's tutorials or the class-level documentation) but 
using the CardLayout manager may offer a convenient way of stepping up 
and down the zoom levels.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#31097

On 02/09/2015 08:10 AM, Martin Gregorie wrote:
> On Mon, 09 Feb 2015 03:43:38 -0500, Jeff Higgins wrote:
>
>> On 02/07/2015 09:17 PM, Jeff Higgins wrote:
>>> and conclude you are not looking to visualize results but for a massive
>>> spreadsheet.
>>
>> So you'll likely want to tile your matrix.
>> SWT table model has "virtual" tables built into the api.
>> <https://eclipse.org/articles/Article-SWT-Virtual/Virtual-in-SWT.html>
>> But I don't know how (well) it works, I haven't used tried to use.
>>
>> When I open LibreOffice Calc in a maximised window I see fifteen columns
>> and forty-five rows. Perhaps that is a good tile size. So keep a nine by
>> nine block of tiles from your matrix as your "virtual" table viewport
>> and move the viewport about as your user scrolls.
>> ?
>
> This may not be exactly what the OP wants, but should at least be fairly
> easy to implement. In Swing term, here's my suggestion Use the zooming
> image visualiser as the outer layer, and (probably minimising the
> minimising the number of zoom layers by using a large image in a
> JScrollPane with co-ordinates etc used as the frame's title. When zoomed
> in close enough, use a to JTable to show the actual content, populating
> the JTable model with the relevant patch copied from the master data
> matrix. The first column and the header row would be used to show matrix
> co-ordinates.

Huh. Hadn't thought of that approach. Interesting. I thought of some
sort of a "picture in a picture", "you are here" type of a gadget for
navigation.

>
> I've recently been using a JScrollPane containing a set of JPanes managed
> by the CardLayout layout manager to navigate a set of related panels,
> some of which are too big to fit my screen. This worked well once I'd got
> my head round how you're meant to use CardLayouts (which is not well
> explained in the Oracle's tutorials or the class-level documentation) but
> using the CardLayout manager may offer a convenient way of stepping up
> and down the zoom levels.
>
>

[toc] | [prev] | [next] | [standalone]

#31098

On 02/09/2015 08:31 AM, Jeff Higgins wrote:

> sort of a "picture in a picture", "you are here" type of a gadget for
> navigation.

<https://today.java.net/pub/a/today/2007/03/27/navigable-image-panel.html>
But as some sort of pop-up gadget separate from the table view.

[toc] | [prev] | [next] | [standalone]

#31099

On 02/09/2015 08:44 AM, Jeff Higgins wrote:
Just a memento for me.
Hierarchical Data Format
<http://www.hdfgroup.org/>
PyTables & ViTables is something I want to look at.

[toc] | [prev] | [next] | [standalone]

#31110

On 2015-02-09 15:01:31 +0000, Jeff Higgins said:

> On 02/09/2015 08:44 AM, Jeff Higgins wrote:
> Just a memento for me.
> Hierarchical Data Format
> <http://www.hdfgroup.org/>
> PyTables & ViTables is something I want to look at.

I use HDF for data storage, but HDF need a C based component (in Java 
you need the dynamic linked library). The library is used with JNI in 
Java and create a system dependency, because my builded Jar
need the DLL for all OS, so I need a native Java solution

Thanks

[toc] | [prev] | [next] | [standalone]

#31109

On 2015-02-09 13:10:45 +0000, Martin Gregorie said:

> On Mon, 09 Feb 2015 03:43:38 -0500, Jeff Higgins wrote:
> 
>> On 02/07/2015 09:17 PM, Jeff Higgins wrote:
>>> and conclude you are not looking to visualize results but for a massive
>>> spreadsheet.
>> 
>> So you'll likely want to tile your matrix.
>> SWT table model has "virtual" tables built into the api.
>> <https://eclipse.org/articles/Article-SWT-Virtual/Virtual-in-SWT.html>
>> But I don't know how (well) it works, I haven't used tried to use.
>> 
>> When I open LibreOffice Calc in a maximised window I see fifteen columns
>> and forty-five rows. Perhaps that is a good tile size. So keep a nine by
>> nine block of tiles from your matrix as your "virtual" table viewport
>> and move the viewport about as your user scrolls.
>> ?
> 
> This may not be exactly what the OP wants, but should at least be fairly
> easy to implement. In Swing term, here's my suggestion Use the zooming
> image visualiser as the outer layer, and (probably minimising the
> minimising the number of zoom layers by using a large image in a
> JScrollPane with co-ordinates etc used as the frame's title. When zoomed
> in close enough, use a to JTable to show the actual content, populating
> the JTable model with the relevant patch copied from the master data
> matrix. The first column and the header row would be used to show matrix
> co-ordinates.
> 
> I've recently been using a JScrollPane containing a set of JPanes managed
> by the CardLayout layout manager to navigate a set of related panels,
> some of which are too big to fit my screen. This worked well once I'd got
> my head round how you're meant to use CardLayouts (which is not well
> explained in the Oracle's tutorials or the class-level documentation) but
> using the CardLayout manager may offer a convenient way of stepping up
> and down the zoom levels.


Also this sounds great

Thanks

[toc] | [prev] | [next] | [standalone]

#31108

On 2015-02-09 08:43:38 +0000, Jeff Higgins said:

> On 02/07/2015 09:17 PM, Jeff Higgins wrote:
>> and conclude you are not looking to visualize results
>> but for a massive spreadsheet.
> 
> So you'll likely want to tile your matrix.
> SWT table model has "virtual" tables built into the api.
> <https://eclipse.org/articles/Article-SWT-Virtual/Virtual-in-SWT.html>
> But I don't know how (well) it works, I haven't used tried to use.
> 
> When I open LibreOffice Calc in a maximised window I see
> fifteen columns and forty-five rows. Perhaps that is a good
> tile size. So keep a nine by nine block of tiles from your
> matrix as your "virtual" table viewport and move the viewport
> about as your user scrolls.
> ?

That seems to be a nice idea.I'm thinking about a sparse data structure
and this mixed with a viewpoint sounds great

Thanks

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web