Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #38593 > unrolled thread

Displaying a ragged array

Back to article view | Back to comp.lang.java.programmer

Contents

  Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-16 22:17 +0000
    Re: Displaying a ragged array Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2018-09-17 16:00 +0200
      Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-17 19:32 +0000
        Re: Displaying a ragged array Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2018-09-17 23:29 +0200
          Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-17 23:43 +0000
            Re: Displaying a ragged array Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2018-09-19 18:12 +0200
              Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-19 17:49 +0000
                Re: Displaying a ragged array Eric Sosman <esosman@comcast-dot-net.invalid> - 2018-09-19 14:25 -0400
                  Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-19 19:29 +0000
                  Re: Displaying a ragged array Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2018-09-20 15:30 +0200
                    Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-20 15:06 +0000
                Re: Displaying a ragged array Eric Douglas <e.d.programmer@gmail.com> - 2018-09-24 08:11 -0700
                  Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-24 17:52 +0000
                    Re: Displaying a ragged array Eric Douglas <e.d.programmer@gmail.com> - 2018-09-24 12:01 -0700
                      Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-10-23 21:00 +0000
    Re: Displaying a ragged array Eric Sosman <esosman@comcast-dot-net.invalid> - 2018-09-17 10:17 -0400
      Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-17 19:46 +0000
        Re: Displaying a ragged array Martin Gregorie <martin@mydomain.invalid> - 2018-09-17 23:53 +0000

#38593 — Displaying a ragged array

I'm in the process of creating a desktop application to display and edit
data that's best treated as a ragged array of Strings: it is quite
unlikely that all rows in the array will be the same length.

I've built and debugged a Datastore class to hold it as an ArrayList of
ArrayList<String>  - the ArrayList<String> being in a wrapper class
representing a row of fields. The DataStore allows rows, columns and
individual fields to be inserted, deleted, and the fields to be modified.

Datastore extends AbstractTableModel because I was initially intending to
use a JTable to display and manipulate its contents, but now its
successfully manipulating the ragged array I'm having twinges of doubt
about how whether a JTable can is the best way of displaying it and
allowing it to be viewed and modified, i.e. can a JTable deal successfully
with the missing elements at the end of rows in a ragged array?

Would I be better off with a custom built structure containing, say, an
array of JButtons to display fields in the ragged array and that can be
clicked to add a field, edit or delete it?

I'll be very interested to hear how you've handled, or would handle, this
display problem.




-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [next] | [standalone]

#38594

On 2018-09-17 00:17, Martin Gregorie wrote:
> I'm in the process of creating a desktop application to display and edit
> data that's best treated as a ragged array of Strings: it is quite
> unlikely that all rows in the array will be the same length.
> 
> I've built and debugged a Datastore class to hold it as an ArrayList of
> ArrayList<String>  - the ArrayList<String> being in a wrapper class
> representing a row of fields. The DataStore allows rows, columns and
> individual fields to be inserted, deleted, and the fields to be modified.
> 
> Datastore extends AbstractTableModel because I was initially intending to
> use a JTable to display and manipulate its contents, but now its
> successfully manipulating the ragged array I'm having twinges of doubt
> about how whether a JTable can is the best way of displaying it and
> allowing it to be viewed and modified, i.e. can a JTable deal successfully
> with the missing elements at the end of rows in a ragged array?
> 
> Would I be better off with a custom built structure containing, say, an
> array of JButtons to display fields in the ragged array and that can be
> clicked to add a field, edit or delete it?
> 
> I'll be very interested to hear how you've handled, or would handle, this
> display problem.
> 
> Datastore extends AbstractTableModel

I think you should compose, rather than inherit that (i.e. a
DatastoreTableModel layer).

> can a JTable deal successfully
> with the missing elements at the end of rows in a ragged array

I don't see why it wouldn't (all you need to do is render missing values
properly, see below). Also, will missing elements always be at the end,
rather than interspersed?

> I'll be very interested to hear how you've handled, or would handle, this
> display problem.

A _display_ problem I would address at the _rendering_ level, rather
than the model level. That is, using the TableCellRenderer and
TableCellEditor.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#38596

On Mon, 17 Sep 2018 16:00:03 +0200, Daniele Futtorovic wrote:

> I think you should compose, rather than inherit that (i.e. a
> DatastoreTableModel layer).
>
Why? 
 
> I don't see why it wouldn't (all you need to do is render missing values
> properly, see below). Also, will missing elements always be at the end,
> rather than interspersed?
>
Yes, missing elements will always be at the end. They are distinct from 
empty elements, which are represented by empty Strings. Missing elements 
are off the end of the ArrayList, i.e. get(n) would throw an 
IndexOutOfBoundsException if asked to fetch them. Datastore returns null 
and sets an error (testable with hasError()) if you try it.
 
>> I'll be very interested to hear how you've handled, or would handle,
>> this display problem.
> 
> A _display_ problem I would address at the _rendering_ level, rather
> than the model level. That is, using the TableCellRenderer and
> TableCellEditor.
>
The app is an editor for manipulating CSV files with embedded comment 
lines. Yes, I know that yoy can use a spreadsheet, but they can and do 
mess with fields that they think need special treatment, e.g. dates.

One problem I see with using a JTable is that, while a comment line can 
be handled as a line containing one field, it would be would be best 
treated as a complete line, because its width should not affect the first 
column width, i.e. the displayed width of the first fields on other lines.


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38598

On 2018-09-17 21:32, Martin Gregorie wrote:
> On Mon, 17 Sep 2018 16:00:03 +0200, Daniele Futtorovic wrote:
> 
>> I think you should compose, rather than inherit that (i.e. a
>> DatastoreTableModel layer).
>>
> Why? 

The old "favour composition over inheritance" thing. Your Datastore is
not a TableModel, it's a Datastore. E.g. it might get populated by other
sub-components. Why should those have to interact with a TableModel
that's none of their business?

> Yes, missing elements will always be at the end. They are distinct from 
> empty elements, which are represented by empty Strings. Missing elements 
> are off the end of the ArrayList, i.e. get(n) would throw an 
> IndexOutOfBoundsException if asked to fetch them. Datastore returns null 
> and sets an error (testable with hasError()) if you try it.

(A quick size() check might be preferable to throwing an Exception)

>> A _display_ problem I would address at the _rendering_ level, rather
>> than the model level. That is, using the TableCellRenderer and
>> TableCellEditor.
>>
> The app is an editor for manipulating CSV files with embedded comment 
> lines. Yes, I know that yoy can use a spreadsheet, but they can and do 
> mess with fields that they think need special treatment, e.g. dates.
> 
> One problem I see with using a JTable is that, while a comment line can 
> be handled as a line containing one field, it would be would be best 
> treated as a complete line, because its width should not affect the first 
> column width, i.e. the displayed width of the first fields on other lines.

I can see two options.
One would be to display comments separately, in a dedicated UI
component, when a specific data row is selected. This assumes that
comment lines "belong" to a specific data row.
The other would be to use a JTreeTable. I'd love to get you a link for
that, but that Oracle's Java site is so goddamn fucking useless I
haven't been able to find the relevant article. I think it was somewhere
in the Swing tutorials.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#38599

On Mon, 17 Sep 2018 23:29:39 +0200, Daniele Futtorovic wrote:

> On 2018-09-17 21:32, Martin Gregorie wrote:
>> On Mon, 17 Sep 2018 16:00:03 +0200, Daniele Futtorovic wrote:
>> 
>>> I think you should compose, rather than inherit that (i.e. a
>>> DatastoreTableModel layer).
>>>
>> Why?
> 
> The old "favour composition over inheritance" thing. Your Datastore is
> not a TableModel, it's a Datastore. E.g. it might get populated by other
> sub-components. Why should those have to interact with a TableModel
> that's none of their business?
>
Fair comment, though its unlikely to gain subcomponents as I designed it 
to do nothing except store data fields and support the insert*/delete*/
get* methods needed to manage its contents plus, of course, the three 
TableModel interface methods (getColumnCount, getRowCount, getValueAt).
 
>> Yes, missing elements will always be at the end. They are distinct from
>> empty elements, which are represented by empty Strings. Missing
>> elements are off the end of the ArrayList, i.e. get(n) would throw an
>> IndexOutOfBoundsException if asked to fetch them. Datastore returns
>> null and sets an error (testable with hasError()) if you try it.
> 
> (A quick size() check might be preferable to throwing an Exception)
> 
I did that already: Datastore checks row and column indexes and, if out 
of range or pointing at a numexistent field, sets an error condition 
(visible with hasError() and returns null.
 
>>> A _display_ problem I would address at the _rendering_ level, rather
>>> than the model level. That is, using the TableCellRenderer and
>>> TableCellEditor.
>>>
Sure. Datastore is part of the Model class . The display mechanics will 
be in View (not yet written).

>> The app is an editor for manipulating CSV files with embedded comment
>> lines. Yes, I know that yoy can use a spreadsheet, but they can and do
>> mess with fields that they think need special treatment, e.g. dates.
>> 
>> One problem I see with using a JTable is that, while a comment line can
>> be handled as a line containing one field, it would be would be best
>> treated as a complete line, because its width should not affect the
>> first column width, i.e. the displayed width of the first fields on
>> other lines.
> 
> I can see two options.
> One would be to display comments separately, in a dedicated UI
> component, when a specific data row is selected. This assumes that
> comment lines "belong" to a specific data row.
>
I'm still thinking about that. You're right that they belong in a 
specific row, so should be displayed at the correct position(s) in the 
row list.

> The other would be to use a JTreeTable. I'd love to get you a link for
> that, but that Oracle's Java site is so goddamn fucking useless I
> haven't been able to find the relevant article. I think it was somewhere
> in the Swing tutorials.
>
Is that like a JTree, but with the content of branches displayed 
horizontally rather than hanging down as in a JTree? It doesn't appear to 
be part of the Java 8 standard class library. 

FWIW I've dropped Oracle Java in favour of Open Java, which is currently 
at v1.8, because Open Java is a standard Fedora package and updates are 
automatically part of my normal weekly Linux update.


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38601

On 2018-09-18 01:43, Martin Gregorie wrote:
> On Mon, 17 Sep 2018 23:29:39 +0200, Daniele Futtorovic wrote:
>> I can see two options.
>> One would be to display comments separately, in a dedicated UI
>> component, when a specific data row is selected. This assumes that
>> comment lines "belong" to a specific data row.
>>
> I'm still thinking about that. You're right that they belong in a 
> specific row, so should be displayed at the correct position(s) in the 
> row list.

I was thinking of a separate component, say a TextArea in some corner,
for the comment, which would get shown when the row it belongs to is
selected. It depends on whether you need to see comments all the time,
or if it's fine just having it when the row is selected.

>> The other would be to use a JTreeTable. I'd love to get you a link for
>> that, but that Oracle's Java site is so goddamn fucking useless I
>> haven't been able to find the relevant article. I think it was somewhere
>> in the Swing tutorials.
>>
> Is that like a JTree, but with the content of branches displayed 
> horizontally rather than hanging down as in a JTree? It doesn't appear to 
> be part of the Java 8 standard class library. 

I might be misremembering it. Doc says it's a JTable "using a JTree as a
renderer (and editor) for the cells in a particular column in the JTable".

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#38602

On Wed, 19 Sep 2018 18:12:20 +0200, Daniele Futtorovic wrote:

> I was thinking of a separate component, say a TextArea in some corner,
> for the comment, which would get shown when the row it belongs to is
> selected. It depends on whether you need to see comments all the time,
> or if it's fine just having it when the row is selected.
>
OK, Now I understand why you suggested using a tree structure. 

The first time I needed to edit the files that I'd use the editor on, I 
used a plain text editor, which was OK because big sections of it had 
similar length data in adjacent rows. Last time I had to change these 
files I did the job with OfficeLibre Calc, which was much easier because 
the data presentation was better. The main bugbear of using spreadsheets 
comes from the way that can mangle date and numeric columns. This wasn't 
a problem for me (either because Calc is fairly benign in this respect or 
because the data in these file is mostly alphanumeric) but friends say 
that data mangling is a real problem if you try using Excel to edit CSV 
files. Consequently I really want a similar window layout to a typical 
spreadsheet.

As I said, the files its most likely to get used on are quite large  - 14 
columns by 5800 lines, they are around 450KB in size. These are backup 
files used by a program I wrote (in Java of course) to manage setting up 
duty rosters for a large gliding club. This deals with creating six 
weekend rosters for a six month season, manned by just over 100 people 
and taking into account when they're available, that some are on more 
than one roster and that there are good reasons (such as shared glider 
ownership) why some people much not be rostered on the same day. IOW the 
rosters are not independent - so can't be backed up separately because 
doing so can mess up the crosslinks. Fairly infrequently something 
happens while the rosters are being set up (a multi-day process) which 
can require somebody to be removed from a roster that hasn't yet been 
worked on: they can't be removed from the 
list of members on the specific roster without messing up the backup 
files, which don't contain static data like who is on what roster, but do 
contain links to it. Hence the necessity to also edit them out of the 
latest backup as well - this is the reason I'm writing the editor.

Apologies for that over-large explanation, but I felt that an explanation 
of the problem size and why it needs the ability to display a lot of data 
in a reasonably compact form.

>> Is that like a JTree, but with the content of branches displayed
>> horizontally rather than hanging down as in a JTree? It doesn't appear
>> to be part of the Java 8 standard class library.
> 
> I might be misremembering it. Doc says it's a JTable "using a JTree as a
> renderer (and editor) for the cells in a particular column in the
> JTable".
>
OK. Thanks anyway.

I now have the datastore working more or less as I want it. It can 
operate at the column and row level (insert/delete operations) and also 
has the ability to insert/delete/modify a specified field in a selected 
row. It now does the right thing with comment lines, which occupy a 
single, rather wide field in a row, and understands this always starts 
with a marker character and is ignored when calculating the width of the 
first column (unlike a spreadsheet). 


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38603

On 9/19/2018 1:49 PM, Martin Gregorie wrote:
> [...]
> Last time I had to change these
> files I did the job with OfficeLibre Calc, which was much easier because
> the data presentation was better. The main bugbear of using spreadsheets
> comes from the way that can mangle date and numeric columns. This wasn't
> a problem for me (either because Calc is fairly benign in this respect or
> because the data in these file is mostly alphanumeric) but friends say
> that data mangling is a real problem if you try using Excel to edit CSV
> files. Consequently I really want a similar window layout to a typical
> spreadsheet.

     It might be better to teach the users how to use a spreadsheet in a
mode that isn't "helpful."  With LibreOffice (and other StarOffice
offspring), the open-file dialog allows you to designate one or more
columns as "Text" rather than as "Standard" (or various "Date" forms),
which prevents the spreadsheet from trying to guess at the contents and
format "accordingly."  I'm neither Excel-ent nor a Lotus-eater, but I'd
be surprised if they didn't have equivalent capabilities.

     If you're in an environment where the users can't be taught (maybe
they're downloaders unknown to you, maybe they're just dense), then
you'll be in the unpleasant position of offering them Yet Another Editor
Interface To Learn Unaided, and they'll be in the unpleasant position of
Struggling With An Editor Less Powerful Than Their Favorite.  The
unpleasant positions may be unavoidable, but if there's even a slight
chance of avoiding them I'd advise you to try.

     One further possibility (only you can say whether it's "possible" or
"out of the question"): Store the data as actual spreadsheet files --
as .ods rather than as .csv, for LibreOffice.  That way you can set up
the proper formatting (all "Text," most likely) to begin with, and the
spreadsheet will remember it and preserve it each time it's opened.
Java tools for reading and writing ODF exist (I know of jOpenDocument
and ODFDOM, although the freely-available documentation isn't great),
or you can even use UNO to control a running LibreOffice instance from
Java.  I found the official ODF spec rather daunting and gave up trying
to read it; nonetheless I was able to reverse-engineer enough of the
format to load simple .ods spreadsheets.  Unsupported code available on
request.

-- 
esosman@comcast-dot-net.invalid
Eight hundred fifty-four days to go.

[toc] | [prev] | [next] | [standalone]

#38604

On Wed, 19 Sep 2018 14:25:36 -0400, Eric Sosman wrote:

> It might be better to teach the users how to use a spreadsheet in a
> mode that isn't "helpful."
>
That's not an issue: the Roster Manager, thats me currently, is the only 
person who uses the roster setup program and, as I said, that only done 
every six months and takes more than a day but less than a week to do. 
Once the rosters are built they get uploaded to our webserver which does 
everything else: displays rosters, sends reminders of upcoming duties and 
has a swaps tool for dealing with cases when a member can't do a 
scheduled duty due to illness, family commitments, work, etc.

Learning to set up the rosters is quite a big deal (there are 
manuals...). The reason I built my rostering program is because my 
predecessor did it with a giant spreadsheet (8 large interconnected 
worksheets that needed to be edited before each season to adjust each 
roster page for the number of rosterees and days in the season before it 
could be used, IOW almost unmaintainable by anybody except its author. 

I replaced it with a Java program that has a very similar user interface. 
That was pretty good on the original, both to use and in the way its 
presented data it its user. My tool configures itself automatically when 
it sucks in the static data: lists of members, their availability 
patterns, preferences and new season's calendar. I'm pleased with the 
result: its easy to use, fast enough and also automates several 
crosschecks, duty spreading rules etc. that were purely manual tasks with 
the spreadsheet.

> If you're in an environment where the users can't be taught (maybe
> they're downloaders unknown to you, maybe they're just dense), then
> you'll be in the unpleasant position of offering them Yet Another Editor
> Interface To Learn Unaided, and they'll be in the unpleasant position of
> Struggling With An Editor Less Powerful Than Their Favorite.
>
I think its a safe bet that anybody like that won't take on being Roster 
Manager in the first place: apart from having to get used to the 
rostering process, which is much harder IME than than using any rostering 
tool such as my RosterEdit, they'll also be faced with downloading member 
details and their availability & preferences data from the website and 
uploading the completed rosters. 

Both the downloads and uploads are screen scraping cut 'n paste 
operations via a simple transfer program (also Java) that I wrote to make 
that as fast and simple as possible: it just converts the websites' TAB 
delimited CSV formats to comma delimited files when I pull stuff off the 
website and does the opposite when uploading completed rosters for 
publication. The previous incumbent just cut 'n pasted directly into and 
from his spreadsheet.

In any case, once somebody else gets lumbered with the Roster Manager 
job, I'll pass over this CSV editor (always assuming that I think its any 
good when its built - it may not be) but I won't really care if they use 
that, a spreadsheet, or their favourite text editor if they have to tweak 
any of the files used or output by RosterEdit.

> Store the data as actual spreadsheet files -- as
> .ods rather than as .csv, for LibreOffice.  That way you can set up the
> proper formatting (all "Text," most likely) to begin with, and the
> spreadsheet will remember it and preserve it each time it's opened. Java
> tools for reading and writing ODF exist (I know of jOpenDocument and
> ODFDOM, although the freely-available documentation isn't great), or you
> can even use UNO to control a running LibreOffice instance from Java.  I
> found the official ODF spec rather daunting and gave up trying to read
> it; nonetheless I was able to reverse-engineer enough of the format to
> load simple .ods spreadsheets.  Unsupported code available on request.
>
Could be done: an interesting thought, though probably not worth the 
effort to change my code now its been settled down and stable for the 
last couple of years. 

Using CSV was a no-brainer since the import/export formats used by the 
website were already CSV and I already had a decent, well-tested CSV file 
i/o class when  I started work on the rostering system.  In any case the 
extra complexity may make the load/save operations somewhat slow. I 
prefer comma separators to TABs because I find those easier to read and 
manipulate during development and trouble-shooting, but that's trivia.

BTW, I'm surprised there isn't a CSV file handling class in the Java 
standard library.


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38608

On 2018-09-19 20:25, Eric Sosman wrote:
>     One further possibility (only you can say whether it's "possible" or
> "out of the question"): Store the data as actual spreadsheet files --
> as .ods rather than as .csv, for LibreOffice.  That way you can set up
> the proper formatting (all "Text," most likely) to begin with, and the
> spreadsheet will remember it and preserve it each time it's opened.
> Java tools for reading and writing ODF exist (I know of jOpenDocument
> and ODFDOM, although the freely-available documentation isn't great),
> or you can even use UNO to control a running LibreOffice instance from
> Java.  I found the official ODF spec rather daunting and gave up trying
> to read it; nonetheless I was able to reverse-engineer enough of the
> format to load simple .ods spreadsheets.  Unsupported code available on
> request.

I remember using the (Star|Open|Libre)Office API from Java to create PDF
files, a couple years ago. It was a harrowing experience - akin to the
COM mess (which is to say that you can still make something worthwhile
out of it).

CSV is a good data format. Until your process involves a human opening
those files in a spreadsheet editor.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#38609

On Thu, 20 Sep 2018 15:30:47 +0200, Daniele Futtorovic wrote:

> CSV is a good data format. Until your process involves a human opening
> those files in a spreadsheet editor.
>
Indeed. One of its prime benefits being that its readable without using 
anything more complex than 'less', your favourite text editor or a 
printer. That is, unless its using TAB field separators which can make 
reading the file problematic when other fields are highly variable in 
length and the file contains empty fields. 


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38611

On Wednesday, September 19, 2018 at 1:49:38 PM UTC-4, Martin Gregorie wrote:
> 
> As I said, the files its most likely to get used on are quite large  - 14 
> columns by 5800 lines, they are around 450KB in size. These are backup 

That is really not a lot of data.  I just ran my test program with my custom grid (extended JTable).  It created a window with a menu, a searchable grid and 10 other components, ran a SQL query, populated the grid with 30 columns and 67047 rows, and displayed it all in under 14 seconds from scratch (nothing cached).

Loading data from a text file (or SQL table) should be pretty fast.  I use the Apache POI to save to xlsx format through a custom class I wrote to keep it clean and it does take a lot longer, so the only reason to do that generally is if someone is also using a spreadsheet program to maintain the data.

I'm certain my API can do everything you would need and do it efficiently but I have spent years developing it, so it's hard to say the best way if you're writing from scratch and want it up and running quickly.  We'd need a complete detail of the data structure to determine which controls are required.  Did you want one comment for each table row or multiple?  How long can a comment be?  Can it contain linefeeds?  Do you need to see the entire comment at once to edit it?  Do you want to see multiple entire comments at once?  You'll save time on coding if you can "flesh out" the design more.

You need a treetable control if you need to display spreadsheet style data with hidden data (collapsible rows). There are 2 forms of tree-tables.  I have found an example that uses a JTree and renders each node as a JTable row, though more common seems to be a JTable which renders one column as a JTree.  This is what I did, extended my custom JTable to make a JTreeTable, and extended my custom JTree to make the table renderer.  You can write a custom class like this, use javafx.scene.control.TreeTableView, or search up a custom JTreeTable from github.  Oracle provides a Swing JTreeTable in oracle.ide.controls.JTreeTable which would be from their paid application.

[toc] | [prev] | [next] | [standalone]

#38612

On Mon, 24 Sep 2018 08:11:22 -0700, Eric Douglas wrote:

> On Wednesday, September 19, 2018 at 1:49:38 PM UTC-4, Martin Gregorie
> wrote:
>> 
>> As I said, the files its most likely to get used on are quite large  -
>> 14 columns by 5800 lines, they are around 450KB in size. These are
>> backup
> 
> That is really not a lot of data.  I just ran my test program with my
> custom grid (extended JTable).  It created a window with a menu, a
> searchable grid and 10 other components, ran a SQL query, populated the
> grid with 30 columns and 67047 rows, and displayed it all in under 14
> seconds from scratch (nothing cached).
> 

Yep, you're dealing with a lot of rows and columns but, out of curiosity, 
what's the table size in kb? What length is the data in your widest and 
narrowest columns?

Another interesting comparison would be to know how 'skittish' you find 
scrolling with the scrollbar. I wrote a maintenance app for my mail 
archive, based on a PostgreSQL database. When scrolling up and down the  
16,700 addresses in the database its twitchy enough to make showing a 
specific address relatively difficult - the scrollbar slider is the same 
depth as a line of text in the JTable at this point and so a very small 
slider movement scrolls past quite a lot of lines. I assume you have the 
same problem.

> Loading data from a text file (or SQL table) should be pretty fast.
>
My roster editor sets up six empty rosters containing 106 rosterees and 
52 days (rows) per roster in 5.6 - 7 seconds on the first run of the day 
and just 2.2 seconds if its subsequently stopped and restarted. It 
restores a backup of all six completed rosters in about a second (0.9-1.1 
secs). All figures from my Seiko sport stopwatch. The restore involves 
loading the backed up data into class instances that were created when 
the program started up and displayed the set of unpopulated rosters.

I think this shows that, for the CSV editor, the time spent reading the 
file is likely to be a lot less than the time spent constructing the 
class instances used to contain and to display the data.
  
> use the Apache POI to save to xlsx format through a custom class I wrote
> to keep it clean and it does take a lot longer, so the only reason to do
> that generally is if someone is also using a spreadsheet program to
> maintain the data.
> 
Yes, agreed.

> I'm certain my API can do everything you would need and do it
> efficiently but I have spent years developing it, so it's hard to say
> the best way if you're writing from scratch and want it up and running
> quickly.
>
I'll try a JTable in a JScrollPane first, initially without editable 
cells. If it can show my data in a reasonably tidy way, that's job done. 
If not, I'll use a variation on the way the roster editor works, which is 
to add a row of class instances to the JViewport as each line of the CSV 
file is read. My intnetion to try this approach first is why my Datastore 
class extends AbstractTableModel.

Your experience shows this may be fast enough to be acceptable, which 
would be great.  However, if it is agonisingly slow OR using scroll bars 
on such a large number of rows is too skittish then I'll try using a non-
scrolling window that is prefilled with rows of cells that can be:

- unused, in which case they'll all be invisible

- used for a comment line which shows the line number and a single
  editable text field with white text on a dark blue background

- used for a data line which shows the line number and an editable text
  field showing black text on a pale grey or white background

Scrolling would work by changing the top left row number and field number 
to move the viewpoint relative to the top left of the ragged array and 
painting a window-full of rows and fields starting from that position. 
Like a spreadsheet the line numbers on the left would always be visible, 
and the refresh after a movement should be should be fast.

Its probable that all fields will be JButtons so that the user clicks the:
 
- line number to insert new row(s) 

- top (field number) row to insert new column(s)

- a field to edit it

Like a spreadsheet, editing will be done will be in a wider, edit field 
just under the menu bar - again, like a spreadsheet.

> Did you want one comment for each table row or multiple?
>
No limit on how many comment lines are used or where they are in the file.

> How long can a comment be? 
No (sensible) limit

> Can it contain linefeeds? 

No. If you want a comment block, you use multiple comment lines, each 
preceded by the comment marker character and the comment line(s) may 
contain field separators, which are treated as just another character in 
the comment line. Maybe there are CSV files that don't work this way but 
I don't remember seeing one. 

Blank lines are OK too: they are treated as empty data lines.

Using the editor to add a row sticks a blank line into the matrix. The 
user then leaves it blank or adds one or more fields: if the row only has 
one field whose text starts with a comment marker its a comment line (and 
adding a second field will be treated as an error). Otherwise its just 
another CSV line, initially with a single field in it. 

Similarly, editing the first field in a row to make it start with a 
comment marker is also treated as an error.  

> Do you need to see the entire comment at once to edit it?
>
Not in the display area, but the field editor will have a much wider 
editing area. Datastore stores a comment line as a row with a single 
(large) text field. The first character in this field determines whether 
the row is treated as a comment line or a row of CSV fields. The comment 
marker is configurable but cannot be the same as the field separator 
character, which is also configurable.

> Do you want to see multiple entire comments at once?
>
Yes, that's implied.

> You'll save time on coding if you can "flesh out" the design more.
> 
Indeed, which is why so far I've concentrated on making the Datastore 
class work as I've just described. Now I'm happy with that, its time to 
add code to load/save DataStore content using my existing and well-tested 
CSV class and then to start work on the display appearance and logic 
before finally implementing the field editing facility.

> You need a treetable control if you need to display spreadsheet style
> data with hidden data (collapsible rows).
>
I don't think I need collapsable rows, but there will be a default column 
width - this will limit the display width of any column (and may cause a 
'+' or '>' to be displayed at the right end - this is currently 
undecided). 

The whole field can always be seen by using the field editor.

Later, I'll also add a capability to let the user to inspect the first 
few lines of the file while they set the comment marker and field 
separator characters.
 

-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38613

On Monday, September 24, 2018 at 1:52:25 PM UTC-4, Martin Gregorie wrote:
> Yep, you're dealing with a lot of rows and columns but, out of curiosity, 
> what's the table size in kb? What length is the data in your widest and 
> narrowest columns?
> 

table size in kb?  Tables are in memory, GUI objects, not sure what size we're looking for.  You mean the size on disk if I can write my results to a csv?  My test data is SQL query results.  My narrowest column on this particular query shows 6 characters, my widest shows 80 characters because I set that as my max width.  If I click into a cell that may hold more (some cells contain much more) the text editor does scroll to see the rest.  If I make a column too wide it doesn't display well.  The bigger your screen, the higher your resolution, the wider you could possibly make it.

> Another interesting comparison would be to know how 'skittish' you find 
> scrolling with the scrollbar. I wrote a maintenance app for my mail 
> archive, based on a PostgreSQL database. When scrolling up and down the  
> 16,700 addresses in the database its twitchy enough to make showing a 
> specific address relatively difficult - the scrollbar slider is the same 
> depth as a line of text in the JTable at this point and so a very small 
> slider movement scrolls past quite a lot of lines. I assume you have the 
> same problem.
> 

On the table with 67k rows if I click and drag the slider it does do many rows at once.  You can click the arrow boxes on the ends of the slider to move it one row, move the scroll wheel to scroll 3 rows at once, or navigate with the arrow keys.  I also use a SearchableGrid so if I want to maintain just one row or small group of rows I can type into my search box.  To make a SearchableGrid you implement the lucene library with a RowSorter.

> > Loading data from a text file (or SQL table) should be pretty fast.
> >
> My roster editor sets up six empty rosters containing 106 rosterees and 
> 52 days (rows) per roster in 5.6 - 7 seconds on the first run of the day 
> and just 2.2 seconds if its subsequently stopped and restarted. It 
> restores a backup of all six completed rosters in about a second (0.9-1.1 
> secs). All figures from my Seiko sport stopwatch. The restore involves 
> loading the backed up data into class instances that were created when 
> the program started up and displayed the set of unpopulated rosters.
> 

I use my custom StopWatch class, which grabs System.nanoTime() to start, then again to stop and subtracts.  Apparently we copied this from the internet, as it starts with a comment line @author Carlos Quintanilla.

> I think this shows that, for the CSV editor, the time spent reading the 
> file is likely to be a lot less than the time spent constructing the 
> class instances used to contain and to display the data.
>   

Of course file read times depend on your hard disk speed, and constructing instances depend on your CPU and RAM but I would expect getting the data from a local text file to be the faster part.  My test data is on a local server, running 15k hard drives, over a gigabit network.

[toc] | [prev] | [next] | [standalone]

#38645

On Mon, 24 Sep 2018 12:01:06 -0700, Eric Douglas wrote:

Following up: this hasn't had a lot of time spent on it, but I've proved 
to myself that using a JTable doesn't do what I wanted, so I'm using a 
ControlPanel (comes for free with the RiverLayout layout manager from 
datadosen.se). This is used to set up a rectangular grid of EditorFields 
which extend JButton and are initially set to 'invisible'

CSV data is parsed into fields and loaded into an instance of my DataStore 
class. Then a refresh method is called which displays the ragged array by 
loading the fields into left-justified rows of EditorFields. The result 
looks like this:

Line|  1   |  2   |  3   |+|
   1|#|
   2|# Clean CVS input test data|
   3|#|
   4|field1|field2|field3|+|
   5|field4|+|
   6|field5|field6|+|
   7|field7|field8|field9|+|
++|                     

- the Line column and column headings are black on grey.

- comments are white on blue and, in this example, comment lines start 
with a '#'

- Fields with a '+' in them are controls, not data. Clicking on the one
  at the end of the column headings appends a new column Clicking on a
  column heading inserts a new column before the one that was clicked

- The '++' field at the bottom appends a new line if clicked. Clicking a
  line number inserts a new line in front of the one that was clicked

- Clicking the '+' at the end of a data line appends a new field to that
  line. Clicking a field containing data or a column will allow it to be
  edited, deleted or to insert a new field  in that line on front of it.
 
You'll notice that all data fields are aligned in a column which is the 
width of the widest data item in that column while comment lines take up 
as much space as they need without affecting the width of the first field 
in any way and that comment lines can't be extended (no '+] button,
but if a new field is inserted in front of the comment, the line becomes 
a non-comment and behaves like any other data-containing line. You can 
get the same effect by editing the comment to remove its leading marker 
character. Similarly, editing a one field data line and adding a comment 
marker at the front of its only field turns it into a comment line.

Multiple files can be inspected/edited/created in one session.

At present the program displays data as described and saves files but 
doesn't yet accept changes to displayed material or scroll, but I'm now 
happy that the display had the characteristics I wanted and adding 
scrolling and editing will be reasonably straight-forward. 

It loads my 0.5MB CSV files in 2-3 seconds. From launch to displaying the 
sort of small test file shown above is under a second, even with fairly 
extensive diagnostics turned on.


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38595

On 9/16/2018 6:17 PM, Martin Gregorie wrote:
> I'm in the process of creating a desktop application to display and edit
> data that's best treated as a ragged array of Strings: it is quite
> unlikely that all rows in the array will be the same length.
> 
> I've built and debugged a Datastore class to hold it as an ArrayList of
> ArrayList<String>  - the ArrayList<String> being in a wrapper class
> representing a row of fields. The DataStore allows rows, columns and
> individual fields to be inserted, deleted, and the fields to be modified.
> 
> Datastore extends AbstractTableModel because I was initially intending to
> use a JTable to display and manipulate its contents, but now its
> successfully manipulating the ragged array I'm having twinges of doubt
> about how whether a JTable can is the best way of displaying it and
> allowing it to be viewed and modified, i.e. can a JTable deal successfully
> with the missing elements at the end of rows in a ragged array?

     When queried for an "absent" cell, your TableModel could return
something -- null, "", whatever -- instead of throwing an exception.
The JTable could then display the "absent" value in whatever way you
pleased.  Implementing your own TableCellRenderer would probably be
helpful.

     Whether the tabular display works well would seem to depend on
"how ragged" the grid becomes.  Your TableModel must advertise the
number of rows and columns in the grid, which probably means a lot of
work whenever the maximum row length changes.  If you've got twenty
rows of four columns and one of a hundred, things may get awkward.

> Would I be better off with a custom built structure containing, say, an
> array of JButtons to display fields in the ragged array and that can be
> clicked to add a field, edit or delete it?

     It appears the items in a single row are related somehow; are
those in a single column also related?  If so, a tabular display might
make the "vertical" relationships more apparent, and perhaps more
useful (you might sort on Column 3, for example).

     If there's no "vertical" relationship worth preserving, a table
of one column where each cell displays multiple items (maybe as a
bunch of JButtons, maybe as a JList, maybe as ...?) might work.
Again, wild discrepancies in the row lengths could be unpleasant.

> I'll be very interested to hear how you've handled, or would handle, this
> display problem.

     Have you considered using a JTree?

-- 
esosman@comcast-dot-net.invalid
Eight hundred fifty-six days to go.

[toc] | [prev] | [next] | [standalone]

#38597

On Mon, 17 Sep 2018 10:17:22 -0400, Eric Sosman wrote:

>  When queried for an "absent" cell, your TableModel could return
> something -- null, "", whatever -- instead of throwing an exception.
> 
Agreed.

> The JTable could then display the "absent" value in whatever way you
> pleased.  Implementing your own TableCellRenderer would probably be
> helpful.
> 
Probably necessary. As I said, this is intended to be a CSV file editor 
and I'd certainly want to use different background colours for (empty) 
data fields, non-existent fields and comment lines.

> Whether the tabular display works well would seem to depend on
> "how ragged" the grid becomes.  Your TableModel must advertise the
> number of rows and columns in the grid, which probably means a lot of
> work whenever the maximum row length changes.
>
Its easy to determine the max width of a column, but how fast it runs 
might be another issue.

> If you've got twenty rows
> of four columns and one of a hundred, things may get awkward.
>
It goes without saying that the table must scroll on both axes and it 
should be possible to set a max width on columns, say 20 chars by 
default. Wider columns (apart from comment lines) would only be viewable 
as a whole by using the field editor as a viewer.
 
>> Would I be better off with a custom built structure containing, say, an
>> array of JButtons to display fields in the ragged array and that can be
>> clicked to add a field, edit or delete it?
> 
> It appears the items in a single row are related somehow
>
Yes they are field in a single line.

> [Are] those in a single column also related?
>
Somewhat - in parts of the some (most?) files they will contain the same 
data, but some files have several sections, separated by some form of 
delimiter line, and these will contain different data, so the "same 
column holds the same data type" only applies with a section of this type 
of file.

> If so, a tabular display might
> make the "vertical" relationships more apparent, and perhaps more useful
> (you might sort on Column 3, for example).
>
Yes, that's the idea, though sorting is probably not terribly useful.
 
> If there's no "vertical" relationship worth preserving, a table
> of one column where each cell displays multiple items (maybe as a bunch
> of JButtons, maybe as a JList, maybe as ...?) might work. Again, wild
> discrepancies in the row lengths could be unpleasant.
> 
Indeed. I'll probably end up trying both types - whichever is easiest to 
set up being tried first.

>      Have you considered using a JTree?
>
No, but will take a look at it.


-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [next] | [standalone]

#38600

On Mon, 17 Sep 2018 19:46:10 +0000, Martin Gregorie wrote:

>>      Have you considered using a JTree?
>>
> No, but will take a look at it.
>
I've just looked and, given that some of the files I'd want to edit have 
14 columns in some sections and 5800 rows overall, the impression I get 
from a quick glance at the tutorial is that the sheer verticality of a 
JTree-based display would make the thing unmanageable.

The ideal would be for a comment to occupy one row showing its line 
number, for reference, plus a single field containing the text, while 
other rows would show the line number followed by the fields in the same 
sequence as they appear in the source file.
 

-- 
Martin    | martin at
Gregorie  | gregorie dot org

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web