Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #51689 > unrolled thread

Editing tabular data [was: PEP8 79 char max]

Back to article view | Back to comp.lang.python

Contents

  Editing tabular data [was: PEP8 79 char max] Skip Montanaro <skip@pobox.com> - 2013-07-31 13:39 -0500
    Re: Editing tabular data [was: PEP8 79 char max] Grant Edwards <invalid@invalid.invalid> - 2013-07-31 19:02 +0000
      Re: Editing tabular data [was: PEP8 79 char max] Skip Montanaro <skip@pobox.com> - 2013-07-31 14:35 -0500
      Re: Editing tabular data [was: PEP8 79 char max] Chris Angelico <rosuav@gmail.com> - 2013-08-01 01:50 +0100
        Re: Editing tabular data [was: PEP8 79 char max] Neil Cerutti <neilc@norwich.edu> - 2013-08-01 14:04 +0000
        Re: Editing tabular data [was: PEP8 79 char max] wxjmfauth@gmail.com - 2013-08-02 02:15 -0700
      Re: Editing tabular data Ben Finney <ben+python@benfinney.id.au> - 2013-08-02 16:35 +1000
        Re: Editing tabular data rusi <rustompmody@gmail.com> - 2013-09-03 09:58 -0700
      LibreOffice (was: Editing tabular data) Ben Finney <ben+python@benfinney.id.au> - 2013-08-02 17:14 +1000
      Re: LibreOffice (was: Editing tabular data) Chris Angelico <rosuav@gmail.com> - 2013-08-02 10:37 +0100
        Re: LibreOffice (was: Editing tabular data) Grant Edwards <invalid@invalid.invalid> - 2013-08-02 14:00 +0000
      Re: LibreOffice Terry Reedy <tjreedy@udel.edu> - 2013-08-02 21:26 -0400
      Re: LibreOffice David Robinow <drobinow@gmail.com> - 2013-08-03 11:08 -0400
    Re: Editing tabular data [was: PEP8 79 char max] "Rhodri James" <rhodri@wildebst.demon.co.uk> - 2013-07-31 20:03 +0100
      Re: Editing tabular data [was: PEP8 79 char max] Neil Cerutti <neilc@norwich.edu> - 2013-07-31 19:07 +0000
    Re: Editing tabular data [was: PEP8 79 char max] Wanderer <wanderer@dialup4less.com> - 2013-07-31 13:37 -0700
      Re: Editing tabular data [was: PEP8 79 char max] Skip Montanaro <skip.montanaro@gmail.com> - 2013-07-31 18:40 -0500

#51689 — Editing tabular data [was: PEP8 79 char max]

> I don't understand.  That just moves them to a different file --
> doesn't it?  You've still got to deal with editing a large table of
> data (for example when I want to add instructions to your assembler).

My guess is it would be more foolproof to edit that stuff with a spreadsheet.

Skip

[toc] | [next] | [standalone]

#51692

On 2013-07-31, Skip Montanaro <skip@pobox.com> wrote:
>> I don't understand.  That just moves them to a different file --
>> doesn't it?  You've still got to deal with editing a large table of
>> data (for example when I want to add instructions to your assembler).
>
> My guess is it would be more foolproof to edit that stuff with a
> spreadsheet.

Many years ago, I worked with somebody who used a spreadsheet like
that.  I tried it and found it to be way too cumbersome. The overhead
involved of putting tables in to slew of different files and starting
up LibreOffice to edit/view them is huge compared to just editing them
with emacs in a file along with the source code.  Maybe my computer is
too old/slow.  Maybe it's just due to how bad I am at Excel/LibreOffice...

-- 
Grant Edwards               grant.b.edwards        Yow! I haven't been married
                                  at               in over six years, but we
                              gmail.com            had sexual counseling every
                                                   day from Oral Roberts!!

[toc] | [prev] | [next] | [standalone]

#51699

>> My guess is it would be more foolproof to edit that stuff with a
>> spreadsheet.
>
> Many years ago, I worked with somebody who used a spreadsheet like
> that.

I really love Emacs, however...  One of the traders here where I work
(who shall not be named) had a space-delimited data file with hundreds
of rows and 50 or so columns.  I could never get him to edit it in any
kind of spreadsheet or put it in a database (expecting him to master
SQL would have been pointless - I would have had to write a GUI tool
for him).  He always modified it in Emacs, and would delete columns,
add extra spaces, fragmentary rows, etc.  He'd edit this file late at
night, the automated processes the next morning would crap out, and I
would scramble to try and find and fix the problem before the market
opened.

This is clearly a case where choosing the proper tool is important.  I
agree that using a spreadsheet to edit a 3x5 CSV file is likely
overkill (might just as well use Notepad or TextEdit), but tabular
data are tabular data, no matter how they might be delimited, and if
there are many of those little data critters, there are better tools
than a text editor (or Python IDE) for maintaining them.

Skip

[toc] | [prev] | [next] | [standalone]

#51712

On Wed, Jul 31, 2013 at 8:02 PM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2013-07-31, Skip Montanaro <skip@pobox.com> wrote:
>>> I don't understand.  That just moves them to a different file --
>>> doesn't it?  You've still got to deal with editing a large table of
>>> data (for example when I want to add instructions to your assembler).
>>
>> My guess is it would be more foolproof to edit that stuff with a
>> spreadsheet.
>
> Many years ago, I worked with somebody who used a spreadsheet like
> that.  I tried it and found it to be way too cumbersome. The overhead
> involved of putting tables in to slew of different files and starting
> up LibreOffice to edit/view them is huge compared to just editing them
> with emacs in a file along with the source code.  Maybe my computer is
> too old/slow.  Maybe it's just due to how bad I am at Excel/LibreOffice...

I'm glad someone else feels that way!

At work, we have a number of CSV files (at my boss's insistence; I
would much rather they be either embedded in the source, or in some
clearer and simpler format) which I like to manipulate in SciTE,
rather than OO/LibreOffice. (I'll not distinguish those two. Far as
I'm concerned, they're one product with two names.) My boss can't
understand why I do this. I can't understand why he objects to having
to edit code files to alter internal data. I have pointed him to [1]
but to no avail.

The one thing I would do, though, is align with tabs rather than
spaces. That gives you an 8:1 (if you keep your tabs at eight, which I
do) improvement in maintainability, because edits that don't cross a
boundary don't require fiddling with the layout.

[1] http://thedailywtf.com/Articles/Soft_Coding.aspx

ChrisA

[toc] | [prev] | [next] | [standalone]

#51733

On 2013-08-01, Chris Angelico <rosuav@gmail.com> wrote:
> On Wed, Jul 31, 2013 at 8:02 PM, Grant Edwards <invalid@invalid.invalid> wrote:
>> On 2013-07-31, Skip Montanaro <skip@pobox.com> wrote:
>>>> I don't understand.  That just moves them to a different
>>>> file -- doesn't it?  You've still got to deal with editing a
>>>> large table of data (for example when I want to add
>>>> instructions to your assembler).
>>>
>>> My guess is it would be more foolproof to edit that stuff
>>> with a spreadsheet.
>>
>> Many years ago, I worked with somebody who used a spreadsheet
>> like that.  I tried it and found it to be way too cumbersome.
>> The overhead involved of putting tables in to slew of
>> different files and starting up LibreOffice to edit/view them
>> is huge compared to just editing them with emacs in a file
>> along with the source code.  Maybe my computer is too
>> old/slow.  Maybe it's just due to how bad I am at
>> Excel/LibreOffice...
>
> I'm glad someone else feels that way!
>
> At work, we have a number of CSV files (at my boss's
> insistence; I would much rather they be either embedded in the
> source, or in some clearer and simpler format) which I like to
> manipulate in SciTE, rather than OO/LibreOffice. (I'll not
> distinguish those two. Far as I'm concerned, they're one
> product with two names.) My boss can't understand why I do
> this. I can't understand why he objects to having to edit code
> files to alter internal data. I have pointed him to [1] but to
> no avail.
>
> The one thing I would do, though, is align with tabs rather
> than spaces. That gives you an 8:1 (if you keep your tabs at
> eight, which I do) improvement in maintainability, because
> edits that don't cross a boundary don't require fiddling with
> the layout.
>
> [1] http://thedailywtf.com/Articles/Soft_Coding.aspx

Thanks for that link. Good food for thought.

Here's an excerpt from one of my more questionable tables:

Attribute, Description, Fund, Amount
AFSO,Air Force Special Ops Command,,
CSEN,English Proficiency Met,,
CSMT,Math Proficiency Met,,
GBFP,MBA Full Program,,
GBMP,MBA Prereq Met,,
GCEC,Continuing Education Civilian Tuition Rate,,
GCEM,Continuing Education Military Tuition Rate,,
GCFP,MCA Prereq Needed,,
GCMP,MCE Prereq Met,,
GCRT,Certificate Student,,
GE25,25% to XCompany,,
GE40,40% to XCompany,,
GEMP,Employee,Fac,100%
GI03,CISSP Scholarship,CISSP,1500
GIHR,Grad In-House Recruiting,,
GRMS,Graduate Military Scholarship,Milit,1200

It lists all the student atributes, a description, what fund that
attribute requires, if any, and what amount. A tiny amount of DSL
is involved, with Faculty Scholarship paying 100% of tuition
instead of a fixed number. Another _ (not shown above), which
means the fund takes an arbitrary amount determined by a person
we have to literally query to discover.

I think I can see the potential problems. Two special codes for
amount is managable, but the more special cases I end up creating
the more of a mess I get. Plus, I haven't really documented the
file.

Most of the information is irrelevant, though I do like receiving
an exception when Admissions tries to sneak in a new attribute
without telling me.

If I instead had a function that handled only the interesting
attributes it might be pretty small. I'll have to think on this.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#51788

Le jeudi 1 août 2013 02:50:13 UTC+2, Chris Angelico a écrit :

> ...

> rather than OO/LibreOffice. (I'll not distinguish those two. Far as
> 
> I'm concerned, they're one product with two names.) 

...

Very interesting aspect in LibreOffice.

As the "center of gravity of the development" has moved
from "US" to "Europe", the product becomes clearly 
less (practicaly no more traces) ascii oriented.

jmf

[toc] | [prev] | [next] | [standalone]

#51785 — Re: Editing tabular data

Skip Montanaro <skip@pobox.com> writes:

> I really love Emacs, however... […]
>
> This is clearly a case where choosing the proper tool is important. I
> agree that using a spreadsheet to edit a 3x5 CSV file is likely
> overkill (might just as well use Notepad or TextEdit), but tabular
> data are tabular data, no matter how they might be delimited, and if
> there are many of those little data critters, there are better tools
> than a text editor (or Python IDE) for maintaining them.

It seems an obvious thing for powerful text editors like Emacs and Vim
to have a third-party mode for editing CSV data with a tabular
interface.

Indeed, such modes exist; one that I found immediately for Emacs is
<URL:http://www.emacswiki.org/emacs/csv-mode.el>. Has anyone got a good
Emacs mode for editing CSV data as a table and saving it back to CSV
data?

-- 
 \          “Isn't it enough to see that a garden is beautiful without |
  `\      having to believe that there are fairies at the bottom of it |
_o__)                                             too?” —Douglas Adams |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#53574 — Re: Editing tabular data

On Friday, August 2, 2013 12:05:53 PM UTC+5:30, Ben Finney wrote:
> Skip Montanaro writes:
> 
> > I really love Emacs, however... […]
> >
> > This is clearly a case where choosing the proper tool is important. I
> > agree that using a spreadsheet to edit a 3x5 CSV file is likely
> > overkill (might just as well use Notepad or TextEdit), but tabular
> > data are tabular data, no matter how they might be delimited, and if
> > there are many of those little data critters, there are better tools
> > than a text editor (or Python IDE) for maintaining them.
> 
> It seems an obvious thing for powerful text editors like Emacs and Vim
> to have a third-party mode for editing CSV data with a tabular
> interface.
> 
> 
> Indeed, such modes exist; one that I found immediately for Emacs is
> <URL:http://www.emacswiki.org/emacs/csv-mode.el>. Has anyone got a good
> Emacs mode for editing CSV data as a table and saving it back to CSV
> data?


Emacs users can have the cake and eat it too; ie use spreadsheet
functionality without having to use a separate spreadsheet file
and software.

The basic idea is to use org-mode which has a table
editor with spreadsheet functionality while continuing to live within a 
plain text editor.

It allows to edit a table entirely written in plain text in a visually appealing and clean way, while keeping a (less readable) python data
structure in sync with it.

The example file is below the ---------------

Caveats:
The orig_table string is there only make the source for the table.
The name orig_table is not necessary; a naked triple-string will also work.
The triple string is there to pacify python in the face of non valid syntax.
Ideal would have been comments but python does not have multiline comments

The table between the #BEGIN RECEIVE ORGTBL marks
and the #END RECEIVE ORGTBL marks
is the target or the recipient for the transformed version of the
plain text table.

Experiment as follows:
1. Save the stuff below --------- as something.py
2. Start editing the file in emacs

3. Join the 3 lines into 1 line with single space separators.
 +ORGTBL: SEND marks orgtbl-to-generic
  :lfmt "  \"%s\": [%s,%s,%s,%s,%s],"
  :llfmt "  \"%s\": [%s,%s,%s,%s,%s]"

   [It has to be one line, but if I kept it one line, it
   will be randomly be garbled in the mail!]

 This line gives the table a name ("marks") so that you can use
 several tables in one file, and it specifies how the syntax should
 be changed when syncing the python version of  the table data.

4. Start orgtbl minor mode with M-x orgtbl-mode
   Mode line should show python and orgtbl

5. Delete the contents (keep the 2 # lines intact) of the python table

6. Place cursor within the orig_table and 'send' it as follows

7. Send is done with any one of 'C-c C-c' or 'C-u C-c C-c' or 'C-u C-u C-c C-c'
   The first just sends the table as is
   The second recomputes the formulas top-down and then sends
   The third recomputes until fixpoint (you really should not be making such a  
   table!!)

8. Play with the table editor by using TAB and S-TAB to
   walk through fields and change them, use C-u C-c C-c again to
   sync the python version of the table

9. In case the above does now work (if your orgmode is too old)
   the orig_table_2 should hopefully work even for older org versions
   It furthermore shows the ability to skip columns and to format
   column widths to convenience.

----------------------------------
orig_table = """

#+ORGTBL: SEND marks orgtbl-to-generic
:lfmt "  \"%s\": [%s,%s,%s,%s,%s],"
:llfmt "  \"%s\": [%s,%s,%s,%s,%s]"
| abe    |  1 |  2 |  3 |  4 |    10 |
| beth   |  9 |  1 |  5 |  9 |    24 |
| catherine |  5 |  6 |  7 |  5 |    23 |
#+TBLFM: $6=$2+$3+$4+$5
"""

stud_db = {
# Dont handedit
# BEGIN RECEIVE ORGTBL marks
  "abe": [1,2,3,4,10],
  "beth": [9,1,5,9,24],
  "catherine": [5,6,7,5,23]
# END RECEIVE ORGTBL marks
}

## In case the above does not work (if org-version too old)

orig_table_2 = """

#+ORGTBL: SEND marks2 orgtbl-to-csv :skip 2
| Name   | T1 | T2 | T3 | T4 | Total |
| <6>    |    |    |    |    |       |
| abe    |  1 |  2 |  3 |  4 |    10 |
| beth   |  9 |  1 |  5 |  9 |    24 |
| catherine |  5 |  6 |  7 |  5 |    23 |
#+TBLFM: $6=$2+$3+$4+$5
"""

stud_db_2 = {
# Dont handedit
# BEGIN RECEIVE ORGTBL marks2
abe,1,2,3,4,10
beth,9,1,5,9,24
catherine,5,6,7,5,23
# END RECEIVE ORGTBL marks2

[toc] | [prev] | [next] | [standalone]

#51786 — LibreOffice (was: Editing tabular data)

Chris Angelico <rosuav@gmail.com> writes:

> […] rather than OO/LibreOffice. (I'll not distinguish those two. Far
> as I'm concerned, they're one product with two names.)

That's simply false. LibreOffice has, since the 2010 fork of the code
base and especially since the exodus of developers to The Document
Foundation [0], gained a great number of improvements [1] and is now the
clear inheritor of active development.

Oracle's behaviour (as holder of the OpenOffice trademark) shows they
have no interest in a free-software OpenOffice. Even Oracle Linux no
longer has OpenOffice, instead shipping LibreOffice [2].

Claiming they're the same product is ignoring the transfer of
development away from the OpenOffice.org code dump, and to LibreOffice
as the actively-developed product.

[0] https://en.wikipedia.org/wiki/LibreOffice#History_after_the_establishment_of_The_Document_Foundation
[1] https://en.wikipedia.org/wiki/LibreOffice#Release_history
[2] https://oss.oracle.com/ol6/docs/RELEASE-NOTES-U3-en.html

-- 
 \         “True greatness is measured by how much freedom you give to |
  `\      others, not by how much you can coerce others to do what you |
_o__)                                               want.” —Larry Wall |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#51789 — Re: LibreOffice (was: Editing tabular data)

On Fri, Aug 2, 2013 at 8:14 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
> Chris Angelico <rosuav@gmail.com> writes:
>
>> […] rather than OO/LibreOffice. (I'll not distinguish those two. Far
>> as I'm concerned, they're one product with two names.)
>
> That's simply false. ...
>
> Claiming they're the same product is ignoring the transfer of
> development away from the OpenOffice.org code dump, and to LibreOffice
> as the actively-developed product.

To be sure, they're different; but they're part of one family tree.
It's like referring to "Debian/Ubuntu" when you're discussing
something where it makes absolutely zero difference which one you're
talking about. The difference between using LibreOffice and using
OpenOffice is nothing compared to the difference between working with
either of the above and putting a literal in your code.

ChrisA

[toc] | [prev] | [next] | [standalone]

#51812 — Re: LibreOffice (was: Editing tabular data)

On 2013-08-02, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, Aug 2, 2013 at 8:14 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
>> Chris Angelico <rosuav@gmail.com> writes:
>>
>>> [?] rather than OO/LibreOffice. (I'll not distinguish those two. Far
>>> as I'm concerned, they're one product with two names.)
>>
>> That's simply false. ...
>>
>> Claiming they're the same product is ignoring the transfer of
>> development away from the OpenOffice.org code dump, and to LibreOffice
>> as the actively-developed product.
>
> To be sure, they're different; but they're part of one family tree.
> It's like referring to "Debian/Ubuntu" when you're discussing
> something where it makes absolutely zero difference which one you're
> talking about. The difference between using LibreOffice and using
> OpenOffice is nothing compared to the difference between working with
> either of the above and putting a literal in your code.

In the context in which I mentioned LibreOffice, I don't even consider
there to be a significant difference between Libre/OO and Excel.

-- 
Grant Edwards               grant.b.edwards        Yow! Those people look
                                  at               exactly like Donnie and
                              gmail.com            Marie Osmond!!

[toc] | [prev] | [next] | [standalone]

#51832 — Re: LibreOffice

On 8/2/2013 3:14 AM, Ben Finney wrote:
> Chris Angelico <rosuav@gmail.com> writes:
>
>> […] rather than OO/LibreOffice. (I'll not distinguish those two. Far
>> as I'm concerned, they're one product with two names.)
>
> That's simply false. LibreOffice has, since the 2010 fork of the code
> base and especially since the exodus of developers to The Document
> Foundation [0], gained a great number of improvements [1] and is now the
> clear inheritor of active development.

Of relevance to this list, Libre Office upgraded the included Python 
interpreter to 3.3. I have no idea whether OO is still using 2.3 or also 
updated.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#51865 — Re: LibreOffice

On Fri, Aug 2, 2013 at 9:26 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> ...
> Of relevance to this list, Libre Office upgraded the included Python
> interpreter to 3.3. I have no idea whether OO is still using 2.3 or also
> updated.
 They're up to 2.7 now.

[toc] | [prev] | [next] | [standalone]

#51693

On Wed, 31 Jul 2013 19:39:29 +0100, Skip Montanaro <skip@pobox.com> wrote:

>> I don't understand.  That just moves them to a different file --
>> doesn't it?  You've still got to deal with editing a large table of
>> data (for example when I want to add instructions to your assembler).
>
> My guess is it would be more foolproof to edit that stuff with a  
> spreadsheet.

There's nothing foolproof about using a spreadsheet!

-- 
Rhodri James *-* Wildebeest Herder to the Masses

[toc] | [prev] | [next] | [standalone]

#51694

On 2013-07-31, Rhodri James <rhodri@wildebst.demon.co.uk> wrote:
> On Wed, 31 Jul 2013 19:39:29 +0100, Skip Montanaro <skip@pobox.com> wrote:
>
>>> I don't understand.  That just moves them to a different file --
>>> doesn't it?  You've still got to deal with editing a large table of
>>> data (for example when I want to add instructions to your assembler).
>>
>> My guess is it would be more foolproof to edit that stuff with a  
>> spreadsheet.
>
> There's nothing foolproof about using a spreadsheet!

I edit csv files using Excel all the time. But I don't use it to
edit my hand-created data files. It does too much meddling.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#51704

On Wednesday, July 31, 2013 2:39:29 PM UTC-4, Skip Montanaro wrote:
> > I don't understand.  That just moves them to a different file --
> 
> > doesn't it?  You've still got to deal with editing a large table of
> 
> > data (for example when I want to add instructions to your assembler).
> 
> 
> 
> My guess is it would be more foolproof to edit that stuff with a spreadsheet.
> 
> 
> 
> Skip

Has anyone tried Pyspread?

http://manns.github.io/pyspread/

[toc] | [prev] | [next] | [standalone]

#51708

[Multipart message — attachments visible in raw view] — view raw

> Has anyone tried Pyspread?

I have not.

I have a fundamental problem with spreadsheets, the extremely narrow view
of the workspace. There was a piece on NPR the other day about some errors
in some modeling applications. I missed most of it (does someone have a
link? I'm on my phone right now), but the expert commentator was saying
that they are working on standard structures for these sorts of complex
modeling simulations. I think they will need more than that. Something like
pyspread might allow you to mix structured and object oriented programming
with the convenience if a spreadsheet.

Skip

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web