Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #16004 > unrolled thread

How to: Coordinate DictReader and Reader for CSV

Back to article view | Back to comp.lang.python

Contents

  How to:  Coordinate DictReader and Reader for CSV ray <ray@aarden.us> - 2011-11-21 05:18 -0800
    Re: How to:  Coordinate DictReader and Reader for CSV Neil Cerutti <neilc@norwich.edu> - 2011-11-21 13:59 +0000
      Re: How to: Coordinate DictReader and Reader for CSV ray <ray@aarden.us> - 2011-11-21 07:16 -0800
        Re: How to: Coordinate DictReader and Reader for CSV Neil Cerutti <neilc@norwich.edu> - 2011-11-21 15:41 +0000
          Re: How to: Coordinate DictReader and Reader for CSV Neil Cerutti <neilc@norwich.edu> - 2011-11-21 15:43 +0000
        Re: How to: Coordinate DictReader and Reader for CSV Tim Chase <python.list@tim.thechases.com> - 2011-11-21 12:36 -0600

#16004 — How to: Coordinate DictReader and Reader for CSV

I am trying to get the data from a CSV file into variables.  I have
used DictReader to get the field names and I can report them.  When I
attempt to look at the data, every row shows the combination of
fieldname:data.  How do I get the data out?
linelist=open( "C:/Users/rjoseph/Documents/Projects/Bootstrap Plan
Design Tool/Sandbox/line_list_r0a.csv", "rb" )
csvDictReader=csv.DictReader( linelist, dialect='excel' )
for data in csvReader:
        print data[0]
        print data[1]
        print data[2]
        print data[3]
linelist.close()

Thanks,
ray

[toc] | [next] | [standalone]

#16006

On 2011-11-21, ray <ray@aarden.us> wrote:
> I am trying to get the data from a CSV file into variables.  I have
> used DictReader to get the field names and I can report them.  When I
> attempt to look at the data, every row shows the combination of
> fieldname:data.  How do I get the data out?
> linelist=open( "C:/Users/rjoseph/Documents/Projects/Bootstrap Plan
> Design Tool/Sandbox/line_list_r0a.csv", "rb" )
> csvDictReader=csv.DictReader( linelist, dialect='excel' )
> for data in csvReader:
>         print data[0]
>         print data[1]
>         print data[2]
>         print data[3]
> linelist.close()

The elements yielded by a DictReader iterator are dictionaries,
and the keys are the headings of the csv file. So replace those
integers with strings representing the headings of your file.

If the headings are actually those numbers, you want:

[...]
    print data['0']
    print data['1']
    print data['2']
    print data['3']
    print data['4']
[...]

-- 
Neil Cerutti
  "This room is an illusion and is a trap devisut by Satan.  Go
ahead and dauntlessly!  Make rapid progres!"
  --Ghosts 'n Goblins

[toc] | [prev] | [next] | [standalone]

#16014 — Re: How to: Coordinate DictReader and Reader for CSV

On Nov 21, 7:59 am, Neil Cerutti <ne...@norwich.edu> wrote:
> On 2011-11-21, ray <r...@aarden.us> wrote:
>
> > I am trying to get the data from a CSV file into variables.  I have
> > used DictReader to get the field names and I can report them.  When I
> > attempt to look at the data, every row shows the combination of
> > fieldname:data.  How do I get the data out?

> > linelist=open( "C:/Users/thisuser/Documents/Projects/Bootstrap Plan
> > Design Tool/Sandbox/line_list_r0a.csv", "rb" )
> > csvDictReader=csv.DictReader( linelist, dialect='excel' )
> > for data in csvReader:
> >         print data[0]
> >         print data[1]
> >         print data[2]
> >         print data[3]
> > linelist.close()
>
> The elements yielded by a DictReader iterator are dictionaries,
> and the keys are the headings of the csv file. So replace those
> integers with strings representing the headings of your file.
>
> If the headings are actually those numbers, you want:
>
> [...]
>     print data['0']
>     print data['1']
>     print data['2']
>     print data['3']
>     print data['4']
> [...]
>
> --
> Neil Cerutti
>   "This room is an illusion and is a trap devisut by Satan.  Go
> ahead and dauntlessly!  Make rapid progres!"
>   --Ghosts 'n Goblins

Neil,

Thank you for your efforts.

When I use the 0, 1, etc. in the data[x] slot, I get some data.  When
I put a string in, I get an error stating:
TypeError: list indices must be integers, not str

But your suggestion has helped my better understand my problem. The
output is first a list of the keys and then the associated data.  The
difficulty is that I want to pass the data to another function will I
am in the 'for' loop.  But the first data out is keys and that is not
the data I want to send to the other function.  Is there a way to
capture the keys outside of the for loop so when the for loop is
entered, only data is extracted?

Thanks,
ray

[toc] | [prev] | [next] | [standalone]

#16016 — Re: How to: Coordinate DictReader and Reader for CSV

On 2011-11-21, ray <ray@aarden.us> wrote:
> Is there a way to capture the keys outside of the for loop so
> when the for loop is entered, only data is extracted?

I have sometimes done the following type of thing, since
DictReader doesn't offer an attribute providing the field names.
This is Python 3.3.2 code, so revise boilerplate if necessary.

# Open once as a csv.reader instance to get the field names, in
# order.
with open(in_file_name, newline='') as in_file:
    reader = csv.reader(in_file)
    fields = next(reader)

# Open it again as a csv.DictReader instance to do actual work,
# writing revised lines to the output file as I go.
with open(in_file_name, newline=') as in_file:
  with open(out_file_name, "w", newline='') as out_file:
    reader = csv.DictReader(in_file)
    writer = csv.DictWriter(out_file, fieldnames=fields)
    # Write header line
    writer.writerow({f: f for n in fields})
    for record in reader:
        # Change a few fields
	# [...]
	writer.writerow(record)

-- 
Neil Cerutti
  "This room is an illusion and is a trap devisut by Satan.  Go
ahead and dauntlessly!  Make rapid progres!"
  --Ghosts 'n Goblins

[toc] | [prev] | [next] | [standalone]

#16017 — Re: How to: Coordinate DictReader and Reader for CSV

On 2011-11-21, Neil Cerutti <neilc@norwich.edu> wrote:
> On 2011-11-21, ray <ray@aarden.us> wrote:
>> Is there a way to capture the keys outside of the for loop so
>> when the for loop is entered, only data is extracted?
>
> I have sometimes done the following type of thing, since
> DictReader doesn't offer an attribute providing the field names.
> This is Python 3.3.2 code, so revise boilerplate if necessary.
>
> # Open once as a csv.reader instance to get the field names, in
> # order.
> with open(in_file_name, newline='') as in_file:
>     reader = csv.reader(in_file)
>     fields = next(reader)

Equal to reader.next() in 2.x Python, I believe.

> # Open it again as a csv.DictReader instance to do actual work,
> # writing revised lines to the output file as I go.
> with open(in_file_name, newline=') as in_file:
>   with open(out_file_name, "w", newline='') as out_file:
>     reader = csv.DictReader(in_file)
>     writer = csv.DictWriter(out_file, fieldnames=fields)
>     # Write header line
>     writer.writerow({f: f for n in fields})

Oops! {f: f for f in fields}. Sorry about that.

>     for record in reader:
>         # Change a few fields
> 	# [...]
> 	writer.writerow(record)


-- 
Neil Cerutti
  "This room is an illusion and is a trap devisut by Satan.  Go
ahead and dauntlessly!  Make rapid progres!"
  --Ghosts 'n Goblins

[toc] | [prev] | [next] | [standalone]

#16043 — Re: How to: Coordinate DictReader and Reader for CSV

On 11/21/11 09:16, ray wrote:
> Is there a way to capture the keys outside of the for loop so
> when the for loop is entered, only data is extracted?

I frequently do this for things like tweaking headers (stripping 
space, normalizing case, etc because clients love to send us 
messy data):

   def norm_header(h):
     return h.strip().upper()
   def norm_item(i):
     return i.strip()

   f = file("example.csv", "rb")
   try:
     r = csv.reader(f)
     headers = r.next()
     header_map = dict(
       (norm_header(h), i)
       for i, h in enumerate(headers)
       )
     for row in r:
       item = lambda h: norm_item(row[header_map[norm_header(h)]])
       value1 = item("Item1")
       value2 = item("ITEM3")
       ...
   finally:
     f.close()

Should work in 2.x, possibly in 3.x (though you might need to 
change from "headers = r.next()" to "headers = next(r)")

-tkc

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web