Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #54650 > unrolled thread

What's the best way to extract 2 values from a CSV file from each row systematically?

Back to article view | Back to comp.lang.python

Contents

  What's the best way to extract 2 values from a CSV file from each row systematically? quarantinemiles@gmail.com - 2013-09-23 10:10 -0700
    Re: What's the best way to extract 2 values from a CSV file from each row systematically? Neil Cerutti <neilc@norwich.edu> - 2013-09-23 17:20 +0000
    Re: What's the best way to extract 2 values from a CSV file from each row systematically? Joel Goldstick <joel.goldstick@gmail.com> - 2013-09-23 13:25 -0400
    Re: What's the best way to extract 2 values from a CSV file from each row systematically? Tim Chase <python.list@tim.thechases.com> - 2013-09-23 12:47 -0500
      Re: What's the best way to extract 2 values from a CSV file from each row systematically? Alex Lee <quarantinemiles@gmail.com> - 2013-09-24 07:04 -0700
        Re: What's the best way to extract 2 values from a CSV file from each row systematically? Roland Mueller <roland.em0001@googlemail.com> - 2013-09-27 12:22 +0300
    Re: What's the best way to extract 2 values from a CSV file from each row systematically? Luca Cerone <luca.cerone@gmail.com> - 2013-09-28 03:21 -0700

#54650 — What's the best way to extract 2 values from a CSV file from each row systematically?

Hey guys,

I'm a little new to Python, and am still learning!

I'm test building a web scraper that extracts prices from a website, based on two values I want to extract from a CSV file. The CSV has at least 1000 rows, an example:

0,0,KGD,0,DME,0,,0,0

The values I want to extract are KGD and DME (columns 3 and 5).

Each row in the CSV file contains values in columns 3 and 5 that I'd like to extract. What's the best way to extract these data, so I can insert them as inputs in two different fields in a form? A list, dictionary, or MySQL? I try not to do anything with MySQL as I'm not familiar with it at all.

I'm thinking of dictionary because at least I can make it work as a key/value pair (for columns 3 and 5), but a dictionary is unordered. I'd like to automatically go through each row in the CSV file from beginning to end to extract the two values in columns 3 and 5 and insert them into fields in a form.

I'd really appreciate any suggestions or help, thanks in advance!

[toc] | [next] | [standalone]

#54651

On 2013-09-23, quarantinemiles@gmail.com <quarantinemiles@gmail.com> wrote:
> Hey guys,
>
> I'm a little new to Python, and am still learning!
>
> I'm test building a web scraper that extracts prices from a
> website, based on two values I want to extract from a CSV file.
> The CSV has at least 1000 rows, an example:
>
> 0,0,KGD,0,DME,0,,0,0
>
> The values I want to extract are KGD and DME (columns 3 and 5).

Use the csv module.

http://docs.python.org/2/library/csv.html

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#54653

[Multipart message — attachments visible in raw view] — view raw

On Mon, Sep 23, 2013 at 1:10 PM, <quarantinemiles@gmail.com> wrote:

> Hey guys,
>
> I'm a little new to Python, and am still learning!
>
> I'm test building a web scraper that extracts prices from a website, based
> on two values I want to extract from a CSV file. The CSV has at least 1000
> rows, an example:
>
> 0,0,KGD,0,DME,0,,0,0
>
> The values I want to extract are KGD and DME (columns 3 and 5).
>
> Each row in the CSV file contains values in columns 3 and 5 that I'd like
> to extract. What's the best way to extract these data, so I can insert them
> as inputs in two different fields in a form? A list, dictionary, or MySQL?
> I try not to do anything with MySQL as I'm not familiar with it at all.
>
> I'm thinking of dictionary because at least I can make it work as a
> key/value pair (for columns 3 and 5), but a dictionary is unordered. I'd
> like to automatically go through each row in the CSV file from beginning to
> end to extract the two values in columns 3 and 5 and insert them into
> fields in a form.
>
> I'd really appreciate any suggestions or help, thanks in advance!
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

You should check out the csv module here:
http://docs.python.org/2/library/csv.html#module-csv
It will read your csv file into a list (the file rows) of lists (the
columns).  You can easily loop over the data to extract the columns you
want using list indexing

-- 
Joel Goldstick
http://joelgoldstick.com

[toc] | [prev] | [next] | [standalone]

#54656

On 2013-09-23 10:10, quarantinemiles@gmail.com wrote:
> based on two values I want to extract from a CSV file. The
> CSV has at least 1000 rows, an example:
> 
> 0,0,KGD,0,DME,0,,0,0
[snip]
> I'd like to automatically go through each row in the CSV file from
> beginning to end to extract the two values in columns 3 and 5 and
> insert them into fields in a form.

The csv module has several tools that make this easy to do.  If there
are column-headers, you can do

  import csv
  with file("myfile.csv", "rb") as f:
    for row in csv.DictReader(f)
      insert_fields_into_form(
        row["KGD"],
        row["DME"],
        )

which I like for clarity, ease of updating, and robustness (if for
some reason, the columns get moved around, or columns get
added/removed, as long as the headers remain the same, you can get
the data).

If it doesn't have headers, then you'd have to manually pick out the
columns, either by tuple-unpacking:

  with file("myfile.csv", "rb") as f:
    for _, _, kgd, _, dme in csv.reader(f)
      insert_fields_into_form(kgd, dme)

or by directly indexing:

  KGD_COL = 3
  DME_COL = 5
  with file("myfile.csv", "rb") as f:
    for row in csv.reader(f)
      insert_fields_into_form(
        row[KGD_COL],
        row[DME_COL],
        )

both of which are more fragile than DictReader when it comes to
columns being added/removed.  I leave the implementation of
insert_fields_into_form() up to you :-)

-tkc

[toc] | [prev] | [next] | [standalone]

#54699

Thanks for the help guys! I'll definitely read up on the csv module documentation.

Tim, that's incredibly helpful, thanks a lot! :) My CSV file doesn't have headers, but I'm sure I can just as easily add it in manually.

[toc] | [prev] | [next] | [standalone]

#54869

[Multipart message — attachments visible in raw view] — view raw

Hello,

2013/9/24 Alex Lee <quarantinemiles@gmail.com>

> Thanks for the help guys! I'll definitely read up on the csv module
> documentation.
>
> Tim, that's incredibly helpful, thanks a lot! :) My CSV file doesn't have
> headers, but I'm sure I can just as easily add it in manually.
>

a better way to provide column headers is to use the fieldname parameter
when creating the CSV reader. fieldnames is a string array that should
match the number of columns in the csv file:

*class *csv.DictReader(*csvfile*, *fieldnames=None*, *restkey=None*, *
restval=None*, *dialect='excel'*, **args*, ***kwds*)

BR,
Roland

--
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#54934

> I'd really appreciate any suggestions or help, thanks in advance!

Hi Alex if you know that you want only columns 3 and 5, you could also use list comprehension to fetch the values:

import csv

with open('yourfile.csv','rU') as fo: 
 #the rU means read using Universal newlines
 cr = csv.reader(fo)
 values_list = [(r[2],r[4]) for r in cr] #you have a list of tuples containing the values you need

Cheers,
Luca

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web