Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #41548 > unrolled thread

Grouping on and exporting to csv files

Back to article view | Back to comp.lang.python

Contents

  Grouping on  and exporting to csv files accessnewbie@gmail.com - 2013-03-19 17:34 -0700
    Re: Grouping on  and exporting to csv files Peter Otten <__peter__@web.de> - 2013-03-20 08:21 +0100

#41548 — Grouping on and exporting to csv files

My current Test_Sort.csv looks like this (which constantly changes-although never more than 4 records, thus far anyway):

RecNo,Count,District,Fruit,StoreNo
1,100,0,oranges,1254
2,30,3,apples,654
3,100,0,bananas,21
4,0,4,grapes,478

I wrote the following python 2.7 script (stuck with 2.7)

---------------------
import sys
import csv
import operator


source = "E:\\Data\\Test_Sort.csv"
result = "E:\\Data\\Test_Sort1.csv"

with open(source,"rb") as source:
    rdr= csv.reader( source )
    with open(result,"wb") as result:
        next(rdr, None) # Skip header
        wtr= csv.writer( result )
        for r in rdr:
            wtr.writerow( (r[2], r[3])) # Only write columns 3 and 4 to a csv file
    exit
exit

data = csv.reader(open("E:\\Data\\Test_Sort1.csv"),delimiter=',')
sortedlist = sorted(data, key=operator.itemgetter(0,1)) # sort by column 1, the 2
print sortedlist

-----------------------------

My sortedlist results are:

[['0', 'bananas'], ['0', 'oranges'], ['3', 'apples'], ['4', 'grapes']]

I want to output all fruits with the same District number out to a csv file called by its District number. (No headers)

Thus in above example I would ultimately have three csv files:

0.csv
bananas
oranges

3.csv
apples

4.csv
grapes

Question is : Any ideas as to how export this sortedlist to the tables described above?

I already have a script that already will look at each file and count the number of lines and insert the correct number of commas and the word "and" where appropriate in a sentence.

My ultimate goal is to read each file and print out a sentence based on whether it has record or not, and if so how many records and then add the word "and" and/or commas between records if greater than 1. I will then concatenate all the sentences together to form a paragraph.

Thus, the above example would say: 

The western district sold bananas and oranges. The central district sold apples. The southern district sold grapes.

or if the files were:

0.csv
apples
bananas
oranges

2.csv
grapes

The resulting paragraph would read:

The western district sold apples, bananas, and oranges. The eastern district sold grapes.

It may seem like a crazy way about getting my results but it was the best I could come up with given my limited knowledge of python.

Thanks for any and all help.

[toc] | [next] | [standalone]

#41573

accessnewbie@gmail.com wrote:

> My current Test_Sort.csv looks like this (which constantly
> changes-although never more than 4 records, thus far anyway):
> 
> RecNo,Count,District,Fruit,StoreNo
> 1,100,0,oranges,1254
> 2,30,3,apples,654
> 3,100,0,bananas,21
> 4,0,4,grapes,478
> 
> I wrote the following python 2.7 script (stuck with 2.7)
> 
> ---------------------
> import sys
> import csv
> import operator
> 
> 
> source = "E:\\Data\\Test_Sort.csv"
> result = "E:\\Data\\Test_Sort1.csv"
> 
> with open(source,"rb") as source:
>     rdr= csv.reader( source )
>     with open(result,"wb") as result:
>         next(rdr, None) # Skip header
>         wtr= csv.writer( result )
>         for r in rdr:
>             wtr.writerow( (r[2], r[3])) # Only write columns 3 and 4 to a
>             csv file
>     exit
> exit
> 
> data = csv.reader(open("E:\\Data\\Test_Sort1.csv"),delimiter=',')
> sortedlist = sorted(data, key=operator.itemgetter(0,1)) # sort by column
> 1, the 2 print sortedlist
> 
> -----------------------------
> 
> My sortedlist results are:
> 
> [['0', 'bananas'], ['0', 'oranges'], ['3', 'apples'], ['4', 'grapes']]
> 
> I want to output all fruits with the same District number out to a csv
> file called by its District number. (No headers)
> 
> Thus in above example I would ultimately have three csv files:
> 
> 0.csv
> bananas
> oranges
> 
> 3.csv
> apples
> 
> 4.csv
> grapes
> 
> Question is : Any ideas as to how export this sortedlist to the tables
> described above?

The standard library offers itertools.groupby()

for district, group in itertools.groupby(
        sortedlist, key=operator.itemgetter(0)):
    with open(district + ".csv", "wb") as outfile:
        csv.writer(outfile).writerows(group)

If you want try to do it by hand as an exercise -- you need a variable to 
keep track of the current district:

last_district = None
for row in sortedlist:
    district = row[0]
    if last_district != district:
        # open new file/writer
        last_district = district
    # write row

> I already have a script that already will look at each file and count the
> number of lines and insert the correct number of commas and the word "and"
> where appropriate in a sentence.
> 
> My ultimate goal is to read each file and print out a sentence based on
> whether it has record or not, and if so how many records and then add the
> word "and" and/or commas between records if greater than 1. I will then
> concatenate all the sentences together to form a paragraph.
> 
> Thus, the above example would say:
> 
> The western district sold bananas and oranges. The central district sold
> apples. The southern district sold grapes.

Have a look at str.join():

 >>> colors = ["red", "yellow", "blue"]
>>> print "Who's afraid of {}, and {}?".format(", ".join(colors[:-1]), 
colors[-1])
Who's afraid of red, yellow, and blue?

> or if the files were:
> 
> 0.csv
> apples
> bananas
> oranges
> 
> 2.csv
> grapes
> 
> The resulting paragraph would read:
> 
> The western district sold apples, bananas, and oranges. The eastern
> district sold grapes.
> 
> It may seem like a crazy way about getting my results but it was the best
> I could come up with given my limited knowledge of python.

No, it is what experienced programmers do all the time -- break a complex 
task into manageable steps, come up with a simple solution for each step, 
and then improve these steps as needed.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web