Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #68889

Merge/append CSV files with different headers

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

I have several csv file I need to append (vertically). They have different
but overlapping headers. For example;
file1 headers ['a', 'b', 'c']
file2 headers ['d', 'e']
file3 headers ['c', 'd']

Is there a better way than this
import csv
def merge_csv(fileList, newFileName):
    allHeaders = set([])
    for afile in fileList:
        with open(afile, 'rb') as csvfilesin:
            eachheader = csv.reader(csvfilesin, delimiter=',').next()
            allHeaders.update(eachheader)
    print(allHeaders)
    with open(newFileName, 'wb') as csvfileout:
        outfile = csv.DictWriter(csvfileout, allHeaders)
        outfile.writeheader()
        for afile in fileList:
            print('***'+afile)
            with open(afile, 'rb') as csvfilesin:
                rows = csv.DictReader(csvfilesin, delimiter=',')
                for r in rows:
                    print(allHeaders.issuperset(r.keys()))
                    outfile.writerow(r)

Vincent Davis

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Merge/append CSV files with different headers Vincent Davis <vincent@vincentdavis.net> - 2014-03-24 11:50 -0600

csiph-web