Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.003 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'append': 0.09; 'received:209.85.219': 0.09; 'rows': 0.09; 'subject:files': 0.09; 'def': 0.12; "'b',": 0.16; "'e']": 0.16; "'rb')": 0.16; "['a',": 0.16; 'csv': 0.16; 'file1': 0.16; 'headers.': 0.16; 'subject:CSV': 0.16; 'subject:Merge': 0.16; 'subject:headers': 0.16; 'import': 0.22; 'to:name:python-list@python.org': 0.22; 'headers': 0.24; 'subject:/': 0.26; 'skip:p 30': 0.29; 'message- id:@mail.gmail.com': 0.30; 'file': 0.32; 'skip:d 20': 0.34; 'subject:with': 0.35; 'received:209.85': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; '8bit%:9': 0.36; 'received:209': 0.37; 'skip:o 20': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'skip:n 10': 0.64; 'different': 0.65 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=JNCCNPPzHFDtWem317OOrrxIy5L7C8XFVmeqtKiV3Zk=; b=Mg3gESqPx+5xIF3vnnZLNZMayTWj8yjAHWIPKYoRYftiVhM3/hFl2oNoouufWEnmJx 60kCDMPL2Oavm4wDoiYepU8f2/eCYeCmp+g88maJkXILE6I4vov0cR8lrV72KPizsFRl 4ogA0ogDR1VJEdNbYGkFluVSa7cEhXi6WeWXFWQ2xKKBim1j1iX1y1kLspGybPSdcQrA ujFuE/GJMqnIBvJbNv93FknZLHlVYor9eAwFDLaDMDAkeWo8yqvtvBmSg4jvr6Aghw8v 4L5Qs7Z2DULpuLG27gsrA4lMxRhcrXGPyb94YkMXRHQOxVgcdO+MmKwmY0Ms8YrktAgw Gg+A== X-Gm-Message-State: ALoCoQmDEx4IS1vvDwJ7emhC9kris3A9Obs8rHJ1xzv/8cuhwwi7S2MT8F43NoLwBRwJZY2a35eT X-Received: by 10.60.172.70 with SMTP id ba6mr57665456oec.17.1395683451030; Mon, 24 Mar 2014 10:50:51 -0700 (PDT) MIME-Version: 1.0 From: Vincent Davis Date: Mon, 24 Mar 2014 11:50:07 -0600 Subject: Merge/append CSV files with different headers To: "python-list@python.org" Content-Type: multipart/alternative; boundary=bcaec54b53665291d904f55de0c2 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 105 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1395683453 news.xs4all.nl 2960 [2001:888:2000:d::a6]:45914 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:68889 --bcaec54b53665291d904f55de0c2 Content-Type: text/plain; charset=UTF-8 I have several csv file I need to append (vertically). They have different but overlapping headers. For example; file1 headers ['a', 'b', 'c'] file2 headers ['d', 'e'] file3 headers ['c', 'd'] Is there a better way than this import csv def merge_csv(fileList, newFileName): allHeaders = set([]) for afile in fileList: with open(afile, 'rb') as csvfilesin: eachheader = csv.reader(csvfilesin, delimiter=',').next() allHeaders.update(eachheader) print(allHeaders) with open(newFileName, 'wb') as csvfileout: outfile = csv.DictWriter(csvfileout, allHeaders) outfile.writeheader() for afile in fileList: print('***'+afile) with open(afile, 'rb') as csvfilesin: rows = csv.DictReader(csvfilesin, delimiter=',') for r in rows: print(allHeaders.issuperset(r.keys())) outfile.writerow(r) Vincent Davis --bcaec54b53665291d904f55de0c2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I have several csv file I need to append (verti= cally). They have different but overlapping headers. For example;
file1 headers ['a', 'b', 'c']
file2= headers ['d', 'e']
file3 headers ['c', 'd']

Is there a better way than this
import csv
def merge_csv(fileList, newFileName):<= /div>
=C2=A0 =C2=A0 allHeaders =3D set([])
= =C2=A0 =C2=A0 for afile in fileList:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 with open= (afile, 'rb') as csvfilesin:
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 eachheader =3D csv.reader(csvfilesin, delim= iter=3D',').next()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 allHe= aders.update(eachheader)
=C2=A0 =C2= =A0 print(allHeaders)
=C2=A0 =C2=A0 with open(newFileName, 'wb')= as csvfileout:
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 outfile =3D csv.DictWriter(csvfileout, allHeaders)=
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 outfile.writeheader()
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 for afile in fileList:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 print('***'+afile)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 with open(afile, 'rb') as csvfilesin:
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 rows =3D csv.DictReader(csvfilesin, delimit= er=3D',')
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 for r in rows:
=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 print(allHeaders.i= ssuperset(r.keys()))
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 outfile.writero= w(r)

Vincent Davis

--bcaec54b53665291d904f55de0c2--