Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #54405

Re: *.csv to *.txt after adding columns

Show all headers | View raw

On Tue, 17 Sep 2013 18:42:21 -0700 (PDT), Bryan Britten
<britten.bryan@gmail.com> declaimed the following:

>Hey, gang, I've got a problem here that I'm sure a handful of you will know how to solve. I've got about 6 *.csv files that I am trying to open; change the header names (to get rid of spaces); add two new columns, which are just the results of a string.split() command; drop the column I just split; and then finally export to *.txt files. Here's the code I'm using:
>
>import os
>import csv
>
>
>fileHandle = 'Path/To/Data'

	That seems a misnomer... It is not a "file handle" of any sort, just a
base path.

>varNames = 'ID\tCaseNum\tDate\tTime\tBlock\tIUCR\tPrimaryType\tDescription\tLocDesc\tArrest\tDomestic\tBeat\tDistrict\tWard\tCommArea\tFBICode\tXCoord\tYCoord\tYear\tUpdatedOn\tLat\tLong\tLoc\n'
>
>for csvFile in os.listdir(fileHandle):
>    outFile = open(fileHandle + os.path.splitext(csvFile)[0] + '.txt', 'w')

	Note that your "fileHandle" does not end with a / -- so I have no idea
what types of names you are trying to open...

	Recommend you use

	os.path.join(fileHande, os.path.splitext(csvFile)[0] + ".txt")


>    inFile = open(fileHandle + csvFile, 'rb')

	ditto

>    reader = csv.reader(inFile, delimiter=',')
>    rowNum = 0
>    for row in reader:
>        if rowNum == 0:
>            outFile.write(varNames)
>            rowNum += 1
>        else:
>            date, time = row[2].split()
>            row.insert(3, date)
>            row.insert(4, time)
>            row.remove(row[2])
>            outFile.write('\t'.join(row) + '\n')

	Apparently your "text file" is a tab-separated file...

	The CSV module is capable of processing TSV just as easily as CSV.

	I'd also drop the whole confusion of rowNum


-=-=-=-=- pseudo-code, not runnable

	fin = open(os.path.join(basepath, filename), "rb")
	csvin = csv.reader(fin, delimiter=",")
	fout = open(os.path.join(basepath, otherfilename), "wb")
	csvout = csv.writer(fout, delimiter="\t")

	junk = reader.next()	#skip header
	csvout.write(["ID", "caseNum", ... ])

	for row in reader:
		out = row[:2] 
		out.extend(row[2].split())
		out.extend(row[3:])
		csvout.write(row)

	fout.close()
	fin.close()
	del csvout	#just paranoia, free up structures
	del csvin	#ditto

-=-=-=-=-

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

*.csv to *.txt after adding columns Bryan Britten <britten.bryan@gmail.com> - 2013-09-17 18:42 -0700
  Re: *.csv to *.txt after adding columns Dave Angel <davea@davea.name> - 2013-09-18 02:18 +0000
  Re: *.csv to *.txt after adding columns Bryan Britten <britten.bryan@gmail.com> - 2013-09-17 19:28 -0700
    Re: *.csv to *.txt after adding columns Dave Angel <davea@davea.name> - 2013-09-18 07:55 +0000
  Re: *.csv to *.txt after adding columns Peter Otten <__peter__@web.de> - 2013-09-18 09:14 +0200
  Re: *.csv to *.txt after adding columns rusi <rustompmody@gmail.com> - 2013-09-18 00:44 -0700
  Re: *.csv to *.txt after adding columns Bryan Britten <britten.bryan@gmail.com> - 2013-09-18 04:42 -0700
  Re: *.csv to *.txt after adding columns Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-09-18 18:47 -0400

csiph-web