Path: csiph.com!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed8.news.xs4all.nl!nzpost1.xs4all.net!not-for-mail
To: python-list@python.org
From: Jaydip Chakrabarty <chalao.adda@gmail.com>
Subject: Re: Finding Blank Columns in CSV
Date: Tue, 6 Oct 2015 11:24:33 +0000 (UTC)
References: <muttuu$pc5$1@ger.gmane.org> <CAPTjJmrdE-n=VazddU7PZS6E=rbBf1XVoz_g2ArxK6454D5qxw@mail.gmail.com> <CAPTjJmqYwD0=KNLaHu_w_sOjUoacQRXFhb4Zxdv6n++5A-Y8iQ@mail.gmail.com> <20151005090652.1c9faed7@bigbox.christie.dr> <CAPTjJmpv1OOEomjjaqQ5qiGQDHpxAR4RpxLxn1Rt+j=TQBz4Yg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
User-Agent: Pan/0.139 (Sexual Chocolate; GIT bf56508 git://git.gnome.org/pan2)
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.418.1444130689.28679.python-list@python.org>
Lines: 43
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:97440

On Tue, 06 Oct 2015 01:34:17 +1100, Chris Angelico wrote:

> On Tue, Oct 6, 2015 at 1:06 AM, Tim Chase
> <python.list@tim.thechases.com> wrote:
>> That way, if you determine by line 3 that your million-row CSV file has
>> no blank columns, you can get away with not processing all million
>> rows.
> 
> Sure, although that effectively means the entire job is moot. I kinda
> assume that the OP knows that there are some blank columns (maybe lots
> of them). The extra check is unnecessary unless it's actually plausible
> that there'll be no blanks whatsoever.
> 
> Incidentally, you have an ordered_headers list which is the blank
> columns in order; I think the OP was looking for a list of the
> _non_blank columns. But that's a trivial difference, easy to tweak.
> 
> ChrisA

Thanks to you all. I got it this far. But while writing back to another 
csv file, I got this error - "ValueError: dict contains fields not in 
fieldnames: None". Here is my code.

rdr = csv.DictReader(fin, delimiter=',')
header_set = set(rdr.fieldnames)
for r in rdr:
    header_set = set(h for h in header_set if not r[h])
    if not header_set:
        break

for r in rdr:
    data = list(r[i] for i in header_set)

dw = csv.DictWriter(fout, header_set)
dw.writeheader()
dw.writerows(data)

Also, there is difference between len(header_set) and len(data[0].keys). 
Why is so?
Thanks again for all your help.

Thanks.