losing-end-of-row values when manipulating CSV input

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!selfless.tophat.at!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<nberg@atmos.ucla.edu>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.001
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'filename:fname piece:py': 0.04; 'advance,': 0.07; 'sanity': 0.07; 'script,': 0.07; 'subject:when': 0.07; '#print': 0.09; 'append': 0.09; 'csv': 0.09; 'issue?': 0.09; 'skip:\\ 30': 0.09; 'statement.': 0.09; 'trailing': 0.09; '"m"': 0.16; 'columns': 0.16; 'numpy': 0.16; 'row': 0.16; 'rows': 0.16; 'skip:# 60': 0.16; 'skip:# 70': 0.16; 'strangely': 0.16; 'subject:CSV': 0.16; 'subject:manipulating': 0.16; 'val': 0.16; 'script.': 0.19; 'column': 0.22; 'file,': 0.22; 'filter': 0.22; 'hourly': 0.23; 'appear': 0.23; 'extract': 0.25; 'received:169': 0.25; 'tried': 0.27; 'all,': 0.28; 'import': 0.29; 'fix': 0.29; 'second': 0.29; 'example': 0.30; 'values,': 0.30; 'thanks': 0.31; 'values': 0.31; 'print': 0.32; 'remaining': 0.32; 'list': 0.32; "i've": 0.33; 'to:addr:python-list': 0.34; 'there': 0.34; 'however,': 0.34; 'integer': 0.35; 'data,': 0.35; 'charset :us-ascii': 0.36; 'file': 0.36; 'data.': 0.36; 'issue': 0.37; 'date,': 0.37; 'some': 0.37; 'but': 0.37; 'several': 0.37; 'returning': 0.38; 'think': 0.38; 'two': 0.38; 'should': 0.39; 'header:Mime-Version:1': 0.39; 'data': 0.39; 'i.e.': 0.39; 'ways': 0.39; 'to:addr:python.org': 0.39; 'missing': 0.40; 'mark': 0.40; 'where': 0.40; 'third': 0.40; 'header:Message-Id:1': 0.61; 'maximum': 0.62; 'here': 0.66; 'lost': 0.73; 'exclude': 0.77; 'station': 0.82; 'purpose:': 0.84; 'temps,': 0.84; 'temps': 0.91; 'temperature': 0.93
From	Neil Berg <nberg@atmos.ucla.edu>
Content-Type	multipart/mixed; boundary=Apple-Mail-3--578610568
Subject	losing-end-of-row values when manipulating CSV input
Date	Wed, 13 Jul 2011 13:22:34 -0700
To	python-list@python.org
Mime-Version	1.0 (Apple Message framework v1082)
X-Mailer	Apple Mail (2.1082)
X-Probable-Spam	no
X-Spam-Report	none
X-Scanned-By	smtp.ucla.edu on 169.232.46.240
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.1000.1310588475.1164.python-list@python.org> (permalink)
Lines	92
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1310588476 news.xs4all.nl 23971 [2001:888:2000:d::a6]:57990
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:9433

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

Hello all,

I am having an issue with my attempts to accurately filter some data from a CSV file I am importing.  I have attached both a sample of the CSV data and my script.  

The attached CSV file contains two rows and 27 columns of data.  The first column is the station ID "BLS", the second column is the sensor number "4", the third column is the date, and the remaining 24 columns are hourly temperature readings. 

In my attached script, I read in row[3:] to extract just the temperatures, do a sanity check to make sure there are 24 values, remove any missing or "m" values, and then append the non-missing values into the "hour_list".  

Strangely the the first seven rows appear to be empty after reading into the CSV file, so that's what I had to incorporate the if len(temps) == 24 statement.  

But the real issue is that for days with no missing values, for example the second row of data, the length of the hour_list should be 24.  My script, however, is returning 23.  I think this is because the end-of-row-values have a trailing "\". This must mark these numbers as non-digits and are lost in my "isdig" filter line.  I've tried several ways to remove this trailing "\", but to no success. 

Do you have any suggestions on how to fix this issue?

Many thanks in advance,

Neil Berg

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

losing-end-of-row values when manipulating CSV input Neil Berg <nberg@atmos.ucla.edu> - 2011-07-13 13:22 -0700
  Re: losing-end-of-row values when manipulating CSV input Marco Nawijn <nawijn@gmail.com> - 2011-07-13 13:44 -0700
  Re: losing-end-of-row values when manipulating CSV input Marco Nawijn <nawijn@gmail.com> - 2011-07-13 13:42 -0700

csiph-web