Path: csiph.com!usenet.pasdenom.info!nntpfeed.proxad.net!news.muarf.org!news.roellig-ltd.de!open-news-network.org!border2.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Wed, 13 May 2015 17:06:55 -0700
From: Ethan Furman <ethan@stoneleaf.us>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Looking for direction
References: <5553DD2E.2080600@pacbell.net>
In-Reply-To: <5553DD2E.2080600@pacbell.net>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.470.1431562589.12865.python-list@python.org>
Lines: 47
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:90594

On 05/13/2015 04:24 PM, 20/20 Lab wrote:
> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.
>
> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
>
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.
>
> Now is were I have my problem:
>
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.  The
> number of rows change based on which run it is, so I couldnt even get
> away with using hundreds of compare loops.
>
> If someone could point me to some documentation on the functions I would
> need, or a tutorial it would be a great help.

You could try using a dictionary, combining when needed:

# untested
data = {}
for row in all_rows:
   key = row[0], row[1]
   if key in data:
     item, qty, noise = data[key]
     qty += row[3]
   else:
     item, qty, noise = row[2:]
   data[key] = item, qty, noise

for (account, staff), (item, qty, noise) in data.items():
   do_stuff_with(account, staff, item, qty, noise)

At the end, data should have what you want.  It won't, however, be in 
the same order, so hopefully that's not an issue for you.

--
~Ethan~