Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90594

Re: Looking for direction

Show all headers | View raw

On 05/13/2015 04:24 PM, 20/20 Lab wrote:
> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.
>
> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
>
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.
>
> Now is were I have my problem:
>
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.  The
> number of rows change based on which run it is, so I couldnt even get
> away with using hundreds of compare loops.
>
> If someone could point me to some documentation on the functions I would
> need, or a tutorial it would be a great help.

You could try using a dictionary, combining when needed:

# untested
data = {}
for row in all_rows:
   key = row[0], row[1]
   if key in data:
     item, qty, noise = data[key]
     qty += row[3]
   else:
     item, qty, noise = row[2:]
   data[key] = item, qty, noise

for (account, staff), (item, qty, noise) in data.items():
   do_stuff_with(account, staff, item, qty, noise)

At the end, data should have what you want.  It won't, however, be in 
the same order, so hopefully that's not an issue for you.

--
~Ethan~

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Looking for direction Ethan Furman <ethan@stoneleaf.us> - 2015-05-13 17:06 -0700

csiph-web