Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #88361 > unrolled thread

Using csv DictWriter - add a extra field

Back to article view | Back to comp.lang.python

Contents

  Using csv DictWriter - add a extra field Victor Hooi <victorhooi@gmail.com> - 2015-03-30 21:47 -0700
    Re: Using csv DictWriter - add a extra field Peter Otten <__peter__@web.de> - 2015-03-31 10:22 +0200
      Re: Using csv DictWriter - add a extra field Victor Hooi <victorhooi@gmail.com> - 2015-03-31 01:41 -0700
        Re: Using csv DictWriter - add a extra field Peter Otten <__peter__@web.de> - 2015-03-31 11:03 +0200

#88361 — Using csv DictWriter - add a extra field

Hi,

I have a dict named "connections", with items like the following:

In [18]: connections
Out[18]:
{'3424234': {'end_timestamp': datetime.datetime(2015, 3, 25, 5, 31, 30, 406000, tzinfo=datetime.timezone(datetime.timedelta(-1, 61200))),
  'ip_address': '10.168.8.36:52440',
  'open_timestamp': datetime.datetime(2015, 3, 25, 5, 31, 0, 383000, tzinfo=datetime.timezone(datetime.timedelta(-1, 61200))),
  'time_open': datetime.timedelta(0, 30, 23000)}}

In this case, the key is a connection id (e.g. "3424234"), and the value is a another dict, which contains things like 'end_timestamp', 'ip_address" etc.

I'm writing the output of "connections" to a CSV file using DictWriter:

fieldnames = ['connection_id', 'ip_address', 'open_timestamp', 'end_timestamp', 'time_open']
with open('output.csv', 'w') as csvfile:
    writer = DictWriter(csvfile, fieldnames)
    writer.writeheader()
    for connection, values in sorted(connections.items()):
        if 'time_open' in values:
            writer.writerow(values, {'connection_id': connection})
        else:
            pass
            # DO SOME STUFF

The only problem is, I'd also like output the connection_id field as part of each CSV record.

However, connection_id in this case is the key for the parent dict.

Is there a clean way to add a extra field to DictWriter writerows, or it is the contents of the dict and that's it?

Cheers,
Victor

[toc] | [next] | [standalone]

#88369

Victor Hooi wrote:

> Hi,
> 
> I have a dict named "connections", with items like the following:
> 
> In [18]: connections
> Out[18]:
> {'3424234': {'end_timestamp': datetime.datetime(2015, 3, 25, 5, 31, 30,
> {406000, tzinfo=datetime.timezone(datetime.timedelta(-1, 61200))),
>   'ip_address': '10.168.8.36:52440',
>   'open_timestamp': datetime.datetime(2015, 3, 25, 5, 31, 0, 383000,
>   tzinfo=datetime.timezone(datetime.timedelta(-1, 61200))), 'time_open':
>   datetime.timedelta(0, 30, 23000)}}
> 
> In this case, the key is a connection id (e.g. "3424234"), and the value
> is a another dict, which contains things like 'end_timestamp',
> 'ip_address" etc.
> 
> I'm writing the output of "connections" to a CSV file using DictWriter:
> 
> fieldnames = ['connection_id', 'ip_address', 'open_timestamp',
> 'end_timestamp', 'time_open'] with open('output.csv', 'w') as csvfile:
>     writer = DictWriter(csvfile, fieldnames)
>     writer.writeheader()
>     for connection, values in sorted(connections.items()):
>         if 'time_open' in values:
>             writer.writerow(values, {'connection_id': connection})
>         else:
>             pass
>             # DO SOME STUFF
> 
> The only problem is, I'd also like output the connection_id field as part
> of each CSV record.
> 
> However, connection_id in this case is the key for the parent dict.
> 
> Is there a clean way to add a extra field to DictWriter writerows, or it
> is the contents of the dict and that's it?

The latter. The obvious solution is to add the extra field to the dict:

values['connection_id'] = connection
writer.writerow(values)

Or you use a collections.ChainMap (Python 3 only I think, but 
<http://code.activestate.com/recipes/305268/> should work as well here)

writer.writerow(
    collections.ChainMap(values, {"connection_id": connection}))

[toc] | [prev] | [next] | [standalone]

#88370

Hi,

Aha, yeah, I can add the connection_id as another field in the inner dict - the only drawback is that the data is duplicated twice. However, I suppose even if it's not elegant, it does work.

However, that ChainMap does look interesting =). And yes, I am actually using Python 3.x (mainly because of http://bugs.python.org/issue6641).

So if I understand correctly, I can just use ChainMap to join any arbitrary number of dicts together - it seems like the right solution here.

Are there any drawbacks to using ChainMap here? (Aside from needing Python 3.x).

Cheers,
Victor

[toc] | [prev] | [next] | [standalone]

#88371

Victor Hooi wrote:

> Aha, yeah, I can add the connection_id as another field in the inner dict
> - the only drawback is that the data is duplicated twice. However, I
> suppose even if it's not elegant, it does work.

The elegance lies in its simplicity, so it's still my personal favourite.

> However, that ChainMap does look interesting =). And yes, I am actually
> using Python 3.x (mainly because of http://bugs.python.org/issue6641).
> 
> So if I understand correctly, I can just use ChainMap to join any
> arbitrary number of dicts together - it seems like the right solution
> here.
> 
> Are there any drawbacks to using ChainMap here? (Aside from needing Python
> 3.x).

As it introduces another level of indirection it costs a little time, but 
compared to the actual I/O that should be negligible. 

So no, if there is a drawback I don't see it.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web