Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!weretis.net!feeder4.news.weretis.net!cs.uu.nl!news.stack.nl!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.005 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python,': 0.02; 'programmer': 0.03; 'tree': 0.05; 'args': 0.07; 'parser': 0.07; 'string': 0.09; '"__main__":': 0.09; '__name__': 0.09; 'ascii': 0.09; 'python': 0.11; '###': 0.16; "'%s'": 0.16; "'size'": 0.16; 'dictionary.': 0.16; 'hostname': 0.16; 'pythonic': 0.16; 'size,': 0.16; 'sender:addr:gmail.com': 0.17; 'thanks,': 0.17; '<': 0.19; 'file,': 0.19; 'normally': 0.19; "skip:' 30": 0.19; 'skip:p 40': 0.19; 'import': 0.22; 'print': 0.22; 'file.': 0.24; 'message- id:@mail.gmail.com': 0.30; "i'm": 0.30; 'code': 0.31; 'lines': 0.31; 'url:wiki': 0.31; 'keyerror:': 0.31; 'own,': 0.31; 'perl': 0.31; 'url:wikipedia': 0.31; 'file': 0.32; 'this.': 0.32; 'regular': 0.32; '(most': 0.33; 'skip:# 10': 0.33; "i'd": 0.34; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'add': 0.35; 'there': 0.35; '8bit%:9': 0.36; 'subject:data': 0.36; 'url:org': 0.36; 'skip:& 10': 0.38; 'to:addr:python-list': 0.38; 'files': 0.38; 'recent': 0.39; 'skip:& 20': 0.39; 'structure': 0.39; 'to:addr:python.org': 0.39; 'skip:p 20': 0.39; 'how': 0.40; 'read': 0.60; 'most': 0.60; 'information': 0.63; 'field': 0.63; 'our': 0.64; '8bit%:10': 0.64; 'world': 0.66; 'here': 0.66; 'header:Reply-To:1': 0.67; 'default': 0.69; 'reply-to:no real name:2**0': 0.71; 'skip:a 40': 0.72; '100': 0.79; 'do:': 0.91; 'visitors.': 0.91; '***': 0.95 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:date:message-id:subject:from:to :content-type; bh=rTQ3LxnmUp6d9obYi719DY6HEzsAwl6OSqpmkJPoDCg=; b=osLqgfEwpVUgr71OGNZ3NZTj0gHrjRO8jasoj+WZfcjMm9W39//kI/HvmSH6dyPPqA eSvDvghJ294XWYtRJqz5P/OOgmk1k27S9C+a5pXLV25XhP1FujZybuCoOEc27jNhI/6+ x1FPcarrjD6WMsaf4DyMrysPwB5LzBO0EFVX5MmVWdPK6MXuWXKp9U94QDLS31pjy9e7 HlHnXfuaEVN9Ycwk2fhoDsl5YsZ5qc6TZzuGNeUfzROFj5J+mvKXm8gD/rlpIdupToz3 MyQjRsMlNZKFeP+8qvURMQwJsklfOhKRIb6rkJPEACvfgX9wAjko6+hrfAgRKbTM3CD+ U9zg== MIME-Version: 1.0 X-Received: by 10.180.87.161 with SMTP id az1mr62557675wib.13.1411949060980; Sun, 28 Sep 2014 17:04:20 -0700 (PDT) Sender: extasia@gmail.com Date: Sun, 28 Sep 2014 17:04:20 -0700 X-Google-Sender-Auth: _NR0H5h37xTKn9bFfBtfUo6I804 Subject: trouble building data structure From: David Alban To: python-list@python.org Content-Type: multipart/alternative; boundary=14dae9cc9c683887ec05042902fe X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: extasia@extasia.org List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 150 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1411949456 news.xs4all.nl 2841 [2001:888:2000:d::a6]:33827 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:78444 --14dae9cc9c683887ec05042902fe Content-Type: text/plain; charset=UTF-8 greetings, i'm writing a program to scan a data file. from each line of the data file i'd like to add something like below to a dictionary. my perl background makes me want python to autovivify, but when i do: file_data = {} [... as i loop through lines in the file ...] file_data[ md5sum ][ inode ] = { 'path' : path, 'size' : size, } i get: Traceback (most recent call last): File "foo.py", line 45, in file_data[ md5sum ][ inode ] = { 'path' : path, 'size' : size, } KeyError: '91b152ce64af8af91dfe275575a20489' what is the pythonic way to build my "file_data" data structure above that has the above structure? on http://en.wikipedia.org/wiki/Autovivification there is a section on how to do autovivification in python, but i want to learn how a python programmer would normally build a data structure like this. here is the code so far: #!/usr/bin/python import argparse import os ASCII_NUL = chr(0) HOSTNAME = 0 MD5SUM = 1 FSDEV = 2 INODE = 3 NLINKS = 4 SIZE = 5 PATH = 6 file_data = {} if __name__ == "__main__": parser = argparse.ArgumentParser(description='scan files in a tree and print a line of information about each regular file') parser.add_argument('--file', '-f', required=True, help='File from which to read data') parser.add_argument('--field-separator', '-s', default=ASCII_NUL, help='Specify the string to use as a field separator in output. The default is the ascii nul character.') args = parser.parse_args() file = args.file field_separator = args.field_separator with open( file, 'rb' ) as f: for line in f: line = line.rstrip('\n') if line == 'None': continue fields = line.split( ASCII_NUL ) hostname = fields[ HOSTNAME ] md5sum = fields[ MD5SUM ] fsdev = fields[ FSDEV ] inode = fields[ INODE ] nlinks = int( fields[ NLINKS ] ) size = int( fields[ SIZE ] ) path = fields[ PATH ] if size < ( 100 * 1024 * 1024 ): continue ### print "'%s' '%s' '%s' '%s' '%s' '%s' '%s'" % ( hostname, md5sum, fsdev, inode, nlinks, size, path, ) file_data[ md5sum ][ inode ] = { 'path' : path, 'size' : size, } thanks, david -- Our decisions are the most important things in our lives. *** Live in a world of your own, but always welcome visitors. --14dae9cc9c683887ec05042902fe Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
greetings,

i'm writing a= program to scan a data file. =C2=A0from each line of the data file i'd= like to add something like below to a dictionary. =C2=A0my perl background= makes me want python to autovivify, but when i do:

=C2=A0 =C2=A0 =C2=A0 file_data =3D {}

=C2=A0 =C2= =A0 =C2=A0 [... as i loop through lines in the file ...]

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 file_data[ md5sum ][ inode ] =3D = { 'path' : path, 'size' : size, }

= i get:

Traceback (most recent call last):
=C2=A0 File "foo.py", line 45, in <module>
=C2= =A0 =C2=A0 file_data[ md5sum ][ inode ] =3D { 'path' : path, 's= ize' : size, }
KeyError: '91b152ce64af8af91dfe275575a2048= 9'

what is the pythonic way to build my "= file_data" data structure above that has the above structure?

on http://en.wikipedia.org/wiki/Autovivification there is a section on = how to do autovivification in python, but i want to learn how a python prog= rammer would normally build a data structure like this.

here is the code so far:

#!/usr/bin/python

import argparse
import os

<= /div>
ASCII_NUL =3D chr(0)

HOSTNAME =3D 0
MD5SUM =C2=A0 =3D 1
FSDEV =C2=A0 =C2=A0=3D 2
INOD= E =C2=A0 =C2=A0=3D 3
NLINKS =C2=A0 =3D 4
SIZE =C2=A0 = =C2=A0 =3D 5
PATH =C2=A0 =C2=A0 =3D 6

fi= le_data =3D {}

if __name__ =3D=3D "__main__&q= uot;:
=C2=A0 parser =3D argparse.ArgumentParser(description=3D= 9;scan files in a tree and print a line of information about each regular f= ile')
=C2=A0 parser.add_argument('--file', '-f= 9;, required=3DTrue, help=3D'File from which to read data')
=C2=A0 parser.add_argument('--field-separator', '-s', de= fault=3DASCII_NUL, help=3D'Specify the string to use as a field separat= or in output.=C2=A0 The default is the ascii nul character.')
=C2=A0 args =3D parser.parse_args()

=C2=A0 file = =3D args.file
=C2=A0 field_separator =3D args.field_separator

=C2=A0 with open( file, 'rb' ) as f:
=C2=A0 =C2=A0 for line in f:
=C2=A0 =C2=A0 =C2=A0 line =3D line= .rstrip('\n')
=C2=A0 =C2=A0 =C2=A0 if line =3D=3D 'No= ne': continue
=C2=A0 =C2=A0 =C2=A0 fields =3D line.split( ASC= II_NUL )

=C2=A0 =C2=A0 =C2=A0 hostname =3D fields[= HOSTNAME ]
=C2=A0 =C2=A0 =C2=A0 md5sum =C2=A0 =3D fields[ MD5SUM= ]
=C2=A0 =C2=A0 =C2=A0 fsdev =C2=A0 =C2=A0=3D fields[ FSDEV ]
=C2=A0 =C2=A0 =C2=A0 inode =C2=A0 =C2=A0=3D fields[ INODE ]
=C2=A0 =C2=A0 =C2=A0 nlinks =C2=A0 =3D int( fields[ NLINKS ] )
=C2=A0 =C2=A0 =C2=A0 size =C2=A0 =C2=A0 =3D int( fields[ SIZE ] )
=C2=A0 =C2=A0 =C2=A0 path =C2=A0 =C2=A0 =3D fields[ PATH ]

=
=C2=A0 =C2=A0 =C2=A0 if size < ( 100 * 1024 * 1024 ): continu= e

=C2=A0 =C2=A0 =C2=A0 ### print "'%s'= ; '%s' '%s' '%s' '%s' '%s' '%s&= #39;" % ( hostname, md5sum, fsdev, inode, nlinks, size, path, )
<= div>
=C2=A0 =C2=A0 =C2=A0 file_data[ md5sum ][ inode ] =3D { = 'path' : path, 'size' : size, }

th= anks,
david
--
Our decisions are t= he most important things in our lives.
***
Live in a world of= your own, but always welcome visitors.
--14dae9cc9c683887ec05042902fe--