Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #108755 > unrolled thread

OrderedDict

Back to article view | Back to comp.lang.python

Contents

  OrderedDict silver0346@gmail.com - 2016-05-18 01:32 -0700
    Re: OrderedDict Chris Angelico <rosuav@gmail.com> - 2016-05-18 18:54 +1000
    Re: OrderedDict Peter Otten <__peter__@web.de> - 2016-05-18 11:28 +0200
    Re: OrderedDict Chris Angelico <rosuav@gmail.com> - 2016-05-18 20:47 +1000
    Re: OrderedDict Peter Otten <__peter__@web.de> - 2016-05-18 14:24 +0200
      Re: OrderedDict silver0346@gmail.com - 2016-05-19 22:15 -0700
        Re: OrderedDict silver0346@gmail.com - 2016-06-09 03:02 -0700

#108755 — OrderedDict

Hi all,

I have a understanding problem with return values from xmltodict.

I have a xml file. Content:

<?xml version="1.0" encoding="utf-8" ?>
<profiles>
  <profile id='visio02' revision='2015051501' >
  <package package-id='0964-gpg4win' />
  </profile>
</profiles>
<!--
-->

With code

__f_name = '<path_to_file>'
with open(__f_name) as __fd:
    __doc = xmltodict.parse(__fd.read())
   
__doc

I get

OrderedDict([(u'profiles', OrderedDict([(u'profile', OrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), (u'package', OrderedDict([(u'@package-id', u'0964-gpg4win')]))]))]))])

If I use

__doc['profiles']['profile']['package'][0]['@package-id']

I get 

Traceback (most recent call last):
  File "<input>", line 1, in <module>
KeyError: 0

If I change xml file like this:

<?xml version="1.0" encoding="utf-8" ?>
<profiles>
  <profile id='visio02' revision='2015051501' >
  <package package-id='0964-gpg4win' />
  <package package-id='0965-gpg4win' />
  </profile>
</profiles>

and run code from above the result is:

OrderedDict([(u'profiles', OrderedDict([(u'profile', OrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), (u'package', [OrderedDict([(u'@package-id', u'0964-gpg4win')]), OrderedDict([(u'@package-id', u'0965-gpg4win')])])]))]))])

No prints __doc['profiles']['profile']['package'][0]['@package-id']:

u'0964-gpg4win'

Can everybody explain this?

Many thanks in advance

[toc] | [next] | [standalone]

#108756

On Wed, May 18, 2016 at 6:32 PM,  <silver0346@gmail.com> wrote:
> Hi all,
>
> I have a understanding problem with return values from xmltodict.
>
> I have a xml file. Content:
>
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   </profile>
> </profiles>
>
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   <package package-id='0965-gpg4win' />
>   </profile>
> </profiles>
>
> No prints __doc['profiles']['profile']['package'][0]['@package-id']:
>
> u'0964-gpg4win'
>
> Can everybody explain this?

This is one of the inherent problems of trying to convert XML (a
structured document format) into a nested dictionary (a structured
tree structure). In a dict, you can't have more than one value
associated with a given key; in XML, and similar container-tag-based
structures, you can - as in this example, where you have two package
tags. When that gets converted into a dictionary, they get joined up
into a list, which is why you can subscript it with the integer 0 to
get the first one. If it's not made into a list, subscriping gives you
the next level of dict straight away.

If you're dealing with some files that have one package and some that
have multiple, the easiest way would be something like this:

packages = __doc['profiles']['profile']['package']
if isinstance(packages, dict):
    packages = [packages]
for package in packages:
    # Do whatever you need with the package
    print(package['@package-id'])

Incidentally, the double-leading-underscore names are not what I'd
recommend; use simple names, and if you need them to be private, put
this into a function (which will make them all function-local by
default). The double leading underscore has special meaning inside a
class, which you most likely don't intend.

ChrisA

[toc] | [prev] | [next] | [standalone]

#108757

silver0346@gmail.com wrote:

> Hi all,
> 
> I have a understanding problem with return values from xmltodict.
> 
> I have a xml file. Content:
> 
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   </profile>
> </profiles>
> <!--
> -->
> 
> With code
> 
> __f_name = '<path_to_file>'
> with open(__f_name) as __fd:
>     __doc = xmltodict.parse(__fd.read())
>    
> __doc
> 
> I get
> 
> OrderedDict([(u'profiles', OrderedDict([(u'profile', OrderedDict([(u'@id',
> u'visio02'), (u'@revision', u'2015051501'), (u'package',
> OrderedDict([(u'@package-id', u'0964-gpg4win')]))]))]))])
> 
> If I use
> 
> __doc['profiles']['profile']['package'][0]['@package-id']
> 
> I get
> 
> Traceback (most recent call last):
>   File "<input>", line 1, in <module>
> KeyError: 0
> 
> If I change xml file like this:
> 
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   <package package-id='0965-gpg4win' />
>   </profile>
> </profiles>
> 
> and run code from above the result is:
> 
> OrderedDict([(u'profiles', OrderedDict([(u'profile', OrderedDict([(u'@id',
> u'visio02'), (u'@revision', u'2015051501'), (u'package',
> [OrderedDict([(u'@package-id', u'0964-gpg4win')]),
> OrderedDict([(u'@package-id', u'0965-gpg4win')])])]))]))])
> 
> No prints __doc['profiles']['profile']['package'][0]['@package-id']:
> 
> u'0964-gpg4win'
> 
> Can everybody explain this?

Not everybody, but Chris and a few others can ;)
 
> Many thanks in advance

I don't see an official way to pass a custom dict type to the library, 
but if you are not afraid to change its source code the following patch
will allow you to access the value of dictionaries with a single entry as d[0]:

$ diff -u py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
--- py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py       2016-05-18 11:18:44.000000000 +0200
+++ py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py        2016-05-18 11:11:13.417665697 +0200
@@ -35,6 +35,13 @@
 __version__ = '0.10.1'
 __license__ = 'MIT'
 
+_OrderedDict = OrderedDict
+class OrderedDict(_OrderedDict):
+    def __getitem__(self, key):
+        if key == 0:
+            [result] = self.values()
+            return result
+        return _OrderedDict.__getitem__(self, key)
 
 class ParsingInterrupted(Exception):
     pass

[toc] | [prev] | [next] | [standalone]

#108758

On Wed, May 18, 2016 at 7:28 PM, Peter Otten <__peter__@web.de> wrote:
> I don't see an official way to pass a custom dict type to the library,
> but if you are not afraid to change its source code the following patch
> will allow you to access the value of dictionaries with a single entry as d[0]:
>
> $ diff -u py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
> --- py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py       2016-05-18 11:18:44.000000000 +0200
> +++ py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py        2016-05-18 11:11:13.417665697 +0200
> @@ -35,6 +35,13 @@
>  __version__ = '0.10.1'
>  __license__ = 'MIT'
>
> +_OrderedDict = OrderedDict
> +class OrderedDict(_OrderedDict):
> +    def __getitem__(self, key):
> +        if key == 0:
> +            [result] = self.values()
> +            return result
> +        return _OrderedDict.__getitem__(self, key)
>
>  class ParsingInterrupted(Exception):
>      pass

Easier than patching might be monkeypatching.

class OrderedDict(OrderedDict):
    ... getitem code as above ...
xmltodict.OrderedDict = OrderedDict

Try it, see if it works.

ChrisA

[toc] | [prev] | [next] | [standalone]

#108760

Chris Angelico wrote:

> On Wed, May 18, 2016 at 7:28 PM, Peter Otten <__peter__@web.de> wrote:
>> I don't see an official way to pass a custom dict type to the library,
>> but if you are not afraid to change its source code the following patch
>> will allow you to access the value of dictionaries with a single entry as
>> d[0]:
>>
>> $ diff -u py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
>> py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
>> --- py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py      
>> 2016-05-18 11:18:44.000000000 +0200
>> +++ py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py       
>> 2016-05-18 11:11:13.417665697 +0200 @@ -35,6 +35,13 @@
>>  __version__ = '0.10.1'
>>  __license__ = 'MIT'
>>
>> +_OrderedDict = OrderedDict
>> +class OrderedDict(_OrderedDict):
>> +    def __getitem__(self, key):
>> +        if key == 0:
>> +            [result] = self.values()
>> +            return result
>> +        return _OrderedDict.__getitem__(self, key)
>>
>>  class ParsingInterrupted(Exception):
>>      pass
> 
> Easier than patching might be monkeypatching.
> 
> class OrderedDict(OrderedDict):
>     ... getitem code as above ...
> xmltodict.OrderedDict = OrderedDict
> 
> Try it, see if it works.

It turns out I was wrong on (at least) two accounts: 

- xmltodict does offer a way to specify the dict type
- the proposed dict implementation will not solve the OP's problem

Here is an improved fix which should work:


$ cat sample.xml 
<?xml version="1.0" encoding="utf-8" ?>
<profiles>
  <profile id='visio02' revision='2015051501' >
  <package package-id='0964-gpg4win' />
  </profile>
</profiles>
$ cat sample2.xml 
<?xml version="1.0" encoding="utf-8" ?>
<profiles>
  <profile id='visio02' revision='2015051501' >
  <package package-id='0964-gpg4win' />
  <package package-id='0965-gpg4win' />
  </profile>
</profiles>
$ cat demo.py
import collections
import sys
import xmltodict


class MyOrderedDict(collections.OrderedDict):
    def __getitem__(self, key):
        if key == 0 and len(self) == 1:
            return self
        return super(MyOrderedDict, self).__getitem__(key)


def main():
    filename = sys.argv[1]
    with open(filename) as f:
        doc = xmltodict.parse(f.read(), dict_constructor=MyOrderedDict)

    print "doc:\n{}\n".format(doc)
    print "package-id: {}".format(
        doc['profiles']['profile']['package'][0]['@package-id'])


if __name__ == "__main__":
    main()
$ python demo.py sample.xml 
doc:
MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
(u'package', MyOrderedDict([(u'@package-id', u'0964-gpg4win')]))]))]))])

package-id: 0964-gpg4win
$ python demo.py sample2.xml 
doc:
MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
(u'package', [MyOrderedDict([(u'@package-id', u'0964-gpg4win')]), 
MyOrderedDict([(u'@package-id', u'0965-gpg4win')])])]))]))])

package-id: 0964-gpg4win

[toc] | [prev] | [next] | [standalone]

#108851

On Wednesday, May 18, 2016 at 2:25:16 PM UTC+2, Peter Otten wrote:
> Chris Angelico wrote:
> 
> > On Wed, May 18, 2016 at 7:28 PM, Peter Otten <__peter__@web.de> wrote:
> >> I don't see an official way to pass a custom dict type to the library,
> >> but if you are not afraid to change its source code the following patch
> >> will allow you to access the value of dictionaries with a single entry as
> >> d[0]:
> >>
> >> $ diff -u py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
> >> py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
> >> --- py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py      
> >> 2016-05-18 11:18:44.000000000 +0200
> >> +++ py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py       
> >> 2016-05-18 11:11:13.417665697 +0200 @@ -35,6 +35,13 @@
> >>  __version__ = '0.10.1'
> >>  __license__ = 'MIT'
> >>
> >> +_OrderedDict = OrderedDict
> >> +class OrderedDict(_OrderedDict):
> >> +    def __getitem__(self, key):
> >> +        if key == 0:
> >> +            [result] = self.values()
> >> +            return result
> >> +        return _OrderedDict.__getitem__(self, key)
> >>
> >>  class ParsingInterrupted(Exception):
> >>      pass
> > 
> > Easier than patching might be monkeypatching.
> > 
> > class OrderedDict(OrderedDict):
> >     ... getitem code as above ...
> > xmltodict.OrderedDict = OrderedDict
> > 
> > Try it, see if it works.
> 
> It turns out I was wrong on (at least) two accounts: 
> 
> - xmltodict does offer a way to specify the dict type
> - the proposed dict implementation will not solve the OP's problem
> 
> Here is an improved fix which should work:
> 
> 
> $ cat sample.xml 
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   </profile>
> </profiles>
> $ cat sample2.xml 
> <?xml version="1.0" encoding="utf-8" ?>
> <profiles>
>   <profile id='visio02' revision='2015051501' >
>   <package package-id='0964-gpg4win' />
>   <package package-id='0965-gpg4win' />
>   </profile>
> </profiles>
> $ cat demo.py
> import collections
> import sys
> import xmltodict
> 
> 
> class MyOrderedDict(collections.OrderedDict):
>     def __getitem__(self, key):
>         if key == 0 and len(self) == 1:
>             return self
>         return super(MyOrderedDict, self).__getitem__(key)
> 
> 
> def main():
>     filename = sys.argv[1]
>     with open(filename) as f:
>         doc = xmltodict.parse(f.read(), dict_constructor=MyOrderedDict)
> 
>     print "doc:\n{}\n".format(doc)
>     print "package-id: {}".format(
>         doc['profiles']['profile']['package'][0]['@package-id'])
> 
> 
> if __name__ == "__main__":
>     main()
> $ python demo.py sample.xml 
> doc:
> MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
> MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
> (u'package', MyOrderedDict([(u'@package-id', u'0964-gpg4win')]))]))]))])
> 
> package-id: 0964-gpg4win
> $ python demo.py sample2.xml 
> doc:
> MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
> MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
> (u'package', [MyOrderedDict([(u'@package-id', u'0964-gpg4win')]), 
> MyOrderedDict([(u'@package-id', u'0965-gpg4win')])])]))]))])
> 
> package-id: 0964-gpg4win

I have tested the first solution. Works nice. Before I used xml.etree to parse 2000 xml files. 

Execution time decrease from more then 5 min to 20 sec. Great. On weekend I will test the solution with the own class.

Many thanks.

[toc] | [prev] | [next] | [standalone]

#109720

On Friday, May 20, 2016 at 7:15:38 AM UTC+2, silve...@gmail.com wrote:
> On Wednesday, May 18, 2016 at 2:25:16 PM UTC+2, Peter Otten wrote:
> > Chris Angelico wrote:
> > 
> > > On Wed, May 18, 2016 at 7:28 PM, Peter Otten <__peter__@web.de> wrote:
> > >> I don't see an official way to pass a custom dict type to the library,
> > >> but if you are not afraid to change its source code the following patch
> > >> will allow you to access the value of dictionaries with a single entry as
> > >> d[0]:
> > >>
> > >> $ diff -u py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
> > >> py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py
> > >> --- py2b_xmltodict/local/lib/python2.7/site-packages/xmltodict.py      
> > >> 2016-05-18 11:18:44.000000000 +0200
> > >> +++ py2_xmltodict/local/lib/python2.7/site-packages/xmltodict.py       
> > >> 2016-05-18 11:11:13.417665697 +0200 @@ -35,6 +35,13 @@
> > >>  __version__ = '0.10.1'
> > >>  __license__ = 'MIT'
> > >>
> > >> +_OrderedDict = OrderedDict
> > >> +class OrderedDict(_OrderedDict):
> > >> +    def __getitem__(self, key):
> > >> +        if key == 0:
> > >> +            [result] = self.values()
> > >> +            return result
> > >> +        return _OrderedDict.__getitem__(self, key)
> > >>
> > >>  class ParsingInterrupted(Exception):
> > >>      pass
> > > 
> > > Easier than patching might be monkeypatching.
> > > 
> > > class OrderedDict(OrderedDict):
> > >     ... getitem code as above ...
> > > xmltodict.OrderedDict = OrderedDict
> > > 
> > > Try it, see if it works.
> > 
> > It turns out I was wrong on (at least) two accounts: 
> > 
> > - xmltodict does offer a way to specify the dict type
> > - the proposed dict implementation will not solve the OP's problem
> > 
> > Here is an improved fix which should work:
> > 
> > 
> > $ cat sample.xml 
> > <?xml version="1.0" encoding="utf-8" ?>
> > <profiles>
> >   <profile id='visio02' revision='2015051501' >
> >   <package package-id='0964-gpg4win' />
> >   </profile>
> > </profiles>
> > $ cat sample2.xml 
> > <?xml version="1.0" encoding="utf-8" ?>
> > <profiles>
> >   <profile id='visio02' revision='2015051501' >
> >   <package package-id='0964-gpg4win' />
> >   <package package-id='0965-gpg4win' />
> >   </profile>
> > </profiles>
> > $ cat demo.py
> > import collections
> > import sys
> > import xmltodict
> > 
> > 
> > class MyOrderedDict(collections.OrderedDict):
> >     def __getitem__(self, key):
> >         if key == 0 and len(self) == 1:
> >             return self
> >         return super(MyOrderedDict, self).__getitem__(key)
> > 
> > 
> > def main():
> >     filename = sys.argv[1]
> >     with open(filename) as f:
> >         doc = xmltodict.parse(f.read(), dict_constructor=MyOrderedDict)
> > 
> >     print "doc:\n{}\n".format(doc)
> >     print "package-id: {}".format(
> >         doc['profiles']['profile']['package'][0]['@package-id'])
> > 
> > 
> > if __name__ == "__main__":
> >     main()
> > $ python demo.py sample.xml 
> > doc:
> > MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
> > MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
> > (u'package', MyOrderedDict([(u'@package-id', u'0964-gpg4win')]))]))]))])
> > 
> > package-id: 0964-gpg4win
> > $ python demo.py sample2.xml 
> > doc:
> > MyOrderedDict([(u'profiles', MyOrderedDict([(u'profile', 
> > MyOrderedDict([(u'@id', u'visio02'), (u'@revision', u'2015051501'), 
> > (u'package', [MyOrderedDict([(u'@package-id', u'0964-gpg4win')]), 
> > MyOrderedDict([(u'@package-id', u'0965-gpg4win')])])]))]))])
> > 
> > package-id: 0964-gpg4win
> 
> I have tested the first solution. Works nice. Before I used xml.etree to parse 2000 xml files. 
> 
> Execution time decrease from more then 5 min to 20 sec. Great. On weekend I will test the solution with the own class.
> 
> Many thanks.

Hi all,

tests with solution with the own class successful. Nice inspiration. I use this solution in my django script.

Many thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web