Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Understanding " 'xml.etree.ElementTree.Element' does not support the buffer interface"
Date: Mon, 11 Jan 2016 16:54:48 -0500
Lines: 283
Message-ID: <mailman.24.1452549315.13488.python-list@python.org>
References: <876e5e0c-42c4-416a-90c0-ac2641e81949@googlegroups.com> <56929498$0$1622$c3e8da3$5496439d@news.astraweb.com> <CAC9q6M435_u-ixxEiQ6Z=GpfaX5rd30RoEni-jxoDJs+dSrAkA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <CAC9q6M435_u-ixxEiQ6Z=GpfaX5rd30RoEni-jxoDJs+dSrAkA@mail.gmail.com>
Precedence: list
Xref: csiph.com comp.lang.python:101494

Hi Steven:

Just as an update - apparently there were bytes in the Windows Command
Terminal that were interrupting the process execution. I didn't realize
this was happening until I dug around Windows' Q&A forum.

Cheers

On Sun, Jan 10, 2016 at 12:53 PM, Saran Ahluwalia <
ahlusar.ahluwalia@gmail.com> wrote:

> Hi Steven:
>
> The previous code was a stand along under the " if __name__ =3D=3D
> '__main__': ". The full function suite that I have made (and indeed
> includes a try and except block):
>
> import os.path
> import sys
> import csv
> from io import StringIO
> import xml.etree.cElementTree as ElementTree
> from xml.etree.ElementTree import XMLParser
> # import xml
> # import xml.sax
> # from xml.sax import ContentHandler
>
>
> def flatten_list(self, aList, prefix=3D''):
>
>     for i, element in enumerate(aList, 1):
>         eprefix =3D "{}{}".format(prefix, i)
>         if element:
>             # treat like dict
>             if len(element) =3D=3D 1 or element[0].tag !=3D element[1].ta=
g:
>                 yield from flatten_dict(element, eprefix)
>             # treat like list
>             elif element[0].tag =3D=3D element[1].tag:
>                 yield from flatten_list(element, eprefix)
>         elif element.text:
>             text =3D element.text.strip()
>             if text:
>                 yield eprefix[:].rstrip('.'), element.text
>
>
> def flatten_dict(parent_element, prefix=3D''):
>
>     prefix =3D prefix + parent_element.tag
>     if parent_element.items():
>         for k, v in parent_element.items():
>             yield prefix + k, v
>     for element in parent_element:
>         eprefix =3D element.tag
>         if element:
>             # treat like dict - we assume that if the first two tags
>             # in a series are different, then they are all different.
>             if len(element) =3D=3D 1 or element[0].tag !=3D element[1].ta=
g:
>                 yield from flatten_dict(element, prefix=3Dprefix)
>             # treat like list - we assume that if the first two tags
>             # in a series are the same, then the rest are the same.
>             else:
>                 # here, we put the list in dictionary; the key is the
>                 # tag name the list elements all share in common, and
>                 # the value is the list itself
>                 yield from flatten_list(element, prefix=3Deprefix)
>             # if the tag has attributes, add those to the dict
>             if element.items():
>                 for k, v in element.items():
>                     yield eprefix+k
>         # this assumes that if you've got an attribute in a tag,
>         # you won't be having any text. This may or may not be a
>         # good idea -- time will tell. It works for the way we are
>         # currently doing XML configuration files...
>         elif element.items():
>             for k, v in element.items():
>                 yield eprefix+k
>         # finally, if there are no child tags and no attributes, extract
>         # the text
>         else:
>             yield eprefix, element.text
>
>
>
> def just_xml_data(path):
>     with open(path, 'rU', encoding=3D'UTF-8') as data:
>         separated =3D data.read().split('","')
>         print(separated)
>         try:
>         x =3D ElementTree.XML(separated[3])
>         print(x)
>         xml.etree.ElementTree.dump(x)
>         y =3D ElementTree.XML(separated[4])
>         xml.etree.ElementTree.dump(y)
>             # response =3D ElementTree.XML(separated[4])  # work on the
> Response column
>             # root =3D ElementTree.XML(response) #serialize and parse int=
o
> XML object
>         except Exception as e:
>             print(e)
>         else:
>             xml_field =3D dict(flatten_dict(y))
>             return xml_field
>
> def read_data(path):
>     headers=3D set()
>     rows =3D []
>     with open(path, 'rU', encoding=3D'utf-8') as data:
>         reader =3D csv.DictReader(data, dialect=3Dcsv.excel,
> skipinitialspace=3DTrue)
>         for row in reader:
>             xml_field =3D row["CLIENT_RESP_DATA"]
>             # xml_data =3D just_xml_data(xml_field) ## function
>             if xml_data is not None:
>                 row.update(xml_data)
>                 headers.update(row.keys())
>                 rows.append(row)
>             else:
>                 print("Failure")
>                 pass
>     with open(os.path.splitext(textFile)[0] + '_' + 'parsed' + '.csv',
> "wt", newline=3D'') as output_file:
>         wr =3D csv.writer(output_file)
>         csv_headers =3D list(headers)
>         wr.writerow(csv_headers)
>         for row in rows:
>             values =3D []
>             for field in csv_headers:
>                 value =3D row.get(field, None)
>                 values.append(value)
>             wr.writerow(values)
>     return output_file
>
>
>
> if __name__ =3D=3D '__main__':
> Response =3D "s.csv"
>     just_xml_data(Response)
>
>
> Hopefully this will provide you with enough information to emulate
> (apologies for any and all indentation errors during the copy and paste).
> FYI - I still receive the same error.
>
>
> On Sun, Jan 10, 2016 at 12:27 PM, Steven D'Aprano <steve@pearwood.info>
> wrote:
>
>> On Mon, 11 Jan 2016 02:04 am, kbtyo wrote:
>>
>> > Hello Everyone:
>> >
>> > I am curious to know why I receive the aforementioned message. I am
>> using
>> > Python 3.4.3 and Windows 7. I am running the following script from
>> Windows
>> > Powershell:
>>
>> I created a file "data" containing the input data you said:
>>
>> > The input data is as follows:
>> >
>> > A,B,C,D,E,F,G,H,I,J
>> > "3","8","1","<Request TransactionID=3D"3"
>> RequestType=3D"FOO"><InstitutionISO
>> > /><CallID>23</CallID><MemberID>12</MemberID><MemberPassword
>> >
>> /><RequestData><AccountNumber>2</AccountNumber><AccountSuffix>85</Accoun=
tSuffix><AccountType>S</AccountType><MPIAcctType>Checking</MPIAcctType><Tra=
nsactionCount>10</TransactionCount></RequestData></Request>","<Response
>> > TransactionID=3D"2"
>> >
>>
>> RequestType=3D"HoldInquiry"><PulledLoans>True</PulledLoans><PulledClosed=
Loans>False</PulledClosedLoans><PulledInvestments>False</PulledInvestments>=
<PulledClosedInvestments>False</PulledClosedInvestments><PulledCards>False<=
/PulledCards><ShareList>0000',0001,0070,</ShareList></Response>","1967-12-2=
5
>> > 22:18:13.471000","2005-12-25 22:18:13.768000","2","70","0"
>>
>>
>>
>> and then a script containing the code you said you used:
>>
>> > import xml.etree.cElementTree as ElementTree
>> > from xml.etree.ElementTree import XMLParser
>>
>> > Response =3D 's.csv'
>> > with open(Response, 'rU', encoding=3D'utf-8') as data:
>> >     separated =3D data.read().split('","')
>> >     x =3D ElementTree.XML(separated[3])
>> >     y =3D ElementTree.XML(separated[4])
>> >     print(dict(flatten_dict(x)))
>> >     print(dict(flatten_dict(y)))
>>
>>
>> I get a completely different error to you, complete with traceback as
>> expected:
>>
>> Traceback (most recent call last):
>>   File "/tmp/testxml.py", line 9, in <module>
>>     print(dict(flatten_dict(x)))
>> NameError: name 'flatten_dict' is not defined
>>
>>
>> This shows me three things:
>>
>> (1) The calls to ElementTree.XML work fine, and don't raise an exception=
;
>>
>> (2) There is no error message referring to xml.etree.ElementTree.Element
>> or
>> the buffer interface;
>>
>> (3) The code you posted is clearly not the code you actually ran. At the
>> very least, it is not *all* the code you ran.
>>
>> We cannot tell what it wrong with your code if you don't show us the cod=
e
>> that fails. I suggest you read this webpage:
>>
>> http://www.sscce.org/
>>
>> and follow the advice given. It's written for Java, but applies to any
>> programming language. Hopefully you will either solve your problem, or b=
e
>> able to generate a sufficiently small piece of code that we can work wit=
h.
>>
>>
>> You also suggest that your code works when running in a Jupyter Notebook=
.
>> It
>> is unlikely (but not impossible!) that exactly the same code will run
>> differently when run as a script and when run under Jupyter. More likely=
,
>> there is some difference between the code, something you have written in
>> the Notebook but not included in the script.
>>
>> If it is exactly the same code, then perhaps it is a difference in the t=
wo
>> environments. Does Jupyter set up the environment differently to what yo=
u
>> get when running a script?
>>
>> Finally, in another post, you state:
>>
>> "That is the only message (*xml.etree.ElementTree.Element' does not
>> support
>> the buffer interface"*). There is no traceback."
>>
>>
>> That is very unlikely with the code sample you posted. If true, that giv=
es
>> more evidence that you are running code which is different from what you
>> have posted here. Perhaps your ACTUAL code (not the pretend code you
>> showed
>> us) includes a try...except block like this:
>>
>> try:
>>     some code goes here
>> except Exception as err:
>>     print(err)
>>     sys.exit()
>>
>>
>> or similar. If so, TAKE IT OUT. That is destroying useful debugging
>> information and making it more difficult to solve your problem.
>>
>>
>>
>>
>>
>> --
>> Steven
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
>