Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Understanding " 'xml.etree.ElementTree.Element' does not support the buffer interface"
Date: Sun, 10 Jan 2016 12:53:28 -0500
Lines: 261
Message-ID: <mailman.6.1452448435.3151.python-list@python.org>
References: <876e5e0c-42c4-416a-90c0-ac2641e81949@googlegroups.com> <56929498$0$1622$c3e8da3$5496439d@news.astraweb.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <56929498$0$1622$c3e8da3$5496439d@news.astraweb.com>
Precedence: list
Xref: csiph.com comp.lang.python:101441

Hi Steven:

The previous code was a stand along under the " if __name__ =3D=3D '__main_=
_':
". The full function suite that I have made (and indeed includes a try and
except block):

import os.path
import sys
import csv
from io import StringIO
import xml.etree.cElementTree as ElementTree
from xml.etree.ElementTree import XMLParser
# import xml
# import xml.sax
# from xml.sax import ContentHandler


def flatten_list(self, aList, prefix=3D''):

    for i, element in enumerate(aList, 1):
        eprefix =3D "{}{}".format(prefix, i)
        if element:
            # treat like dict
            if len(element) =3D=3D 1 or element[0].tag !=3D element[1].tag:
                yield from flatten_dict(element, eprefix)
            # treat like list
            elif element[0].tag =3D=3D element[1].tag:
                yield from flatten_list(element, eprefix)
        elif element.text:
            text =3D element.text.strip()
            if text:
                yield eprefix[:].rstrip('.'), element.text


def flatten_dict(parent_element, prefix=3D''):

    prefix =3D prefix + parent_element.tag
    if parent_element.items():
        for k, v in parent_element.items():
            yield prefix + k, v
    for element in parent_element:
        eprefix =3D element.tag
        if element:
            # treat like dict - we assume that if the first two tags
            # in a series are different, then they are all different.
            if len(element) =3D=3D 1 or element[0].tag !=3D element[1].tag:
                yield from flatten_dict(element, prefix=3Dprefix)
            # treat like list - we assume that if the first two tags
            # in a series are the same, then the rest are the same.
            else:
                # here, we put the list in dictionary; the key is the
                # tag name the list elements all share in common, and
                # the value is the list itself
                yield from flatten_list(element, prefix=3Deprefix)
            # if the tag has attributes, add those to the dict
            if element.items():
                for k, v in element.items():
                    yield eprefix+k
        # this assumes that if you've got an attribute in a tag,
        # you won't be having any text. This may or may not be a
        # good idea -- time will tell. It works for the way we are
        # currently doing XML configuration files...
        elif element.items():
            for k, v in element.items():
                yield eprefix+k
        # finally, if there are no child tags and no attributes, extract
        # the text
        else:
            yield eprefix, element.text



def just_xml_data(path):
    with open(path, 'rU', encoding=3D'UTF-8') as data:
        separated =3D data.read().split('","')
        print(separated)
        try:
        x =3D ElementTree.XML(separated[3])
        print(x)
        xml.etree.ElementTree.dump(x)
        y =3D ElementTree.XML(separated[4])
        xml.etree.ElementTree.dump(y)
            # response =3D ElementTree.XML(separated[4])  # work on the
Response column
            # root =3D ElementTree.XML(response) #serialize and parse into
XML object
        except Exception as e:
            print(e)
        else:
            xml_field =3D dict(flatten_dict(y))
            return xml_field

def read_data(path):
    headers=3D set()
    rows =3D []
    with open(path, 'rU', encoding=3D'utf-8') as data:
        reader =3D csv.DictReader(data, dialect=3Dcsv.excel,
skipinitialspace=3DTrue)
        for row in reader:
            xml_field =3D row["CLIENT_RESP_DATA"]
            # xml_data =3D just_xml_data(xml_field) ## function
            if xml_data is not None:
                row.update(xml_data)
                headers.update(row.keys())
                rows.append(row)
            else:
                print("Failure")
                pass
    with open(os.path.splitext(textFile)[0] + '_' + 'parsed' + '.csv',
"wt", newline=3D'') as output_file:
        wr =3D csv.writer(output_file)
        csv_headers =3D list(headers)
        wr.writerow(csv_headers)
        for row in rows:
            values =3D []
            for field in csv_headers:
                value =3D row.get(field, None)
                values.append(value)
            wr.writerow(values)
    return output_file



if __name__ =3D=3D '__main__':
Response =3D "s.csv"
    just_xml_data(Response)


Hopefully this will provide you with enough information to emulate
(apologies for any and all indentation errors during the copy and paste).
FYI - I still receive the same error.


On Sun, Jan 10, 2016 at 12:27 PM, Steven D'Aprano <steve@pearwood.info>
wrote:

> On Mon, 11 Jan 2016 02:04 am, kbtyo wrote:
>
> > Hello Everyone:
> >
> > I am curious to know why I receive the aforementioned message. I am usi=
ng
> > Python 3.4.3 and Windows 7. I am running the following script from
> Windows
> > Powershell:
>
> I created a file "data" containing the input data you said:
>
> > The input data is as follows:
> >
> > A,B,C,D,E,F,G,H,I,J
> > "3","8","1","<Request TransactionID=3D"3" RequestType=3D"FOO"><Institut=
ionISO
> > /><CallID>23</CallID><MemberID>12</MemberID><MemberPassword
> >
> /><RequestData><AccountNumber>2</AccountNumber><AccountSuffix>85</Account=
Suffix><AccountType>S</AccountType><MPIAcctType>Checking</MPIAcctType><Tran=
sactionCount>10</TransactionCount></RequestData></Request>","<Response
> > TransactionID=3D"2"
> >
>
> RequestType=3D"HoldInquiry"><PulledLoans>True</PulledLoans><PulledClosedL=
oans>False</PulledClosedLoans><PulledInvestments>False</PulledInvestments><=
PulledClosedInvestments>False</PulledClosedInvestments><PulledCards>False</=
PulledCards><ShareList>0000',0001,0070,</ShareList></Response>","1967-12-25
> > 22:18:13.471000","2005-12-25 22:18:13.768000","2","70","0"
>
>
>
> and then a script containing the code you said you used:
>
> > import xml.etree.cElementTree as ElementTree
> > from xml.etree.ElementTree import XMLParser
>
> > Response =3D 's.csv'
> > with open(Response, 'rU', encoding=3D'utf-8') as data:
> >     separated =3D data.read().split('","')
> >     x =3D ElementTree.XML(separated[3])
> >     y =3D ElementTree.XML(separated[4])
> >     print(dict(flatten_dict(x)))
> >     print(dict(flatten_dict(y)))
>
>
> I get a completely different error to you, complete with traceback as
> expected:
>
> Traceback (most recent call last):
>   File "/tmp/testxml.py", line 9, in <module>
>     print(dict(flatten_dict(x)))
> NameError: name 'flatten_dict' is not defined
>
>
> This shows me three things:
>
> (1) The calls to ElementTree.XML work fine, and don't raise an exception;
>
> (2) There is no error message referring to xml.etree.ElementTree.Element =
or
> the buffer interface;
>
> (3) The code you posted is clearly not the code you actually ran. At the
> very least, it is not *all* the code you ran.
>
> We cannot tell what it wrong with your code if you don't show us the code
> that fails. I suggest you read this webpage:
>
> http://www.sscce.org/
>
> and follow the advice given. It's written for Java, but applies to any
> programming language. Hopefully you will either solve your problem, or be
> able to generate a sufficiently small piece of code that we can work with=
.
>
>
> You also suggest that your code works when running in a Jupyter Notebook.
> It
> is unlikely (but not impossible!) that exactly the same code will run
> differently when run as a script and when run under Jupyter. More likely,
> there is some difference between the code, something you have written in
> the Notebook but not included in the script.
>
> If it is exactly the same code, then perhaps it is a difference in the tw=
o
> environments. Does Jupyter set up the environment differently to what you
> get when running a script?
>
> Finally, in another post, you state:
>
> "That is the only message (*xml.etree.ElementTree.Element' does not suppo=
rt
> the buffer interface"*). There is no traceback."
>
>
> That is very unlikely with the code sample you posted. If true, that give=
s
> more evidence that you are running code which is different from what you
> have posted here. Perhaps your ACTUAL code (not the pretend code you show=
ed
> us) includes a try...except block like this:
>
> try:
>     some code goes here
> except Exception as err:
>     print(err)
>     sys.exit()
>
>
> or similar. If so, TAKE IT OUT. That is destroying useful debugging
> information and making it more difficult to solve your problem.
>
>
>
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>