Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #89950 > unrolled thread

Is it normal to cry when given XML?

Back to article view | Back to comp.lang.python

Contents

  Is it normal to cry when given XML? Sayth Renshaw <flebber.crue@gmail.com> - 2015-05-05 02:28 -0700
    Re: Is it normal to cry when given XML? Chris Angelico <rosuav@gmail.com> - 2015-05-05 19:44 +1000
    Re: Is it normal to cry when given XML? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-05 20:04 +1000
    Re: Is it normal to cry when given XML? Adam Tauno Williams <awilliam@whitemice.org> - 2015-05-05 06:20 -0400
      Re: Is it normal to cry when given XML? Sayth Renshaw <flebber.crue@gmail.com> - 2015-05-05 05:03 -0700
    Re: Is it normal to cry when given XML? Tim <jtim.arnold@gmail.com> - 2015-05-05 06:05 -0700
    Re: Is it normal to cry when given XML? Michael Torrie <torriem@gmail.com> - 2015-05-05 10:54 -0600
    Re: Is it normal to cry when given XML? Grant Edwards <invalid@invalid.invalid> - 2015-05-05 16:58 +0000
      Re: Is it normal to cry when given XML? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-05-05 21:03 -0400
    Re: Is it normal to cry when given XML? Emile van Sebille <emile@fenx.com> - 2015-05-05 16:04 -0700
    Re: Is it normal to cry when given XML? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-06 00:35 +0100
    Re: Is it normal to cry when given XML? Chris Angelico <rosuav@gmail.com> - 2015-05-06 10:07 +1000
      Re: Is it normal to cry when given XML? Rustom Mody <rustompmody@gmail.com> - 2015-05-05 17:19 -0700
        Re: Is it normal to cry when given XML? Chris Angelico <rosuav@gmail.com> - 2015-05-06 10:29 +1000

#89950 — Is it normal to cry when given XML?

Hi

Just checking if the reaction to cry when given XML is normal. 

I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission. 

See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .

See its never basic xml, usually comes from some database with a walk of tables and strange relationships.

Am I doing it wrong is there a simple way I am missing? 

Sayth

[toc] | [next] | [standalone]

#89952

On Tue, May 5, 2015 at 7:28 PM, Sayth Renshaw <flebber.crue@gmail.com> wrote:
> Hi
>
> Just checking if the reaction to cry when given XML is normal.

It's not unsurprising, especially with bad XML structures.

> I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission.
>
> See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .
>
> See its never basic xml, usually comes from some database with a walk of tables and strange relationships.
>
> Am I doing it wrong is there a simple way I am missing?

Generally, I work with XML only as a transport layer; and most of the
time, it's for a document structure that would be better served by
JSON anyway. (This may mean that I have an unfairly negative view of
XML, but it's extremely common.) My usual technique is to parse it
into something native (usually a dictionary - and probably the same
structure that the other end constructed the XML from), then work with
that. For example, querying the ePond API [1] gives back a pile of XML
data, so I might have a single function that performs a synchronous
HTTP query, takes the response body, parses it using a fairly generic
XML parser like lxml, then digs three levels into the resulting tree
to pull out the bit that actually matters, leaving behind all the
framing and stuff.

The less time you spend with actual XML, the better. XML is not the answer.

ChrisA

[1] A completely fictional web site, of course, and in no way implying
that I have had a frustrating time with a well-known online
sales/auction company.

[toc] | [prev] | [next] | [standalone]

#89954

On Tuesday 05 May 2015 19:28, Sayth Renshaw wrote:

> Just checking if the reaction to cry when given XML is normal.


Cry? When people give me XML, sometimes I lose control of my bladder.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#89956

On Tue, 2015-05-05 at 02:28 -0700, Sayth Renshaw wrote:
> Just checking if the reaction to cry when given XML is normal. 

No, not at all.  I leap for ecstatic joy when given, as all to rarely
happens, XML.  Rather than someone's turdy text [which includes JSON]
file.  I wish all 1,200+ of my vendors and suppliers would provide their
data in XML rather than the random swirl of bizarre crap I receive.

> I thought maybe I am approaching it all wrong, using lxml largely or 
> some xquery to club it into submission. 

I do most of my processing with LXML, XSLT, and XPath.  Fast, efficient,
reliable, works great.  And it is easy to abstract these tools to
automate what I need to do over and over again.

> See the usual goal is just to take the entire XML and push it into a
> database. 

I do a lot of pushing into an SQL database, no problem.  XSLT does 99.4%
of the work.

> See its never basic xml, 

"basic xml"? 

> usually comes from some database with a walk of tables and strange relationships.

No problem.

> Am I doing it wrong is there a simple way I am missing? 

I suspect so.  Data is easily transformed into

<ResultSet table="xyz">
  <row>
    <field1 dataType="string" isNull="false"
isPrimaryKey="true">abc</field1>
    <field2 dataType="string" isNull="true" isPrimaryKey="false"/>
    <field3 dataType="string" isNull="false"
isPrimaryKey="false">stanley</field1>
  </row>
 ...
</ResultSet>

And then that data can be processes as SQL insert/update/delete/upsert
operations while being swung around on the end of a stick while you
dance.

<https://sourceforge.net/p/coils/code/ci/default/tree/coils/logic/workflow/actions/sql/upsert.py>

XML is the ultimate file "format".  It solves problems, much better than
everyone trying to re-engineer their needs INTO a format [like JSON].





-- 
Adam Tauno Williams <mailto:awilliam@whitemice.org> GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA

[toc] | [prev] | [next] | [standalone]

#89963

Adam I am glad to hear it in someways because it's something I have never heard it. For a person relatively new to XML most articles and tutorials demonstrate getting it out to a more "manageable" format.

I had been using xbase to inspect the data and query but really ask I want to do was push it to a db and then query it as is more normal for me. 

I will check out your link. 

[toc] | [prev] | [next] | [standalone]

#89964

On Tuesday, May 5, 2015 at 5:28:37 AM UTC-4, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal. 
> 
> Sayth

Hi Sayth,
My experience in general is just like what Chris said. Except when dealing with DocBook XML which is probably not what you have. But I cannot say enough good things about lxml, which is my favorite 3rd-party package ever. Its support for xpath really makes it easy to traverse and select elements in a document tree.  Plus the ability to drop subtrees, move elements around in a live tree, etc. Great library.

Using lxml to parse and restructure the xml to a dictionary isn't too hard, but of course it depends on the xml you have to deal with. Sometimes weeping is just part of the job.

good luck,
--Tim

[toc] | [prev] | [next] | [standalone]

#89974

On 05/05/2015 03:28 AM, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal.

I'd say it is normal.  XML is like violence.  If it doesn't solve your
problems, you're not using enough of it[1].

[1] Can anyone tell me who originated this line?

[toc] | [prev] | [next] | [standalone]

#89975

On 2015-05-05, Sayth Renshaw <flebber.crue@gmail.com> wrote:

> Just checking if the reaction to cry when given XML is normal.

It depends on whether or not you're allowed to swear and throw things.

-- 
Grant Edwards               grant.b.edwards        Yow! I just heard the
                                  at               SEVENTIES were over!!  And
                              gmail.com            I was just getting in touch
                                                   with my LEISURE SUIT!!

[toc] | [prev] | [next] | [standalone]

#90019

On Tue, 5 May 2015 16:58:10 +0000 (UTC), Grant Edwards
<invalid@invalid.invalid> declaimed the following:

>On 2015-05-05, Sayth Renshaw <flebber.crue@gmail.com> wrote:
>
>> Just checking if the reaction to cry when given XML is normal.
>
>It depends on whether or not you're allowed to swear and throw things.

	That's reserved for someone stating they want to use regular
expressions to parse the XML.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#90008

On 5/5/2015 9:54 AM, Michael Torrie wrote:
> On 05/05/2015 03:28 AM, Sayth Renshaw wrote:
>> Hi
>>
>> Just checking if the reaction to cry when given XML is normal.
>
> I'd say it is normal.  XML is like violence.  If it doesn't solve your
> problems, you're not using enough of it[1].
>
> [1] Can anyone tell me who originated this line?
>

I find references from 2006 attributing the quote to "Ted Neward" but 
can't confirm that's where it started.

Emile

[toc] | [prev] | [next] | [standalone]

#90011

On 05/05/2015 10:28, Sayth Renshaw wrote:
> Hi
>
> Just checking if the reaction to cry when given XML is normal.
>
> I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission.
>
> See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .
>
> See its never basic xml, usually comes from some database with a walk of tables and strange relationships.
>
> Am I doing it wrong is there a simple way I am missing?
>
> Sayth
>

When the two young lads from the consultants I was working with back in 
2000 found a bug with xml handling in the supplier's software I almost 
ended up in tears.  Why?  They didn't bother reporting it, they simply 
modified all their unit testing data to work around the problem, so when 
we got to integration testing nothing worked at all.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#90015

On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> When the two young lads from the consultants I was working with back in 2000
> found a bug with xml handling in the supplier's software I almost ended up
> in tears.  Why?  They didn't bother reporting it, they simply modified all
> their unit testing data to work around the problem, so when we got to
> integration testing nothing worked at all.

See, that's not a problem with XML. That's not even a problem with
unit testing. That's a problem with the belief that the most important
thing is to have your tests pass. Sigh.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90017

On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
> > When the two young lads from the consultants I was working with back in 2000
> > found a bug with xml handling in the supplier's software I almost ended up
> > in tears.  Why?  They didn't bother reporting it, they simply modified all
> > their unit testing data to work around the problem, so when we got to
> > integration testing nothing worked at all.
> 
> See, that's not a problem with XML. That's not even a problem with
> unit testing. That's a problem with the belief that the most important
> thing is to have your tests pass. Sigh.
> 

I smell a PHB¹ in the wings...
-------
¹ Pointy-haired-boss

[toc] | [prev] | [next] | [standalone]

#90018

On Wed, May 6, 2015 at 10:19 AM, Rustom Mody <rustompmody@gmail.com> wrote:
> On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
>> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
>> > When the two young lads from the consultants I was working with back in 2000
>> > found a bug with xml handling in the supplier's software I almost ended up
>> > in tears.  Why?  They didn't bother reporting it, they simply modified all
>> > their unit testing data to work around the problem, so when we got to
>> > integration testing nothing worked at all.
>>
>> See, that's not a problem with XML. That's not even a problem with
>> unit testing. That's a problem with the belief that the most important
>> thing is to have your tests pass. Sigh.
>>
>
> I smell a PHB¹ in the wings...

It seems likely, yes. That's where the "c'mon guys, get those tests to
pass" impetus usually comes from. The geeks usually know that tests
are a tool, not a goal, although the more evil ones might still wield
that knowledge...

http://bofh.ntk.net/BOFH/1998/bastard98-35.php

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web