Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #88410 > unrolled thread

New to Programming - XML Processing

Back to article view | Back to comp.lang.python

Contents

  New to Programming - XML Processing catperson <me@anonymous.invalid> - 2015-03-31 20:27 -0700
    Re: New to Programming - XML Processing Ben Finney <ben+python@benfinney.id.au> - 2015-04-01 15:08 +1100
    Re: New to Programming - XML Processing Rustom Mody <rustompmody@gmail.com> - 2015-03-31 21:17 -0700
      Re: New to Programming - XML Processing Andrew Farrell <amfarrell@mit.edu> - 2015-03-31 23:27 -0500
      Re: New to Programming - XML Processing Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-04-01 09:00 +0100
      Re: New to Programming - XML Processing catperson <me@anonymous.invalid> - 2015-04-01 15:33 -0700
        Re: New to Programming - XML Processing sohcahtoa82@gmail.com - 2015-04-01 17:46 -0700
    Re: New to Programming - XML Processing Rustom Mody <rustompmody@gmail.com> - 2015-03-31 21:49 -0700
    Re: New to Programming - XML Processing Andrew Farrell <amfarrell@mit.edu> - 2015-03-31 23:26 -0500
    Re: New to Programming - XML Processing Burak Arslan <burak.arslan@arskom.com.tr> - 2015-04-01 16:44 +0300

#88410 — New to Programming - XML Processing

I am new to programming, though not new to computers.  I'm looking to
teach myself Python 3 and am working my way through a tutorial.  At
the point I'm at in the tutorial I am tasked with parsing out an XML
file created with a Garmin Forerunner and am just having a terrible
time getting my head around the concepts.  What I'm looking for is
some suggested reading that might give me some of the theory of
operation behind ElementTree and then how to parse out specific
elements.  Most of what I have been able to find in examples that I
can understand use very simplistic XML files and this Garmin file is
many levels of sub-elements and some of those elements have attributes
assigned, like <Activity Sport="Running">.

I'm hoping with enough reading I can experiment and work my way
through the problem and end up with a hopefully clear understanding of
the ElementTree module and Dictionairies.  

Thanks for any suggestions in advance.

Jim.

[toc] | [next] | [standalone]

#88412

catperson <me@anonymous.invalid> writes:

> I am new to programming, though not new to computers. I'm looking to
> teach myself Python 3 and am working my way through a tutorial.

Congratulations! Python 3 is a fine language to be your first.

Which tutorial are you following? It may be relevant.

> At the point I'm at in the tutorial I am tasked with parsing out an
> XML file created with a Garmin Forerunner and am just having a
> terrible time getting my head around the concepts.

Have you worked with manipulating XML documents in the past?

If I describe XML as a machine-readable hierarchical data serialisation
format, does that sound like noise or are you comfortable that you
understand that description?

> What I'm looking for is some suggested reading that might give me some
> of the theory of operation behind ElementTree and then how to parse
> out specific elements.

You'll need to understand the principles behind XML, then. Warning: it
is not simple! XML is machine-readable, but rarely pleasant for *humans*
to read or understand.

> I'm hoping with enough reading I can experiment and work my way
> through the problem and end up with a hopefully clear understanding of
> the ElementTree module and Dictionairies.

XML is a standard – a large number of standards, really – from the W3
Consortium. They have a good overview of the “core” of XML
<URL:http://www.w3.org/standards/xml/core>.

There are countless XML formats though; XML is a markup language to use
as a basis for constructing data serialisation formats. You need to know
what the specific format (called a “schema”) is before you can expect to
make sense of it, and so in order to understand what's in the document
you need a description of the specific schema from whoever wrote the
document.

General XML tutorials do exist; here is one comprehensive one
<URL:http://www.xmlobjective.com/category/guide/>. You'll likely want to
learn more about the specifics of whatever XML format is being used,
though, before going much beyond the basic principles.

-- 
 \            “Politics is not the art of the possible. It consists in |
  `\       choosing between the disastrous and the unpalatable.” —John |
_o__)                                    Kenneth Galbraith, 1962-03-02 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#88413

On Wednesday, April 1, 2015 at 8:57:15 AM UTC+5:30, catperson wrote:
> I am new to programming, though not new to computers.  I'm looking to
> teach myself Python 3 and am working my way through a tutorial.  At
> the point I'm at in the tutorial I am tasked with parsing out an XML
> file created with a Garmin Forerunner and am just having a terrible
> time getting my head around the concepts.  What I'm looking for is
> some suggested reading that might give me some of the theory of
> operation behind ElementTree and then how to parse out specific
> elements.  Most of what I have been able to find in examples that I
> can understand use very simplistic XML files and this Garmin file is
> many levels of sub-elements and some of those elements have attributes
> assigned, like <Activity Sport="Running">.
> 
> I'm hoping with enough reading I can experiment and work my way
> through the problem and end up with a hopefully clear understanding of
> the ElementTree module and Dictionairies.  
> 
> Thanks for any suggestions in advance.

Suggestions:
1. Learn to use the interpreter interactively; ie (at the least)¹ ie

a. Start up python (without a program)
b. Play around with trivial expressions
c. Explore introspective features - help(), type() dir()

2. Do you know about triple-quoted strings?
a. Start small (or trivial) sub-parts of your XML as triple-quoted examples in the 
interpreter and start throwing them at elementtree
b. If they dont work trivialize further; if they work add complexity
-----------
¹ At the least because environments like Idle are more conducive to such playing

[toc] | [prev] | [next] | [standalone]

#88415

[Multipart message — attachments visible in raw view] — view raw

You should follow Rustom's advice before just diving into the blog post I
linked to. Otherwise you risk blindly following things and losing your
bearings when you run into bugs.

On Tue, Mar 31, 2015 at 11:17 PM, Rustom Mody <rustompmody@gmail.com> wrote:

> On Wednesday, April 1, 2015 at 8:57:15 AM UTC+5:30, catperson wrote:
> > I am new to programming, though not new to computers.  I'm looking to
> > teach myself Python 3 and am working my way through a tutorial.  At
> > the point I'm at in the tutorial I am tasked with parsing out an XML
> > file created with a Garmin Forerunner and am just having a terrible
> > time getting my head around the concepts.  What I'm looking for is
> > some suggested reading that might give me some of the theory of
> > operation behind ElementTree and then how to parse out specific
> > elements.  Most of what I have been able to find in examples that I
> > can understand use very simplistic XML files and this Garmin file is
> > many levels of sub-elements and some of those elements have attributes
> > assigned, like <Activity Sport="Running">.
> >
> > I'm hoping with enough reading I can experiment and work my way
> > through the problem and end up with a hopefully clear understanding of
> > the ElementTree module and Dictionairies.
> >
> > Thanks for any suggestions in advance.
>
> Suggestions:
> 1. Learn to use the interpreter interactively; ie (at the least)¹ ie
>
> a. Start up python (without a program)
> b. Play around with trivial expressions
> c. Explore introspective features - help(), type() dir()
>
> 2. Do you know about triple-quoted strings?
> a. Start small (or trivial) sub-parts of your XML as triple-quoted
> examples in the
> interpreter and start throwing them at elementtree
> b. If they dont work trivialize further; if they work add complexity
> -----------
> ¹ At the least because environments like Idle are more conducive to such
> playing
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#88417

On 01/04/2015 05:27, Andrew Farrell wrote:
> You should follow Rustom's advice before just diving into the blog post
> I linked to. Otherwise you risk blindly following things and losing your
> bearings when you run into bugs.
>

Sound advice, but would you please be kind enough to intersperse your 
answers or bottom post as top posting is heavily frowned upon here.

TIA.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#88434

On Tue, 31 Mar 2015 21:17:38 -0700 (PDT), Rustom Mody
<rustompmody@gmail.com> wrote:

>On Wednesday, April 1, 2015 at 8:57:15 AM UTC+5:30, catperson wrote:
>> I am new to programming, though not new to computers.  I'm looking to
>> teach myself Python 3 and am working my way through a tutorial.  At
>> the point I'm at in the tutorial I am tasked with parsing out an XML
>> file created with a Garmin Forerunner and am just having a terrible
>> time getting my head around the concepts.  What I'm looking for is
>> some suggested reading that might give me some of the theory of
>> operation behind ElementTree and then how to parse out specific
>> elements.  Most of what I have been able to find in examples that I
>> can understand use very simplistic XML files and this Garmin file is
>> many levels of sub-elements and some of those elements have attributes
>> assigned, like <Activity Sport="Running">.
>> 
>> I'm hoping with enough reading I can experiment and work my way
>> through the problem and end up with a hopefully clear understanding of
>> the ElementTree module and Dictionairies.  
>> 
>> Thanks for any suggestions in advance.
>
>Suggestions:
>1. Learn to use the interpreter interactively; ie (at the least)¹ ie
>
>a. Start up python (without a program)
>b. Play around with trivial expressions
>c. Explore introspective features - help(), type() dir()
>
>2. Do you know about triple-quoted strings?
>a. Start small (or trivial) sub-parts of your XML as triple-quoted examples in the 
>interpreter and start throwing them at elementtree
>b. If they dont work trivialize further; if they work add complexity
>-----------
>¹ At the least because environments like Idle are more conducive to such playing

I thank everyone for their feedback.  I like the above advice and also
Rustom's other comments about learning dictionaries.  I'm thinking in
the back of my mind my issue might be more around the dictionary than
ElementTree at this point.  

 I'll fill in my situation a little based on the queries back to me.

I'm using a book, Python Programming Fundamentals by Kent D. Lee.  

http://knuth.luther.edu/~leekent/IntroToComputing/ 

It looks like he has updated his website since I last looked at it.  I
know there are many tutorials out there, however, I picked this one
and resolved to complete it start to finish.  That means a lot of side
research and reading, which I'm happy to do.  

I've got somewhat of a handle on the basic XML parsing process (my
opinion).  The book uses minidom, but my reading suggests that
ElementTree is a better option, so I thought I'd switch to that and
attempt to duplicate the exercise.

I understand this bit;

import xml.etree.ElementTree as etree
tree = etree.parse('workout.tcx')
root = tree.getroot()

and when I experiment and type root[1][1][0] in a console I can get
the ID field from the xml file with a start time for the workout. That
tells me it parsed properly.  I'm having trouble getting my head
around iterating over the elements, pulling out an activity element
with an attribute of either running or biking, then getting multiple
Trackpoint elements generated by the ForeRunner every 2 minutes, with
an ultimate goal of creating a graph using turtle graphics (later in
the exercise).

I initially thought my trouble was with parsing the xml, but now I'm
thinking my problem is I didn't pay enough attention to iterating over
a dictionary.  I will focus my efforts on Rustom's suggestions so
please don't give me a solution.  Some of my background is automotive
technician, and a statement by one of my instructors has stuck in my
mind over the years.  He said, a friends car won't start, do you take
booster cables or a can of gas with you?  If you understand the theory
of operation of an internal combustion engine, you can solve almost
any problem.

This is a small sample of the xml file.  It's 170239 lines currently.

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<TrainingCenterDatabase
xmlns="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.garmin.com/xmlschemas/ActivityExtension/v2
http://www.garmin.com/xmlschemas/ActivityExtensionv2.xsd
http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2
http://www.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd">

  <Folders/>

  <Activities>
    <Activity Sport="Running">
      <Id>2011-06-06T12:21:04Z</Id>
      <Lap StartTime="2011-06-06T12:21:04Z">
        <TotalTimeSeconds>3283.1700000</TotalTimeSeconds>
        <DistanceMeters>4821.6821289</DistanceMeters>
        <MaximumSpeed>2.8058963</MaximumSpeed>
        <Calories>398</Calories>
        <AverageHeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
          <Value>125</Value>
        </AverageHeartRateBpm>
        <MaximumHeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
          <Value>167</Value>
        </MaximumHeartRateBpm>
        <Intensity>Active</Intensity>
        <TriggerMethod>Manual</TriggerMethod>
        <Track>
          <Trackpoint>
            <Time>2011-06-06T12:21:04Z</Time>
            <Position>
              <LatitudeDegrees>34.5225040</LatitudeDegrees>
              <LongitudeDegrees>-77.3563351</LongitudeDegrees>
            </Position>
            <AltitudeMeters>-0.3433838</AltitudeMeters>
            <DistanceMeters>0.0000000</DistanceMeters>
            <HeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
              <Value>116</Value>
            </HeartRateBpm>
            <SensorState>Absent</SensorState>
          </Trackpoint>

Jim.

[toc] | [prev] | [next] | [standalone]

#88438

On Wednesday, April 1, 2015 at 3:34:15 PM UTC-7, catperson wrote:
> On Tue, 31 Mar 2015 21:17:38 -0700 (PDT), Rustom Mody
> <rustompmody@gmail.com> wrote:
> 
> >On Wednesday, April 1, 2015 at 8:57:15 AM UTC+5:30, catperson wrote:
> >> I am new to programming, though not new to computers.  I'm looking to
> >> teach myself Python 3 and am working my way through a tutorial.  At
> >> the point I'm at in the tutorial I am tasked with parsing out an XML
> >> file created with a Garmin Forerunner and am just having a terrible
> >> time getting my head around the concepts.  What I'm looking for is
> >> some suggested reading that might give me some of the theory of
> >> operation behind ElementTree and then how to parse out specific
> >> elements.  Most of what I have been able to find in examples that I
> >> can understand use very simplistic XML files and this Garmin file is
> >> many levels of sub-elements and some of those elements have attributes
> >> assigned, like <Activity Sport="Running">.
> >> 
> >> I'm hoping with enough reading I can experiment and work my way
> >> through the problem and end up with a hopefully clear understanding of
> >> the ElementTree module and Dictionairies.  
> >> 
> >> Thanks for any suggestions in advance.
> >
> >Suggestions:
> >1. Learn to use the interpreter interactively; ie (at the least)š ie
> >
> >a. Start up python (without a program)
> >b. Play around with trivial expressions
> >c. Explore introspective features - help(), type() dir()
> >
> >2. Do you know about triple-quoted strings?
> >a. Start small (or trivial) sub-parts of your XML as triple-quoted examples in the 
> >interpreter and start throwing them at elementtree
> >b. If they dont work trivialize further; if they work add complexity
> >-----------
> >š At the least because environments like Idle are more conducive to such playing
> 
> I thank everyone for their feedback.  I like the above advice and also
> Rustom's other comments about learning dictionaries.  I'm thinking in
> the back of my mind my issue might be more around the dictionary than
> ElementTree at this point.  
> 
>  I'll fill in my situation a little based on the queries back to me.
> 
> I'm using a book, Python Programming Fundamentals by Kent D. Lee.  
> 
> http://knuth.luther.edu/~leekent/IntroToComputing/ 
> 
> It looks like he has updated his website since I last looked at it.  I
> know there are many tutorials out there, however, I picked this one
> and resolved to complete it start to finish.  That means a lot of side
> research and reading, which I'm happy to do.  
> 
> I've got somewhat of a handle on the basic XML parsing process (my
> opinion).  The book uses minidom, but my reading suggests that
> ElementTree is a better option, so I thought I'd switch to that and
> attempt to duplicate the exercise.
> 
> I understand this bit;
> 
> import xml.etree.ElementTree as etree
> tree = etree.parse('workout.tcx')
> root = tree.getroot()
> 
<snip>
>               <LatitudeDegrees>34.5225040</LatitudeDegrees>
>               <LongitudeDegrees>-77.3563351</LongitudeDegrees>
<snip>
> Jim.

This is on the coast of North Carolina, for anyone curious, just south of Snead's Ferry.

[toc] | [prev] | [next] | [standalone]

#88414

On Wednesday, April 1, 2015 at 8:57:15 AM UTC+5:30, catperson wrote:
> I'm hoping with enough reading I can experiment and work my way
> through the problem and end up with a hopefully clear understanding of
> the ElementTree module and Dictionairies.  

Also:
If you are not familiar with dictionaries, better grok them
(and lists and tuples and some else) from a good tut like

https://docs.python.org/3.4/tutorial/

BEFORE taking on ElementTree

[toc] | [prev] | [next] | [standalone]

#88416

[Multipart message — attachments visible in raw view] — view raw

> I am new to programming, though not new to computers.

One quick tip: when starting a new project, it is sometimes is useful to
see if others have done the same thing and use their approach to guide
yours. In your case, googling "Garmin Forerunner xml python" results in
this blog post which is recent enough that it should be helpful.

http://www.nezirteke.com/2014/02/using-python-to-plot-garmin-forerunner.html

Note that coming up with good phrases for google to help you with a
programming task is a skill that you get better at with time.

Welcome to Python! ^-^

On Tue, Mar 31, 2015 at 11:08 PM, Ben Finney <ben+python@benfinney.id.au>
wrote:

> catperson <me@anonymous.invalid> writes:
>
> > I am new to programming, though not new to computers. I'm looking to
> > teach myself Python 3 and am working my way through a tutorial.
>
> Congratulations! Python 3 is a fine language to be your first.
>
> Which tutorial are you following? It may be relevant.
>
> > At the point I'm at in the tutorial I am tasked with parsing out an
> > XML file created with a Garmin Forerunner and am just having a
> > terrible time getting my head around the concepts.
>
> Have you worked with manipulating XML documents in the past?
>
> If I describe XML as a machine-readable hierarchical data serialisation
> format, does that sound like noise or are you comfortable that you
> understand that description?
>
> > What I'm looking for is some suggested reading that might give me some
> > of the theory of operation behind ElementTree and then how to parse
> > out specific elements.
>
> You'll need to understand the principles behind XML, then. Warning: it
> is not simple! XML is machine-readable, but rarely pleasant for *humans*
> to read or understand.
>
> > I'm hoping with enough reading I can experiment and work my way
> > through the problem and end up with a hopefully clear understanding of
> > the ElementTree module and Dictionairies.
>
> XML is a standard – a large number of standards, really – from the W3
> Consortium. They have a good overview of the “core” of XML
> <URL:http://www.w3.org/standards/xml/core>.
>
> There are countless XML formats though; XML is a markup language to use
> as a basis for constructing data serialisation formats. You need to know
> what the specific format (called a “schema”) is before you can expect to
> make sense of it, and so in order to understand what's in the document
> you need a description of the specific schema from whoever wrote the
> document.
>
> General XML tutorials do exist; here is one comprehensive one
> <URL:http://www.xmlobjective.com/category/guide/>. You'll likely want to
> learn more about the specifics of whatever XML format is being used,
> though, before going much beyond the basic principles.
>
> --
>  \            “Politics is not the art of the possible. It consists in |
>   `\       choosing between the disastrous and the unpalatable.” —John |
> _o__)                                    Kenneth Galbraith, 1962-03-02 |
> Ben Finney
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#88424

On 04/01/15 06:27, catperson wrote:
> I am new to programming, though not new to computers.  I'm looking to
> teach myself Python 3 and am working my way through a tutorial.  At
> the point I'm at in the tutorial I am tasked with parsing out an XML
> file created with a Garmin Forerunner and am just having a terrible
> time getting my head around the concepts.  What I'm looking for is
> some suggested reading that might give me some of the theory of
> operation behind ElementTree and then how to parse out specific
> elements.  Most of what I have been able to find in examples that I
> can understand use very simplistic XML files and this Garmin file is
> many levels of sub-elements and some of those elements have attributes
> assigned, like <Activity Sport="Running">.

As everybody loves objects, there are many libraries that parse xml
documents to regular python objects (with proper hierarchy). These are
supposed to save you from dealing with ElementTree api directly.

Have a look here:

http://stackoverflow.com/questions/19545067/python-joining-and-writing-xml-etrees-trees-stored-in-a-list

Hth,
Burak

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web