Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Dan Strohl <D.Strohl@F5.com>
Newsgroups: comp.lang.python
Subject: RE: Controlling the passing of data
Date: Thu, 28 Apr 2016 14:40:01 +0000
Lines: 133
Message-ID: <mailman.190.1461854474.32212.python-list@python.org>
References: <afa1ab1d-6d6a-421c-9351-c8805cc5fbea@googlegroups.com> <nft3s8$3ku$1@ger.gmane.org> <4a16db24a11c4231b3d006bcca10f502@seaexchmbx03.olympus.F5Net.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Thread-Topic: Controlling the passing of data
Thread-Index: AQHRoUZuWhIhx+A/60mF+gywqyncuZ+fZP9ugAAEVtA=
In-Reply-To: <nft3s8$3ku$1@ger.gmane.org>
Accept-Language: en-US
Content-Language: en-US
Precedence: list
Xref: csiph.com comp.lang.python:107793

In addition to Peter's points,=20
- I would suggest breaking out the list comprehensions into standard for lo=
ops and/or functions.  That makes it easier to read and troubleshoot.  (you=
 can always re-optimize It if needed.)
- Peter's point about making things into functions will also help troublesh=
ooting.  You can better test the data going into and out of the function.  =
In my code, I will often have the file access and data processing as separa=
te functions, then the main routine just calls the function to get data, pa=
sses that data to a function that manipulates it, and then pass the results=
 to a function that writes it out.  This allows for much easier testing and=
 troubleshooting of the individual functions.  (sometimes I will reassemble=
 them into one function when I am done, depending on my needs)

More importantly though in terms of your getting help for your problem, the=
 post is unclear (to me at least) in terms of what you are trying to achiev=
e and what isn't working,=20

Try considering the following suggestions:

- It is unclear what the problem is that you are having, you say you are tr=
ying to do x, and you are having problems in a part after a comment, but no=
ne of the comments say "this is breaking".  I assume you are talking about =
the comment that starts "Here I need to traverse", but don't know for sure,=
 and even if it is, you don't specify what the problem actually is;  are yo=
u receiving an exception message (please let us know what it is), is it run=
ning, but not doing anything?  is it returning incorrect data? or???

- When posting, rather than commenting out code that you aren't using right=
 now, and code that is working and not related to the problem, I recommend =
just deleting them so that people don't have to try to work through it.... =
for example, just remove the GetArgs function and just say fileList =3D "/x=
ml_dir", and the section at the end that is all commented out, just remove =
it.  You should just have the minimum needed to replicate the problem. =20

This will also help in troubleshooting, when I have a problem like this, an=
d I can't figure out what is going on, I will copy the code to a new file a=
nd make a program that will handle a specific set of data, and try to do th=
e one thing that is breaking, removing all the rest of the stuff, and test =
the results.. so, for example, copy a sample of the xml with a couple of da=
ta items into a string var, and have a program that processes that and chec=
ks to see if at the end you end up with a list of the right values by print=
ing a list rather than muddying the waters with file access, and writing ou=
t csv's.  (by the way, this is a great time to start working with unit test=
ing if you aren't already, it is simple to create this as a test case and y=
ou will find that if you start doing testing along the way, the time it tak=
es to troubleshoot errors along the way will go down dramatically.)

- Be clear at the end what you expect to get, especially if it is not what =
you are getting... so, either in the code as a comment, or in a descriptive=
 paragraph, have a section that said something like:  "At the end of the sn=
ippet, meetdata, racedata, clubdata, and raceid should be a list of diction=
aries with the data from the xml" (or whatever... possibly with an example =
of what you would expect).  This is even more important if the problem you =
are having is that the code is not returning correct data.  This may not be=
 as needed if the code is simply blowing up at line xx, though it would sti=
ll help people understand your goal.

- For the example at least (you may choose to do differently in your live c=
ode), use nice explanatory variable names and don't rename imports, so it w=
ould be clearer to say "import pandas", then "frames =3D pandas.DataFrame[]=
)".  That way the reader doesn't have to keep referring to the imports to f=
igure out what is going on.

Remember, you are asking a large number of people for help, most of which a=
re pretty busy already, so the more you can do to simplify and show the exa=
ct problem, the more (and more useful) help you are likely to receive.  To =
this lists credit, even if you are completely unclear in your question, you=
 will likely get *something* back, (as you saw with Peters response), but w=
hat you get back is more likely to be a general suggestion rather than a sp=
ecific fix for your problem.

Dan Strohl




> -----Original Message-----
> From: Python-list [mailto:python-list-bounces+d.strohl=3Df5.com@python.or=
g]
> On Behalf Of Peter Otten
> Sent: Thursday, April 28, 2016 6:40 AM
> To: python-list@python.org
> Subject: Re: Controlling the passing of data
>=20
> Sayth Renshaw wrote:
>=20
> > In my file here I needed to traverse and modify the XML file I don't
> > want to restore it or put it in a new variable or other format I just
> > want to alter it and let it flow onto the list comprehensions as they w=
ere.
>=20
> That looks like an arbitrary limitation to me. It's a bit like "I want to=
 repair my
> car with this big hammer".
>=20
> > In particular here I am taking the id from race and putting it into
> > the children of each race called nomination.
> >
> > I have put a comment above the new code which is causing the difficulty=
.
>=20
> Your actual problem is drowned in too much source code. Can you restate i=
t
> in English, optionally with a few small snippets of Python?
>=20
> It is not even clear what the code you provide should accomplish once it'=
s
> running as desired.
>=20
> To give at least one code-related advice: You have a few repetitions of t=
he
> following structure
>=20
> > meetattrs =3D ('id', 'venue', 'date', 'rail', 'weather',
> > 'trackcondition')
>=20
> >     meet =3D d('meeting')
>=20
> >     meetdata =3D [[meet.eq(i).attr(x)
> >                  for x in meetattrs] for i in range(len(meet))]
>=20
> You should move the pieces into a function that works for meetings, clubs=
,
> races, and so on. Finally (If I am repeating myself so be it): the occure=
nce of
> range(len(something)) in your code is a strong indication that you are no=
t
> using Python the way Guido intended it. Iterate over the `something` dire=
ctly
> whenever possible.
>=20
> --
> https://mail.python.org/mailman/listinfo/python-list