Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!bcyclone01.am1.xlned.com!bcyclone01.am1.xlned.com!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <jerry.rocteur@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.005
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; ';-)': 0.03; 'tree': 0.05; 'indices': 0.07; '%s"': 0.09; 'api': 0.11; 'python': 0.11; 'suggest': 0.14; 'blocks': 0.16; 'ignoring': 0.16; 'looked': 0.18; 'module': 0.19; 'trying': 0.19; 'import': 0.22; '(in': 0.22; 'print': 0.22; 'parse': 0.24; "haven't": 0.24; 'source': 0.25; 'this:': 0.26; 'tutorials': 0.26; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'requests': 0.31; 'subject: (': 0.35; 'classes': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'done': 0.36; 'thanks': 0.36; 'hi,': 0.36; 'e.g.': 0.38; 'to:addr:python- list': 0.38; 'to:addr:python.org': 0.39; 'users': 0.40; 'how': 0.40; 'skip:u 10': 0.60; 'easy': 0.60; 'most': 0.60; 'different': 0.65; 'url:jpeg': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=++dMxsiEuPHVi4GcfT1ft0PS2QFRUkBPb50nsYKuxtg=; b=JHDHBAK+m6xWYBWFwkF80PeYGiNmROuKLBojT2QcZ6tmi7VcRNDEMY4ojZNbkZgvO8 988e1aIrtn5ETCYYmltK3GQPYqXHBUsvbsK5lI3LCRlPZvJgGtsBmR1hBGoHRPJhDm5Z 39QC5oCADon/K1z/mgxW8nTzTaQ8Pn5N9E1Y/LoqSeCLcpX5pnWc5jo1udmleyo2D5// B7NOSElFIS+TXSHne6gkGelXJ9fSBw9LIU9gNr2srbE1q5ypINuRHEevE44MTKQzT1nq HhdO7CMEKnJMQThG9S/Ks4U1ir9uy+O7yo5wmoF5IELcGGe6Fz1u6v3euKP1XvXLEqlB nisQ==
X-Received: by 10.152.191.165 with SMTP id gz5mr14915428lac.37.1421582877367; Sun, 18 Jan 2015 04:07:57 -0800 (PST)
MIME-Version: 1.0
From: Jerry Rocteur <jerry.rocteur@gmail.com>
Date: Sun, 18 Jan 2015 13:07:37 +0100
Subject: Trying to parse matchup.io (lxml, SGMLParser, urlparse)
To: python-list@python.org
Content-Type: text/plain; charset=UTF-8
X-Mailman-Approved-At: Sun, 18 Jan 2015 17:11:00 +0100
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.17835.1421597461.18130.python-list@python.org>
Lines: 71
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1421597461 news.xs4all.nl 2914 [2001:888:2000:d::a6]:37604
X-Complaints-To: abuse@xs4all.nl
X-Received-Bytes: 4641
X-Received-Body-CRC: 2931073960
Xref: csiph.com comp.lang.python:83988

Hi,

I'm trying to parse https://matchup.io/players/rocteur/friends

The body source I'm interested in contains blocks exactly like this

<tr class='friend'>
<td class='text--left'>
<a href="/players/mizucci0"><img alt="mizucci0" class="media__avatar"
src="https://matchup-io.s3.amazonaws.com/uploads/player/avatar/7651/7651_profile_150_square.jpeg"
/>
<div class='friend__info'>
<span>mizucci0</span>
<span>Mizuho</span>
</div>
</a></td>
<td class='delta-alt'>
29,646
<br>
steps
</td>
<td class='delta-alt'>
35,315
<br>
steps
</td>
<td class='delta-alt'>
818.7
<br>
Miles
</td>
</tr>

I wanted to do it Python as I'm learning and I looked at the different
modules but it isn't easy for me to work out the best way to do this
as most tutorials I see use complicated classes and I just want to
parse this one paragraph at a time (as I would do in Perl) and print

1 mizuho 26648 35315
2 xxxxxx  99999 99999
3 xxxxxx 99999 99999

etc. (in the above case I'm ignoring 818.7 and Miles.

The best way I found so far is this:

from lxml import html
import requests
page = requests.get("https://matchup.io/players/rocteur/friends/week/")
tree = html.fromstring(page.text)
a = tree.xpath('//span/text()')
b = tree.xpath('//td/text()')

And the manipulating indices

e.g.
print "%s %s %s %s" % (a[usern], a[users], b[tots], b[weekb])
    tots += 4
    weekb += 4
    usern += 2
    users += 2

But it isn't very scientific ;-)

Which module would you use and how would you suggest is the best way to do it ?

Thanks very much in advance, I haven't done a lot of HTML parsing.. I
would much prefer using WebServices and an API but unfortunately they
don't have it.
-- 
Jerry Rocteur