Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35321

Re: Scrapy/XPath help

Show key headers only | View raw

You might have better luck if you share the python make, version, os,
error message, and some unit tests demonstrating what you expect.

On Fri, Dec 21, 2012 at 3:21 PM, Always Learning <cbrowning@ou.edu> wrote:
> Hello all. I'm new to Python, but have been playing around with it for a few weeks now, following tutorials, etc. I've spun off on my own and am trying to do some basic web scraping. I've used Firebug/View XPath in Firefox for some help with the XPaths, however, I still am receiving errors when I try to run this script. If you could help, it would be greatly appreciated!
>
> from scrapy.spider import BaseSpider
> from scrapy.selector import HtmlXPathSelector
> from cbb_info.items import CbbInfoItem, Field
>
> class GameInfoSpider(BaseSpider):
>     name = "game_info"
>     allowed_domains = ["www.sbrforum.com"]
>     start_urls = [
>         'http://www.sbrforum.com/betting-odds/ncaa-basketball/',
>         ]
>
>     def parse(self, response):
>         hxs = HtmlXPathSelector(response)
>         toplevels = hxs.select("//div[@class='eventLine-value']")
>         items = []
>         for toplevels in toplevels:
>             item = CbbInfoItem()
>             item ["teams"] = toplevels.select("/span[@class='team-name'/text()").extract()
>             item ["lines"] = toplevels.select("/div[@rel='19']").extract()
>             item.append(item)
>         return items
> --
> http://mail.python.org/mailman/listinfo/python-list



-- 
Grant Rettke | ACM, AMA, COG, IEEE
grettke@acm.org | http://www.wisdomandwonder.com/
Wisdom begins in wonder.
((λ (x) (x x)) (λ (x) (x x)))

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Scrapy/XPath help Always Learning <cbrowning@ou.edu> - 2012-12-21 13:21 -0800
  Re: Scrapy/XPath help Grant Rettke <grettke@acm.org> - 2012-12-21 15:34 -0600
    Re: Scrapy/XPath help Always Learning <cbrowning@ou.edu> - 2012-12-21 13:58 -0800
      Re: Scrapy/XPath help Dave Angel <d@davea.name> - 2012-12-21 22:18 -0500
      Re: Scrapy/XPath help donarb <donarb@nwlink.com> - 2012-12-25 11:15 -0800
      Re: Scrapy/XPath help donarb <donarb@nwlink.com> - 2012-12-25 11:15 -0800
    Re: Scrapy/XPath help Always Learning <cbrowning@ou.edu> - 2012-12-21 13:58 -0800

csiph-web