using urllib on a more complex site

Newsgroups	comp.lang.python
Date	2013-02-24 16:02 -0800
Message-ID	<e3e061a0-493b-4b8b-992b-a175dbecd4ac@googlegroups.com> (permalink)
Subject	using urllib on a more complex site
From	"Adam W." <AWasilenko@gmail.com>

Show all headers | View raw

I'm trying to write a simple script to scrape http://www.vudu.com/movies/#tag/99centOfTheDay/99c%20Rental%20of%20the%20day

in order to send myself an email every day of the 99c movie of the day.

However, using a simple command like (in Python 3.0): 
urllib.request.urlopen('http://www.vudu.com/movies/#tag/99centOfTheDay/99c%20Rental%20of%20the%20day').read()

I don't get the all the source I need, its just the navigation buttons.  Now I assume they are using some CSS/javascript witchcraft to load all the useful data later, so my question is how do I make urllib "wait" and grab that data as well?

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

using urllib on a more complex site "Adam W." <AWasilenko@gmail.com> - 2013-02-24 16:02 -0800
  Re: using urllib on a more complex site Chris Rebert <clp2@rebertia.com> - 2013-02-24 16:27 -0800
    Re: using urllib on a more complex site "Adam W." <AWasilenko@gmail.com> - 2013-02-24 17:32 -0800
    Re: using urllib on a more complex site "Adam W." <AWasilenko@gmail.com> - 2013-02-24 17:32 -0800
  Re: using urllib on a more complex site Dave Angel <davea@davea.name> - 2013-02-24 19:30 -0500
    Re: using urllib on a more complex site "Adam W." <AWasilenko@gmail.com> - 2013-02-24 17:28 -0800
    Re: using urllib on a more complex site "Adam W." <AWasilenko@gmail.com> - 2013-02-24 17:28 -0800

csiph-web