Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #52656 > unrolled thread

crawling/parsing a webpage based on dynamic javascript

Back to article view | Back to comp.lang.python

Contents

  crawling/parsing a webpage based on dynamic javascript bruce <badouglas@gmail.com> - 2013-08-18 08:40 -0400

#52656 — crawling/parsing a webpage based on dynamic javascript

[Multipart message — attachments visible in raw view] — view raw

Hi.

Looking at using python/cerely/twisted to test in parsing a test site. Also
looking at being able to parse a site created using dynamic javascript.

I've got test apps to parse a site, but I'm interested in getting a better
understanding of using multi-thread/multi-processing approaches to spin out
as many fetch processes as possible.

At the same time, I'm interested in understanding a bit better what's used
for parsing the javascript pages in the py world.

Also, rather than just point me to something like "scrapy", I'm actually
interested in finding someone who's done this that I can talk to.

Heck, for the right person, I'll even toss some cash your way!!

Thanks

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web