Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #24441

RE: vBulletin scraper -- feasible?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <nick.cash@npcinternational.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.009
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'dynamically': 0.07; 'skip:p 60': 0.07; 'suppose': 0.07; 'feasible.': 0.09; 'nntp': 0.09; 'parsing': 0.09; 'received:10.7': 0.09; 'received:10.7.14': 0.09; 'received:216.32.180': 0.09; 'nick': 0.16; 'received:216.32.180.30': 0.16; 'message-----': 0.18; 'thanks,': 0.19; 'parse': 0.22; 'to:name:python-list@python.org': 0.23; 'header:In-Reply-To:1': 0.23; 'to:2**1': 0.23; "i'd": 0.24; 'wondering': 0.26; 'received:169.254': 0.27; 'andrew': 0.27; 'monday,': 0.27; 'subject: -- ': 0.27; 'url:software': 0.29; 'url:mailman': 0.29; 'hosted': 0.30; 'header:Received:8': 0.31; 'url:python': 0.31; 'received:169': 0.32; 'url:listinfo': 0.33; 'subject:?': 0.33; 'html,': 0.33; 'received:216.32': 0.33; 'skip:- 10': 0.34; 'server.': 0.35; 'feed': 0.35; 'to:addr:python-list': 0.35; 'url:org': 0.35; 'charset:us-ascii': 0.36; 'subject:': 0.37; 'subject:: ': 0.37; 'email addr:python.org': 0.38; 'sent:': 0.38; 'received:10': 0.38; 'to:addr:python.org': 0.39; 'from:': 0.39; 'quite': 0.39; 'easy': 0.60; 'received:216': 0.61; 'email name :python-list': 0.63; 'goal': 0.64; 'taking': 0.67; 'nice,': 0.84; 'vbulletin': 0.84; 'ultimate': 0.93
X-Forefront-Antispam-Report CIP:157.56.244.229; KIP:(null); UIP:(null); IPV:NLI; H:CH1PRD0610HT005.namprd06.prod.outlook.com; RD:none; EFVD:NLI
X-SpamScore -11
X-BigFish PS-11(z5d3Mz542M4015Izz1202hzz8275bh8275dhz2fh2a8h668h839h944hd25hf0ah)
Received-SPF pass (mail108-va3: domain of npcinternational.com designates 157.56.244.229 as permitted sender) client-ip=157.56.244.229; envelope-from=nick.cash@npcinternational.com; helo=CH1PRD0610HT005.namprd06.prod.outlook.com ; .outlook.com ;
From Nick Cash <nick.cash@npcinternational.com>
To Andrew D'Angelo <excel@pharcyde.org>, "python-list@python.org" <python-list@python.org>
Subject RE: vBulletin scraper -- feasible?
Thread-Topic vBulletin scraper -- feasible?
Thread-Index AQHNUwbfJwzu+AVSfU2zvAjf3Vt3LZcLb18g
Date Mon, 25 Jun 2012 19:44:10 +0000
References <jsad2l$2eu$1@dont-email.me>
In-Reply-To <jsad2l$2eu$1@dont-email.me>
Accept-Language en-US
Content-Language en-US
X-MS-Has-Attach
X-MS-TNEF-Correlator
x-originating-ip [70.166.238.194]
Content-Type text/plain; charset="us-ascii"
Content-Transfer-Encoding quoted-printable
MIME-Version 1.0
X-OriginatorOrg npcinternational.com
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.1495.1340654376.4697.python-list@python.org> (permalink)
Lines 26
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1340654376 news.xs4all.nl 6968 [2001:888:2000:d::a6]:47540
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:24441

Show key headers only | View raw

You may want to look into http://www.crummy.com/software/BeautifulSoup/
It's made for parsing (potentially bad) HTML, and is quite easy to use. I'd say it's quite feasible.

Thanks,
Nick Cash
NPC International

-----Original Message-----
From: python-list-bounces+nick.cash=npcinternational.com@python.org [mailto:python-list-bounces+nick.cash=npcinternational.com@python.org] On Behalf Of Andrew D'Angelo
Sent: Monday, June 25, 2012 14:10
To: python-list@python.org
Subject: vBulletin scraper -- feasible?

Taking a look through vBulletin's HTML, I was wondering whether it would be overly difficult to parse it into nice, manipulatible data.
I'd suppose my ultimate goal would be to dynamically parse a vBulletin and feed it into a locally hosted NNTP server. 


--
http://mail.python.org/mailman/listinfo/python-list

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

vBulletin scraper -- feasible? "Andrew D'Angelo" <excel@pharcyde.org> - 2012-06-25 14:10 -0500
  RE: vBulletin scraper -- feasible? Nick Cash <nick.cash@npcinternational.com> - 2012-06-25 19:44 +0000
    Re: vBulletin scraper -- feasible? "Andrew D'Angelo" <excel@pharcyde.org> - 2012-06-26 10:20 -0500

csiph-web