Re: Automatically driving a browser

Path	csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!news.mixmin.net!news2.arglkargh.de!news.karotte.org!uucp.gnuu.de!newsfeed.arcor.de!newsspool3.arcor-online.net!news.arcor.de.POSTED!not-for-mail
Content-Type	text/plain; charset="UTF-8"
Message-ID	<6373251.ZMB7bVNzcP@PointedEars.de> (permalink)
From	Thomas 'PointedEars' Lahn <PointedEars@web.de>
Reply-To	Thomas 'PointedEars' Lahn <usenet@PointedEars.de>
Organization	PointedEars Software (PES)
Date	Wed, 27 Feb 2013 16:45:57 +0100
User-Agent	KNode/4.8.5
Content-Transfer-Encoding	8Bit
X-Face	%i>XG-yXR'\"2P/C_aO%~;2o~?g0pPKmbOw^=NT`tprDEf++D.m7"}HW6.#=U:?2GGctkL,f89@H46O$ASoW&?s}.k+&.<b';Md8`dH6iqhT)6C^.Px\|[=M@7=Ik[_w<%n1Up"LPQNu2m8\|L!/3iby{-]A+#YE}Kl{Cw$\U!kD%K}\2jz"QQP6Uqr],./"?;=4v
Face	iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEXTxa4RFk5dUWANED8PFEfy7+MGBiW+n3ZNF/QuAAACaElEQVQ4jVXUwVOcMBQG8Dc7Rc4PUntdWV2uxjDpGaGeozOp1woar4jd5t/v9wLstMwsA/ntlxdCAgUc1hjTc9/JCZfGoo3wG3HdmdAWrIJRHe7GM/TmpY5VFefuVcAkkPbLIaN8rmPmjloyZxgyR3GuJ4K0AGtJ2htz8o7yqikm759fldQXaMpbDzjKAG+8v+AugVTOPO5DOjLvGtUYQwh0CPjnVMyGd+8/GfUB5nLKJDD2aLDh5HYyMDJGDwQIo2ZmZcKbowNmAdB/AzyFhrmF2MHRb0QJJfaAnwGB6orZhoykLzJtGwF/xpYxI1dswomiUj3gTuAIqCn/4C7cULwGNBtwMTk3Y4LfKB5YUaOKBKYtpplm7u0vip8tU1NWWyI/7XdcSuIDoMt6rVHMWT0DbjHPGqDqZVSa6zleLcUTcIKLoMv3ueJluALtAo9B302zPPlrtiVScRdCjXvVh3e3JpYa/jjkuC9N+LrBMlz/eAN4eQijX2EdLo6c5tGGHwLyHFtXk89dDGHwCVhG9T0S/j55AhRZgkMCmUQXJ49TnS1wnQDvw0eAh9ICeMmEFbCnPMFzjAvsWoEWEFdYEx+S0MoUZ1gT1wId8+AF3Bl2OoEu906AUHx5VLw/gXYg/x84loOah/2UYNrgiwSwGO7RfUzVBbx/kgpckumGOi6QirtD6gkLTitbnxNol47S2jVc2vsN5kPqaAHT8uUdAJM4v/DanjYOwmUjWznGfwB7sGtAtor5BgofDuzaRj4kSQAqDakTsKORa3Q3xKi3gE1fhl71KRMqrdZ2AWNNg/YOhQyrVBnb+i+nEg4bsDA+egAAAABJRU5ErkJggg==
Subject	Re: Automatically driving a browser
Newsgroups	comp.lang.javascript
References	<pBE*2C4su@news.chiark.greenend.org.uk>
MIME-Version	1.0
Lines	47
NNTP-Posting-Date	27 Feb 2013 16:45:58 CET
NNTP-Posting-Host	a05dacbb.newsspool3.arcor-online.net
X-Trace	DXC=o?Q=hba4D9R1`E>oC;JXEZMcF=Q^Z^V3X4Fo<]lROoRQ8kF<OcfhCO[jWdeJAGglJQDZm8W4\YJN\b@mF9jNikdU3>YBYU3JEo]TE8mlPRo^NV
X-Complaints-To	usenet-abuse@arcor.de
Xref	csiph.com comp.lang.javascript:18651

Show key headers only | View raw

Theo Markettos wrote:

> Is there any means of programmatically operating a browser - for example,
> inspecting the DOM, click on links it finds, waiting for the next page to
> load, enter some text into a form, submit it, etc etc?  By this I mean the
> browser is operating as normal (with a UI) not as a perl script that's
> doing HTTP itself.

In theory, there is the possibility that you can access documents in other 
frames, you can call access properties of element objects, and you can 
create events and propagate them to elements.  In practice, that possibility 
is limited by the Same Origin Policy (SOP).

However, it is possible to use a non-standard host environment, such as a 
user profile for testing (and only for that) with sufficiently low (but not 
lower) security settings.  I suspect that it also could be done with a 
browser extension since that is running in a privileged context.

Or you could perhaps run all foreign traffic through a transparent HTTP 
proxy so that the SOP is not an issue despite strong security settings.

The simplest thing to do is, of course, to use the script console on the 
site that you are testing.  I have not yet tried what you could call “cross-
site driving”, though.

> I've seen things like Google Scraper which poke about in the HTML
> hierarchy, but do tools/scripts for actually operating a browser as well
> exist?

Probably yes.  Have you searched for them?

> I'm guessing Javascript is the most likely starting point,

There is no Javascript.  <http://PointedEars.de/es-matrix>

> since it has access to the DOM already, though any other language is fine
> too.  Am happy to be tied to a specific browser, it doesn't need to be
> generic.

In that case Firefox or in general Gecko-based environments are the most 
likely candidates.

-- 
PointedEars

Twitter: @PointedEars2
Please do not Cc: me. / Bitte keine Kopien per E-Mail.

Back to comp.lang.javascript | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Automatically driving a browser Theo Markettos <theom+news@chiark.greenend.org.uk> - 2013-02-27 14:48 +0000
  Re: Automatically driving a browser Christoph Becker <cmbecker69@gmx.de> - 2013-02-27 15:53 +0100
    Re: Automatically driving a browser Theo Markettos <theom+news@chiark.greenend.org.uk> - 2013-02-28 00:34 +0000
      Re: Automatically driving a browser Stefan Weiss <krewecherl@gmail.com> - 2013-02-28 17:33 +0100
  Re: Automatically driving a browser Bjoern Hoehrmann <bjoern@hoehrmann.de> - 2013-02-27 15:57 +0100
    Re: Automatically driving a browser Theo Markettos <theom+news@chiark.greenend.org.uk> - 2013-02-27 15:14 +0000
      Re: Automatically driving a browser Eric Bednarz <bednarz@fahr-zur-hoelle.org> - 2013-02-27 22:38 +0100
        Re: Automatically driving a browser Theo Markettos <theom+news@chiark.greenend.org.uk> - 2013-02-28 00:43 +0000
  Re: Automatically driving a browser Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-02-27 16:45 +0100

csiph-web