Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!feed.xsnews.nl!border-1.ams.xsnews.nl!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.045
X-Spam-Evidence: '*H*': 0.91; '*S*': 0.00; 'socket': 0.04; 'javascript,': 0.05; 'subject:Python': 0.05; 'python': 0.08; 'ajax,': 0.09; 'figuring': 0.09; 'server,': 0.12; '16,': 0.15; 'cookies': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'helps!': 0.16; 'packet': 0.16; 'python;': 0.16; 'simulate': 0.16; "that'll": 0.16; 'wrote:': 0.18; 'received:209.85.210.174': 0.21; 'received:mail- iy0-f174.google.com': 0.21; 'header:In-Reply-To:1': 0.22; 'basis,': 0.23; 'libraries': 0.24; 'load': 0.26; 'monitor': 0.28; 'script': 0.28; 'message-id:@mail.gmail.com': 0.29; 'made.': 0.29; 'pm,': 0.29; 'menu,': 0.30; 'headers': 0.32; 'actual': 0.32; 'there': 0.33; 'fri,': 0.34; 'running': 0.34; 'body.': 0.34; 'to:addr:python-list': 0.35; 'none': 0.35; '...': 0.35; 'be.': 0.35; 'form,': 0.35; 'connection': 0.36; 'post': 0.36; 'ajax': 0.37; 'http': 0.37; 'query': 0.37; 'page': 0.37; 'but': 0.37; 'received:google.com': 0.37; 'using': 0.37; 'another': 0.37; 'received:209.85': 0.38; 'some': 0.38; 'think': 0.38; 'easier': 0.38; 'difficult': 0.39; 'tasks': 0.39; 'received:209': 0.39; 'application': 0.40; 'user': 0.40; 'to:addr:python.org': 0.40; 'simple': 0.61; 'your': 0.61; 'hope': 0.61; 'browser': 0.62; 'here': 0.64; 'request.': 0.65; 'website': 0.65; 'visit': 0.67; 'connection,': 0.73; 'have.': 0.77; 'browser.': 0.80; 'application:': 0.84; 'replay': 0.84; 'sniffer': 0.84; 'tricky': 0.84; 'plugins': 0.91; 'redirect': 0.91; 'subject:activity': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=X/w5PxZiiw5m9P22IEINvVyMQ43TmP5/g1OUELVGxSU=; b=UA42UIm9l4iQSY+1eFayKJJZLqnfYTpDLc4iej7pS2nl3CfjwyWCp46NhZmlJVR7Gx 5GCcN+PkaL+plPfbl0pMTsIhNdZFJVYgC3HJLVFO4sehe4gNaF20hjFGfDcckibKIGYm Ja0m0718sR8iYYnkCcaWO9InrlfjQaF5CIWcv7K2EZF9lG/R+uP+uR719Qz9QmeQ3vRv 5E7et8+Mi+kaD7rq/CnSm1NAkwfQlkLqOiwxWtOpDlC6G8qM8TvO2O/mrlQaXnLsro2M gCQUO3VaP13/Yv42r3g+XrNf8/CHaZWFdnXTBqFp+2IpN6lIOwZ6kcDcdKGBik+NVL2W /rqw==
MIME-Version: 1.0
In-Reply-To: <214c4c0c-f8ec-4030-946b-8becc8e1aa9c@ur9g2000pbc.googlegroups.com>
References: <214c4c0c-f8ec-4030-946b-8becc8e1aa9c@ur9g2000pbc.googlegroups.com>
Date: Sat, 17 Mar 2012 06:20:17 +1100
Subject: Re: Python simulate browser activity
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.731.1331925620.3037.python-list@python.org>
Lines: 30
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1331925620 news.xs4all.nl 6909 [2001:888:2000:d::a6]:59123
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:21771

On Fri, Mar 16, 2012 at 1:23 PM, choi2k <rex.0510@gmail.com> wrote:
> The application aims to simulate user activity including visit a
> website and perform some interactive actions (click on the menu,
> submit a form, redirect to another pages...etc)
> I have found some libraries / plugins which aims to simulate browser
> activity but ... none of them support AJAX request and/or running
> javascript.
>
> Here is one of the simple tasks which I would like to do using the
> application:
> 4. post ajax request using javascript and if the page has been
> redirected, load the new page content

To the web server, everything is a request. Don't think in terms of
Javascript; there is no Javascript, there is no AJAX, there is not
even a web browser. There's just a TCP socket connection and an HTTP
request. That's all you have.

On that basis, your Python script most definitely can simulate the
request. It needs simply to make the same HTTP query that the
Javascript would have made. The only tricky part is figuring out what
that request would be. If you can use a non-SSL connection, that'll
make your job easier - just get a proxy or packet sniffer to monitor
an actual web browser, then save the request headers and body. You can
then replay that using Python; it's not difficult to handle cookies
and such, too.

Hope that helps!

ChrisA