Re: get google scholar using python

From	Grant Edwards <invalid@invalid.invalid>
Newsgroups	comp.lang.python
Subject	Re: get google scholar using python
Date	2012-10-01 17:19 +0000
Organization	PANIX Public Access Internet and UNIX, NYC
Message-ID	<k4cjbb$h4d$1@reader1.panix.com> (permalink)
References	<1349109859.27817.7.camel@roddur> <mailman.1710.1349110326.27098.python-list@python.org>

Show all headers | View raw

On 2012-10-01, Nick Cash <nick.cash@npcinternational.com> wrote:
>> urllib2.urlopen('http://scholar.google.co.uk/scholar?q=albert
>>...
>> urllib2.HTTPError: HTTP Error 403: Forbidden
>> 
>> Will you kindly explain me the way to get rid of this?
>
> Looks like Google blocks non-browser user agents from retrieving this
> query. You *could* work around it by setting the User-Agent header to
> something fake that looks browser-ish, but you're almost certainly
> breaking Google's TOS if you do so.

I don't know about that particular Google service, but Google often
provides an API that's intended for use by non-browser programs. 
Those interfaces are usually both easier to use for the programmer and
impose less load on the servers.

-- 
Grant Edwards               grant.b.edwards        Yow! I am deeply CONCERNED
                                  at               and I want something GOOD
                              gmail.com            for BREAKFAST!

Thread

get google scholar using python রুদ্র ব্যাণার্জী  <bnrj.rudra@gmail.com> - 2012-10-01 17:44 +0100
  RE: get google scholar using python Nick Cash <nick.cash@npcinternational.com> - 2012-10-01 16:51 +0000
    Re: get google scholar using python Grant Edwards <invalid@invalid.invalid> - 2012-10-01 17:19 +0000
    Re: get google scholar using python রুদ্র ব্যাণার্জী  <bnrj.rudra@gmail.com> - 2012-10-01 18:28 +0100
      Re: get google scholar using python Jerry Hill <malaclypse2@gmail.com> - 2012-10-01 14:09 -0400

csiph-web