Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #45818 > unrolled thread

Accessing Json data (I think I am nearly there) complete beginner

Back to article view | Back to comp.lang.python

Contents

  Accessing Json data (I think I am nearly there) complete beginner Andrew Edwards-Adams <aeaprog@gmail.com> - 2013-05-23 09:09 -0700
    Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Berg <bahamutzero8825@gmail.com> - 2013-05-23 11:40 -0500
      Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Edwards-Adams <aeaprog@gmail.com> - 2013-05-23 09:58 -0700
        Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Berg <bahamutzero8825@gmail.com> - 2013-05-23 12:59 -0500
        Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Berg <bahamutzero8825@gmail.com> - 2013-05-23 13:11 -0500
          Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Edwards-Adams <aeaprog@gmail.com> - 2013-05-23 11:19 -0700
            Re: Accessing Json data (I think I am nearly there) complete beginner Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-05-23 19:56 +0100
              Re: Accessing Json data (I think I am nearly there) complete beginner Andrew Edwards-Adams <aeaprog@gmail.com> - 2013-05-23 14:30 -0700
    Re: Accessing Json data (I think I am nearly there) complete beginner MRAB <python@mrabarnett.plus.com> - 2013-05-23 17:53 +0100

#45818 — Accessing Json data (I think I am nearly there) complete beginner

Hey guys
I think its worth stating that I have been trying to code for 1 week. 
I am trying to access some Json data. My innitial code was the below: 

"import mechanize
import urllib
import re
 
def getData():  
    post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/all_time"
    browser = mechanize.Browser()
    browser.set_handle_robots(False)
    browser.addheaders = [('User-agent', 'Firefox')]
 
    #These are the parameters you've got from checking with the aforementioned tools
    parameters = {'page' : '1',
                  'rp' : '10',
                  'sortname' : 'total_pl',
                  'sortorder' : 'desc'}
    #Encode the parameters
    data = urllib.urlencode(parameters)
    trans_array = browser.open(post_url,data).read().decode('UTF-8')
 
    #print trans_array

    myfile = open("test.txt", "w")
    myfile.write(trans_array)
    myfile.close()

getData()
 
raw_input("Complete")"

I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything. I think the guy that recommended me this method must have got something wrong? Or perhaps I am simply incompetent: 

import mechanize
import urllib
import json
def getData():  
    post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
    browser = mechanize.Browser()
    browser.set_handle_robots(False)
    browser.addheaders = [('User-agent', 'Firefox')]
 
    #These are the parameters you've got from checking with the aforementioned tools
    parameters = {'page' : '1',
                  'rp' : '50',
                  'sortname' : 'total_pl',
                  'sortorder' : 'desc'
                 }
    #Encode the parameters
    data = urllib.urlencode(parameters)
    trans_array = browser.open(post_url,data).read().decode('UTF-8')
 
    text1 = json.loads(trans_array)
    print text1['rows'][0]['id']  #play around with these values to access different data..
   
getData()

He told me to "#play around with these values to access different data.." really cant get anything out of this, any ideas? 

Many thanks AEA

[toc] | [next] | [standalone]

#45823

On 2013.05.23 11:09, Andrew Edwards-Adams wrote:
> I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything.
Where exactly is the problem? Do you not get JSON back? Do you get the wrong values? Do you get a KeyError or IndexError trying to get
values from text1? Are there gremlins going around flipping bits in memory? It's good that you posted code, but "really cant get anything
out of this" isn't very useful.
-- 
CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

[toc] | [prev] | [next] | [standalone]

#45825

On Thursday, May 23, 2013 5:40:49 PM UTC+1, Andrew Berg wrote:
> On 2013.05.23 11:09, Andrew Edwards-Adams wrote:
> 
> > I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything.
> 
> Where exactly is the problem? Do you not get JSON back? Do you get the wrong values? Do you get a KeyError or IndexError trying to get
> 
> values from text1? Are there gremlins going around flipping bits in memory? It's good that you posted code, but "really cant get anything
> 
> out of this" isn't very useful.
> 
> -- 
> 
> CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX. I was told that the second piece of code is the more efficient way to pull data from the JSON. When I say it doesnt do anything I can adjust the areas suggested. But it returns nothing, I just get the "= Restart =" line appear in the python shell. If there was a trackback/debug I might know where to look, but its not yielding errors, its simply yielding nothing. What i dont know, is if it is the code that isnt working, or what I am inputting in the " print text1['rows'][0]['id']" that isnt working.


Since this code: 

import mechanize
import urllib
import json
def getData():  
    post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
    browser = mechanize.Browser()
    browser.set_handle_robots(False)
    browser.addheaders = [('User-agent', 'Firefox')]
 
    #These are the parameters you've got from checking with the aforementioned tools
    parameters = {'page' : '1',
                  'rp' : '50',
                  'sortname' : 'total_pl',
                  'sortorder' : 'desc'
                 }
    #Encode the parameters
    data = urllib.urlencode(parameters)
    trans_array = browser.open(post_url,data).read().decode('UTF-8')

 appears in both examples and my first attempt is definitely returning data from the JSON, I can only assume that the error is in the last two lines of his code, or as stated my imputing incorrect parameters.

:S

[toc] | [prev] | [next] | [standalone]

#45828

On 2013.05.23 11:58, Andrew Edwards-Adams wrote:
> If there was a trackback/debug I might know where to look, but its not yielding errors, its simply yielding nothing. What i dont know, is if it is the code that isnt working, or what I am inputting in the " print text1['rows'][0]['id']" that isnt working.
If fed a valid JSON object, json.loads() will return a regular dictionary. You can print (or pretty print with the pprint module) text1 to
see everything. If you're not familiar with dictionaries and lists, thoroughly read the tutorial before writing or modifying any more code.
http://docs.python.org/2/library/json.html#json.loads
-- 
CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

[toc] | [prev] | [next] | [standalone]

#45829

On 2013.05.23 11:58, Andrew Edwards-Adams wrote:
> Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX.
It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very
useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.
-- 
CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

[toc] | [prev] | [next] | [standalone]

#45830

On Thursday, May 23, 2013 7:11:28 PM UTC+1, Andrew Berg wrote:
> On 2013.05.23 11:58, Andrew Edwards-Adams wrote:
> 
> > Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX.
> 
> It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very
> 
> useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.
> 
> -- 
> 
> CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

Thanks Andrew, yes I believe this is what the guy who provided me with the code was thinking. I was about to embark on what was definitely going to be an inefficient long regex.

[toc] | [prev] | [next] | [standalone]

#45834

On 23/05/2013 19:19, Andrew Edwards-Adams wrote:
> On Thursday, May 23, 2013 7:11:28 PM UTC+1, Andrew Berg wrote:
>> On 2013.05.23 11:58, Andrew Edwards-Adams wrote:
>>
>>> Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX.
>>
>> It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very
>>
>> useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.
>>
>> --
>>
>> CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1
>
> Thanks Andrew, yes I believe this is what the guy who provided me with the code was thinking. I was about to embark on what was definitely going to be an inefficient long regex.
>

Funny old world, there doesn't appear to be working code but someone is 
already thinking about definite inefficiency.  What evidence do you have 
to support this claim?

-- 
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#45847

On Thursday, May 23, 2013 7:56:19 PM UTC+1, Mark Lawrence wrote:
> On 23/05/2013 19:19, Andrew Edwards-Adams wrote:
> 
> > On Thursday, May 23, 2013 7:11:28 PM UTC+1, Andrew Berg wrote:
> 
> >> On 2013.05.23 11:58, Andrew Edwards-Adams wrote:
> 
> >>
> 
> >>> Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX.
> 
> >>
> 
> >> It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very
> 
> >>
> 
> >> useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.
> 
> >>
> 
> >> --
> 
> >>
> 
> >> CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1
> 
> >
> 
> > Thanks Andrew, yes I believe this is what the guy who provided me with the code was thinking. I was about to embark on what was definitely going to be an inefficient long regex.
> 
> >
> 
> 
> 
> Funny old world, there doesn't appear to be working code but someone is 
> 
> already thinking about definite inefficiency.  What evidence do you have 
> 
> to support this claim?
> 
> 
> 
> -- 
> 
> If you're using GoogleCrap™ please read this 
> 
> http://wiki.python.org/moin/GoogleGroupsPython.
> 
> 
> 
> Mark Lawrence

haha true Mark, I have made mistakes like this before become ok at coding VBA to achieve things in the most inefficient manner possible. Something that I will learn from for the current python coding project.

[toc] | [prev] | [next] | [standalone]

#45824

On 23/05/2013 17:09, Andrew Edwards-Adams wrote:
> Hey guys
> I think its worth stating that I have been trying to code for 1 week.
> I am trying to access some Json data. My innitial code was the below:
>
> "import mechanize
> import urllib
> import re
>
> def getData():
>      post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/all_time"
>      browser = mechanize.Browser()
>      browser.set_handle_robots(False)
>      browser.addheaders = [('User-agent', 'Firefox')]
>
>      #These are the parameters you've got from checking with the aforementioned tools
>      parameters = {'page' : '1',
>                    'rp' : '10',
>                    'sortname' : 'total_pl',
>                    'sortorder' : 'desc'}
>      #Encode the parameters
>      data = urllib.urlencode(parameters)
>      trans_array = browser.open(post_url,data).read().decode('UTF-8')
>
>      #print trans_array
>
>      myfile = open("test.txt", "w")
>      myfile.write(trans_array)
>      myfile.close()
>
> getData()
>
> raw_input("Complete")"
>
> I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything. I think the guy that recommended me this method must have got something wrong? Or perhaps I am simply incompetent:
>
> import mechanize
> import urllib
> import json
> def getData():
>      post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
>      browser = mechanize.Browser()
>      browser.set_handle_robots(False)
>      browser.addheaders = [('User-agent', 'Firefox')]
>
>      #These are the parameters you've got from checking with the aforementioned tools
>      parameters = {'page' : '1',
>                    'rp' : '50',
>                    'sortname' : 'total_pl',
>                    'sortorder' : 'desc'
>                   }
>      #Encode the parameters
>      data = urllib.urlencode(parameters)
>      trans_array = browser.open(post_url,data).read().decode('UTF-8')
>
>      text1 = json.loads(trans_array)
>      print text1['rows'][0]['id']  #play around with these values to access different data..
>
> getData()
>
> He told me to "#play around with these values to access different data.." really cant get anything out of this, any ideas?
>
> Many thanks AEA
>
I've just tried it. It prints "1048".

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web