Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #72066 > unrolled thread

Verify JSON Data

Back to article view | Back to comp.lang.python

Contents

  Verify JSON Data gaurangnshah@gmail.com - 2014-05-26 07:26 -0700
    Re: Verify JSON Data Roy Smith <roy@panix.com> - 2014-05-26 11:19 -0400
      Re: Verify JSON Data Gene Heskett <gheskett@shentel.net> - 2014-05-26 11:37 -0400
        Re: Verify JSON Data Roy Smith <roy@panix.com> - 2014-05-26 11:55 -0400
          Re: Verify JSON Data Chris Angelico <rosuav@gmail.com> - 2014-05-27 02:02 +1000
          Re: Verify JSON Data Gene Heskett <gheskett@shentel.net> - 2014-05-26 12:43 -0400
      Re: Verify JSON Data Chris Angelico <rosuav@gmail.com> - 2014-05-27 01:58 +1000
      Re: Verify JSON Data Chris Angelico <rosuav@gmail.com> - 2014-05-27 02:00 +1000
    Re: Verify JSON Data Burak Arslan <burak.arslan@arskom.com.tr> - 2014-05-26 17:28 +0200
    Re: Verify JSON Data Denis McMahon <denismfmcmahon@gmail.com> - 2014-05-26 15:43 +0000
    Re: Verify JSON Data Gene Heskett <gheskett@shentel.net> - 2014-05-26 12:44 -0400
    Re: Verify JSON Data Grant Edwards <invalid@invalid.invalid> - 2014-05-26 18:35 +0000
      Re: Verify JSON Data Rustom Mody <rustompmody@gmail.com> - 2014-05-26 21:28 -0700

#72066 — Verify JSON Data

Hi Guys, 

Would someone let me know how to verify JSON data in python. There are so many modules available to verify XML file, however i didn't find any good module to verify JSON Data. 

After searching on the internet i came across JSON module, however it only coverts the JSON data to python. it's good, however the problem comes when JSON response is very large.  

Is there any module through which i can verify JSON file like DOM or Object oriented way. ( i.e. data.key)

[toc] | [next] | [standalone]

#72069

In article <e26d3f14-ac97-4abd-bdfc-699d9ed2175c@googlegroups.com>,
 gaurangnshah@gmail.com wrote:

> Hi Guys, 
> 
> Would someone let me know how to verify JSON data in python. There are so 
> many modules available to verify XML file, however i didn't find any good 
> module to verify JSON Data. 

Python comes with a built-in json module.  Just use json.load() or 
json.loads() to parse your JSON data.  The first call reads from a 
string, the second on from a file, but in all other ways, they're 
identical.

There are a bunch of third-party modules (ujson, etc) which are faster, 
but fundamentally, they're all the same.

If I understand you correctly, you're reading a JSON document which is 
so large that if you store the converted data as a Python object, you 
run out of memory?  If that's the case, I'm not sure if there's a good 
pure Python solution.  I don't know of any json modules which parse, but 
don't store, the data.

Depending on what operating system you're on, there may be a 
command-line utility which parse JSON.  For example, on Ubuntu linux, 
there's "json_xs".  Perhaps shell out to that, use the "-t null" output 
format, redirect the output to /dev/null, and see what exit status you 
get:

# Good JSON
$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
0

# Bad JSON
$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
255

Wrap this up in a subprocess.check_output() call, and you're done.

[toc] | [prev] | [next] | [standalone]

#72072

On Monday 26 May 2014 11:19:53 Roy Smith did opine
And Gene did reply:
> In article <e26d3f14-ac97-4abd-bdfc-699d9ed2175c@googlegroups.com>,
> 
>  gaurangnshah@gmail.com wrote:
> > Hi Guys,
> > 
> > Would someone let me know how to verify JSON data in python. There
> > are so many modules available to verify XML file, however i didn't
> > find any good module to verify JSON Data.
> 
> Python comes with a built-in json module.  Just use json.load() or
> json.loads() to parse your JSON data.  The first call reads from a
> string, the second on from a file, but in all other ways, they're
> identical.
> 
> There are a bunch of third-party modules (ujson, etc) which are faster,
> but fundamentally, they're all the same.
> 
> If I understand you correctly, you're reading a JSON document which is
> so large that if you store the converted data as a Python object, you
> run out of memory?  If that's the case, I'm not sure if there's a good
> pure Python solution.  I don't know of any json modules which parse,
> but don't store, the data.
> 
> Depending on what operating system you're on, there may be a
> command-line utility which parse JSON.  For example, on Ubuntu linux,
> there's "json_xs".  Perhaps shell out to that, use the "-t null" output
> format, redirect the output to /dev/null, and see what exit status you
> get:
> 
> # Good JSON
> $ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 0
> 
> # Bad JSON
> $ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 255
> 
> Wrap this up in a subprocess.check_output() call, and you're done.

Just for S&G, and without checking the version numbers of anything, this 
may not be all that bulletproof a test:

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127

Old, buntu 10.04.4 LTS system, all up to date security patches wise.  
kernal 3.13.9, PAE on a quad core phenom.

Interesting result.  Source of error? DamnedifIknow.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

[toc] | [prev] | [next] | [standalone]

#72073

In article <mailman.10347.1401119046.18130.python-list@python.org>,
 Gene Heskett <gheskett@shentel.net> wrote:

> > $ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 0

> > $ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 255


> gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 127
> gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 127

I don't see what the problem is.  On average, we got the same result :-)

[toc] | [prev] | [next] | [standalone]

#72076

On Tue, May 27, 2014 at 1:55 AM, Roy Smith <roy@panix.com> wrote:
> In article <mailman.10347.1401119046.18130.python-list@python.org>,
>  Gene Heskett <gheskett@shentel.net> wrote:
>
>> > $ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
>> 0
>
>> > $ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
>> 255
>
>
>> gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
>> 127
>> gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
>> 127
>
> I don't see what the problem is.  On average, we got the same result :-)

Ahh but if you were using Python 3, those averages would be 127.5 each.

ChrisA

[toc] | [prev] | [next] | [standalone]

#72077

On Monday 26 May 2014 11:55:29 Roy Smith did opine
And Gene did reply:
> In article <mailman.10347.1401119046.18130.python-list@python.org>,
> 
>  Gene Heskett <gheskett@shentel.net> wrote:
> > > $ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> > 
> > 0
> > 
> > > $ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> > 
> > 255
> > 
> > 
> > gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo
> > $? 127
> > gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo
> > $? 127
> 
> I don't see what the problem is.  On average, we got the same result
> :-)

If I was still smoking Roy, I'd ask for a hit on whatever you are having. 
:)

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

[toc] | [prev] | [next] | [standalone]

#72074

On Tue, May 27, 2014 at 1:37 AM, Gene Heskett <gheskett@shentel.net> wrote:
> Just for S&G, and without checking the version numbers of anything, this
> may not be all that bulletproof a test:
>
> gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 127
> gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
> 127
>
> Old, buntu 10.04.4 LTS system, all up to date security patches wise.
> kernal 3.13.9, PAE on a quad core phenom.
>
> Interesting result.  Source of error? DamnedifIknow.
>

Return value 127 might well mean that json_xs isn't installed. It's
very difficult for a non-program to tell you whether JSON is valid or
not :) So I'd be checking 'which json_xs' before continuing.

ChrisA

[toc] | [prev] | [next] | [standalone]

#72075

On Tue, May 27, 2014 at 1:19 AM, Roy Smith <roy@panix.com> wrote:
> Python comes with a built-in json module.  Just use json.load() or
> json.loads() to parse your JSON data.  The first call reads from a
> string, the second on from a file, but in all other ways, they're
> identical.

Minor nit-pick: they're the other way around - load() reads from a
file and loads() reads from a string. I wouldn't bother commenting,
except that load() could plausibly mean "load from string", and "'str'
object has no attribute 'read'" might be a bit of a surprise else :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#72070

On 26/05/14 16:26, gaurangnshah@gmail.com wrote:
> Hi Guys,
>
> Would someone let me know how to verify JSON data in python. There are so many modules available to verify XML file, however i didn't find any good module to verify JSON Data.
>

Hi,

Spyne re-implements (a useful subset of) Xml Schema validation so that 
it can be applied to other document formats like json. It's 'soft' 
validation in Spyne's terms.

http://spyne.io

Disclosure: I'm the author of Spyne and starting to feel like I'm 
talking a little bit too much about my project on this list :)

Hth,
Burak

[toc] | [prev] | [next] | [standalone]

#72071

On Mon, 26 May 2014 07:26:20 -0700, gaurangnshah wrote:

> Is there any module through which i can verify JSON file like DOM or
> Object oriented way. ( i.e. data.key)

Where is the json data coming from? What do you mean by verify?

https://docs.python.org/2/library/json.html#encoders-and-decoders

explains how json object strings get decoded to python data types. A json 
object string should at the highest level be either an object or an 
array, although the python decoder can also handle strings, numbers and a 
few special values.

Are you trying to check that the json string is valid json code (ie json 
lint) or are you trying to check that it meets some specific structure, 
in which case the only way to verify it is to decode it and check the 
structure.

Note that not all valid python structures can be successfully converted 
to json objects, for example a python dictionary can have tuples as keys, 
but a json object can not have an array as an attribute name. For example:

d = { (1,2,3):'one',('a','b','c'):'two' }
print d
print json.JSONEncoder().encode( d )

Gives a TypeError in the json code "keys must be a string"

If you have a debian based linux distro, you can get jsonlint with:

sudo apt-get install python-demjson

which provides a command line json syntax checker and formatter. 
Otherwise, google json lint, there are several web based tools that seem 
to be able to do something similar.

-- 
Denis McMahon, denismfmcmahon@gmail.com

[toc] | [prev] | [next] | [standalone]

#72078

On Monday 26 May 2014 11:58:06 Chris Angelico did opine
And Gene did reply:
> On Tue, May 27, 2014 at 1:37 AM, Gene Heskett <gheskett@shentel.net> 
wrote:
> > Just for S&G, and without checking the version numbers of anything,
> > this may not be all that bulletproof a test:
> > 
> > gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo
> > $? 127
> > gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo
> > $? 127
> > 
> > Old, buntu 10.04.4 LTS system, all up to date security patches wise.
> > kernal 3.13.9, PAE on a quad core phenom.
> > 
> > Interesting result.  Source of error? DamnedifIknow.
> 
> Return value 127 might well mean that json_xs isn't installed. It's
> very difficult for a non-program to tell you whether JSON is valid or
> not :) So I'd be checking 'which json_xs' before continuing.
> 
> ChrisA

And locate comes back empty. So much for that. ;-)

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

[toc] | [prev] | [next] | [standalone]

#72082

On 2014-05-26, gaurangnshah@gmail.com <gaurangnshah@gmail.com> wrote:

> Would someone let me know how to verify JSON data in python.

Parse the file into a data structure with whatever parser you like,
then write a program to go thorugh the data structure and verify it.

> There are so many modules available to verify XML file, however i
> didn't find any good module to verify JSON Data.

XML has various "schema" languages which can be used to write a
definition of what is valid and what isn't valid.

There really is anything like that in widespread use for JSON.

> After searching on the internet i came across JSON module, however it
> only coverts the JSON data to python. it's good, however the problem
> comes when JSON response is very large.  

What's the problem?

> Is there any module through which i can verify JSON file like DOM or
> Object oriented way. ( i.e. data.key)

I don't know what you're asking.

-- 
Grant Edwards               grant.b.edwards        Yow! Actually, what I'd
                                  at               like is a little toy
                              gmail.com            spaceship!!

[toc] | [prev] | [next] | [standalone]

#72100

On Tuesday, May 27, 2014 12:05:58 AM UTC+5:30, Grant Edwards wrote:
> On 2014-05-26, gaurang shah  wrote:
> > Would someone let me know how to verify JSON data in python.

> Parse the file into a data structure with whatever parser you like,
> then write a program to go thorugh the data structure and verify it.

> > There are so many modules available to verify XML file, however i
> > didn't find any good module to verify JSON Data.

> XML has various "schema" languages which can be used to write a
> definition of what is valid and what isn't valid.

> There really is anything like that in widespread use for JSON.

Google offers:

https://pypi.python.org/pypi/jsonschema
http://python-jsonschema.readthedocs.org/en/latest/validate/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web