Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #46367 > unrolled thread

detect key conflict in a JSON file

Back to article view | Back to comp.lang.python

Contents

  detect key conflict in a JSON file Jabba Laci <jabba.laci@gmail.com> - 2013-05-29 13:16 +0200
    Re: detect key conflict in a JSON file Roy Smith <roy@panix.com> - 2013-05-29 09:25 -0400
      Re: detect key conflict in a JSON file Jabba Laci <jabba.laci@gmail.com> - 2013-05-29 15:41 +0200
        Re: detect key conflict in a JSON file rusi <rustompmody@gmail.com> - 2013-05-29 07:36 -0700
      Re: detect key conflict in a JSON file Ervin Hegedüs <airween@gmail.com> - 2013-05-29 16:04 +0200
      Re: detect key conflict in a JSON file Roy Smith <roy@panix.com> - 2013-05-29 11:20 -0400
        Re: detect key conflict in a JSON file Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-05-30 03:10 +0000

#46367 — detect key conflict in a JSON file

Hi,

How can you detect if a key is duplicated in a JSON file? Example:

{
    "something": [...],
    ...
    "something": [...]
}

I have a growing JSON file that I edit manually and it might happen
that I repeat a key. If this happens, I would like to get notified.
Currently the value of the second key silently overwrites the value of
the first.

Do you know about a command line JSON validator?

Thanks,

Laszlo

[toc] | [next] | [standalone]

#46377

In article <mailman.2347.1369826248.3114.python-list@python.org>,
 Jabba Laci <jabba.laci@gmail.com> wrote:
 
> I have a growing JSON file that I edit manually and it might happen
> that I repeat a key. If this happens, I would like to get notified.

The real answer here is that JSON is probably not the best choice for 
large files that get hand-edited.  For data that you intend to hand-edit 
a lot, YAML might be a better choice.

> Currently the value of the second key silently overwrites the value of
> the first.

Yeah, that's what I would expect.  http://www.json.org/ is mute on the 
question of what to do with duplicate keys, but I would be quite 
surprised to discover any implementation which behaved differently.

> Do you know about a command line JSON validator?

All JSON implementations validate their input.  You are assuming that 
having duplicate keys is "invalid".  I would think it falls more into 
the category of "undefined behavior".

[toc] | [prev] | [next] | [standalone]

#46378

> The real answer here is that JSON is probably not the best choice for
> large files that get hand-edited.  For data that you intend to hand-edit
> a lot, YAML might be a better choice.
>
>> Currently the value of the second key silently overwrites the value of
>> the first.

Thanks but how would it be different with YAML? Can YAML check duplicate keys?

How to process (read) YAML files in Python? Can you give me some hints
how to get started? All I need is read it in and create a dictionary
of it.

Thanks,

Laszlo

[toc] | [prev] | [next] | [standalone]

#46385

On May 29, 6:41 pm, Jabba Laci <jabba.l...@gmail.com> wrote:
> > The real answer here is that JSON is probably not the best choice for
> > large files that get hand-edited.  For data that you intend to hand-edit
> > a lot, YAML might be a better choice.
>
> >> Currently the value of the second key silently overwrites the value of
> >> the first.
>
> Thanks but how would it be different with YAML? Can YAML check duplicate keys?
>
> How to process (read) YAML files in Python? Can you give me some hints
> how to get started? All I need is read it in and create a dictionary
> of it.
>
> Thanks,
>
> Laszlo

There seems to be a suggested patch for duplicate-detection here
http://pyyaml.org/ticket/128

[toc] | [prev] | [next] | [standalone]

#46380

Hello,

On Wed, May 29, 2013 at 03:41:50PM +0200, Jabba Laci wrote:
> > The real answer here is that JSON is probably not the best choice for
> > large files that get hand-edited.  For data that you intend to hand-edit
> > a lot, YAML might be a better choice.
> >
> >> Currently the value of the second key silently overwrites the value of
> >> the first.
> 
> Thanks but how would it be different with YAML? Can YAML check duplicate keys?

no, I think it can't,

I'm using yaml a few months ago, as I noticed, the last key
prevail.


> How to process (read) YAML files in Python? Can you give me some hints
> how to get started? All I need is read it in and create a dictionary
> of it.


import yaml


struct = yaml.load(file("yamlfile.yml" 'r'))



and struct will be a Python dictionary,



As I know, yaml modul is not part of standard Python library, but
most Linux systems has package, eg. Debian/Ubuntu has a
"python-yaml".



Cheers:


a.

[toc] | [prev] | [next] | [standalone]

#46391

[Multipart message — attachments visible in raw view] — view raw

On May 29, 2013, at 9:41 AM, Jabba Laci wrote:

>> The real answer here is that JSON is probably not the best choice for
>> large files that get hand-edited.  For data that you intend to hand-edit
>> a lot, YAML might be a better choice.
>> 
>>> Currently the value of the second key silently overwrites the value of
>>> the first.
> 
> Thanks but how would it be different with YAML? Can YAML check duplicate keys?

Oh, I didn't mean to imply that.  I was just pointing out that in general, YAML is more friendly for hand-editing than JSON.  They both have the same GIGO issue.

> How to process (read) YAML files in Python?

Take a look at http://pyyaml.org/


---
Roy Smith
roy@panix.com

[toc] | [prev] | [next] | [standalone]

#46426

On Wed, 29 May 2013 11:20:59 -0400, Roy Smith wrote:

>> How to process (read) YAML files in Python?
> 
> Take a look at http://pyyaml.org/

Beware that pyaml suffers from the same issue as pickle, namely that it 
can execute arbitrary code when reading untrusted data.


-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web