Path: csiph.com!usenet.pasdenom.info!news.redatomik.org!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Tue, 31 Mar 2015 00:18:06 +0100
From: MRAB <python@mrabarnett.plus.com>
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Error in processing JSON files in Python
References: <63a251f6-01ff-4662-ab9c-588e3ffcd73a@googlegroups.com>
In-Reply-To: <63a251f6-01ff-4662-ab9c-588e3ffcd73a@googlegroups.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.352.1427757495.10327.python-list@python.org>
Lines: 54
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:88352

On 2015-03-30 22:27, Karthik Sharma wrote:
> I have the following python program to read a set of JSON files do some processing on it and dump them back to the same folder. However When I run the below program and then try to see the output of the JSON file using
>
> `cat file.json | python -m json.tool`
>
> I get the following error
>
> `extra data: line 1 column 307 - line 1 column 852 (char 306 - 851)`
>
> What is wrong with my program?
>
>      #Process 'new' events to extract more info from 'Messages'
>      rootDir = '/home/s_parts'
>      for dirName, subdirList, fileList in os.walk(rootDir):
>          print('Found directory: %s' % dirName)
>          for fname in fileList:
>              fname='s_parts/'+fname
>              with open(fname, 'r+') as f:
>                  json_data = json.load(f)
>                  et = json_data['Et']
>                  ms = json_data['Ms']
>                  if (event == 'a.b.c.d') or (event == 'e.f.g.h'):
>                      url = re.sub('.+roxy=([^& ]*).*', r'\1', ms)
>                      nt = re.findall(r"NT:\s*([^,)]*)",ms)[0]
>                      bt = re.findall(r"BT:\s*([^,)]*)",ms)[0]
>                      xt = re.findall(r"XT:\s*([^,)]*)",ms)[0]
>                      appde = ms.split('Appde:')[1].strip().split('<br>')[0]
>                      version = ms.split('version:')[1].strip().split('<br>')[0]
>                      json_data["url"] = url
>                      json_data["BT"] = bt
>                      json_data["XT"] = xt
>                      json_data["NT"] = nt
>                      json_data["Appde"] = appde
>                      json_data["version"] = version
>                  else:
>                      json_data["url"] = "null"
>                      json_data["BT"] = "null"
>                      json_data["XT"] = "null"
>                      json_data["NT"] = "null"
>                      json_data["Appde"] = "null"
>                      json_data["version"] = "null"
>                 json.dump(json_data,f)
>
> If I do a `file` command on the output file I get
> `s_parts/data_95: ASCII text, with very long lines, with no line terminators`
>
open(fname, 'r+') opens the file for update, json.load(f) reads from
the file, and then json.dump(json_data,f) writes back to the file,
_appending_ to it, so the file now contains the old data followed by
the new data.

Another point: "null" is a string and will be written as such. If you
actually want a null in the JSON data, then that should be None.