Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #17529

Re: Parsing stream of JSON objects incrementally

Show all headers | View raw

You probably need to accumulate a buffer and try to decode it, when succeeded return the object and read more. See example below (note that for sockets select might be a better option for reading data).

import json
from time import sleep

def json_decoder(fo):
    buff = ''
    decode = json.JSONDecoder().raw_decode
    while True:
        line = fo.readline()
        if not line:
            break
        buff += line
        print('BUFF: {}'.format(buff))
        try:
            obj, i = decode(buff)
            buff = buff[i:].lstrip()
            yield obj
        except ValueError as e:
            print('ERR: {}'.format(e))
            sleep(0.01) # select will probably be a better option :)


def main():
    import sys
    for obj in json_decoder(sys.stdin):
        print(obj)

if __name__ == '__main__':
    main()

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Parsing stream of JSON objects incrementally Evan Driscoll <edriscoll@wisc.edu> - 2011-12-17 19:29 -0600
  Re: Parsing stream of JSON objects incrementally Miki Tebeka <miki.tebeka@gmail.com> - 2011-12-19 15:18 -0800
  Re: Parsing stream of JSON objects incrementally Miki Tebeka <miki.tebeka@gmail.com> - 2011-12-19 15:18 -0800

csiph-web