X-Received: by 10.66.227.135 with SMTP id sa7mr37201945pac.19.1412039993864; Mon, 29 Sep 2014 18:19:53 -0700 (PDT) X-Received: by 10.50.132.66 with SMTP id os2mr36572igb.3.1412039993660; Mon, 29 Sep 2014 18:19:53 -0700 (PDT) Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!uq10no768118igb.0!news-out.google.com!rp1ni8527igb.0!nntp.google.com!a13no5795194igq.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail Newsgroups: comp.lang.python Date: Mon, 29 Sep 2014 18:19:53 -0700 (PDT) Complaints-To: groups-abuse@google.com Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=184.187.176.110; posting-account=wipd3QoAAACkqCR1nR6c6tC2496M_JVa NNTP-Posting-Host: 184.187.176.110 User-Agent: G2/1.0 MIME-Version: 1.0 Message-ID: <4232c92a-101c-4a1b-89b2-e80d67ef7bc6@googlegroups.com> Subject: JSON-encoding very long iterators From: alfred@54.org Injection-Date: Tue, 30 Sep 2014 01:19:53 +0000 Content-Type: text/plain; charset=ISO-8859-1 Xref: csiph.com comp.lang.python:78476 I would like to add the ability to JSONEncode large iterators. Right now there is no way to do this without modifying the code. The JSONEncoder.default() doc string suggests to do this: For example, to support arbitrary iterators, you could implement default like this:: def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return JSONEncoder.default(self, o) but this method requires the whole serialized object to fit in memory and it's a good chance that your iterator is an iterator to save on memory in the first place. By changing the code to accept iterators it is then possible to stream json as I did here: http://stackoverflow.com/a/26094558/289240 This would ideal if it were included in the standard library. Is there any reason why it shouldn't be?