Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; '"this': 0.03; 'string': 0.09; 'iterate': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:string': 0.09; 'def': 0.12; '5),': 0.16; 'chunks': 0.16; 'itertools': 0.16; 'manageable': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'roy': 0.16; 'size):': 0.16; 'stringio': 0.16; 'do,': 0.16; 'wrote:': 0.18; 'seems': 0.21; '>>>': 0.22; 'import': 0.22; 'header:User-Agent:1': 0.23; 'skip:l 30': 0.24; 'header:X -Complaints-To:1': 0.27; 'character': 0.29; 'getting': 0.31; 'something': 0.35; 'no,': 0.35; 'but': 0.35; 'there': 0.35; 'edge': 0.36; 'yield': 0.36; 'subject:?': 0.36; 'should': 0.36; 'example,': 0.37; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'worry': 0.60; 'anything.': 0.68; 'smith': 0.68; 'partial': 0.84; 'subject:long': 0.84
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: Peter Otten <__peter__@web.de>
Subject: Re: chunking a long string?
Date: Fri, 08 Nov 2013 19:02:39 +0100
Organization: None
References: <9258483E-76BD-4D96-8C46-7262A95E3ED4@panix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
X-Gmane-NNTP-Posting-Host: p5084bd4a.dip0.t-ipconnect.de
User-Agent: KNode/4.7.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2236.1383933759.18130.python-list@python.org>
Lines: 38
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1383933759 news.xs4all.nl 15880 [2001:888:2000:d::a6]:34016
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:58829

Roy Smith wrote:

> I have a long string (several Mbytes).  I want to iterate over it in
> manageable chunks (say, 1 kbyte each).  For (a small) example, if I
> started with "this is a very long string", and I wanted 10 character
> chunks, I should get:
> 
> "this is a "
> "very long "
> "string"
> 
> This seems like something itertools would do, but I don't see anything. 
> Is there something, or do I just need to loop and slice (and worry about
> getting all the edge conditions right) myself?

(x)range() can take care of the edges:

>>> s = "this is a very long string"
>>> def chunks(s, size):
...     for start in xrange(0, len(s), size):
...             yield s[start:start+size]
... 
>>> list(chunks(s, 10))
['this is a ', 'very long ', 'string']
>>> list(chunks(s, 5))
['this ', 'is a ', 'very ', 'long ', 'strin', 'g']
>>> list(chunks(s, 100))
['this is a very long string']

Or you use StringIO:

>>> from functools import partial
>>> from StringIO import StringIO
>>> list(iter(partial(StringIO(s).read, 5), ""))
['this ', 'is a ', 'very ', 'long ', 'strin', 'g']

And no, this need not be a one-liner ;)