Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder1.xlned.com!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <lf0h3g$m7r$1@ger.gmane.org>
References: <lf0h3g$m7r$1@ger.gmane.org>
From: Ian Kelly <ian.g.kelly@gmail.com>
Date: Sun, 2 Mar 2014 18:27:40 -0700
Subject: Re: how to get bytes from bytearray without copying
To: Python <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.7613.1393810104.18130.python-list@python.org>
Lines: 41
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:67494

On Sun, Mar 2, 2014 at 5:07 PM, Juraj Ivan=C4=8Di=C4=87 <juraj.ivancic@gmai=
l.com> wrote:
> Is it possible to somehow 'steal' bytearray's buffer and make it a read-o=
nly
> bytes? I failed to find a way to do this, and would like to make sure.
>
> My use case is, I would expect, fairly common. I read a certain (potentia=
lly
> very large) amount of data from the network into a pre-allocated bytearra=
y.
> From that point on, this data is logically read-only. To prevent making
> redundant copies, I wrap it in a memoryview, and then slice and dice it. =
The
> problem with this memoryview is that it, and its slices, are considered
> writable, and thus cannot be hashed:
>
> ValueError: cannot hash writable memoryview object
>
> The only way (AFAICT) to make this work is to first create a bytes object
> from bytearray, but this copies the data. I don't need this copy, so I'd
> like to avoid it, because of both principle and performance reasons.
>
> Is there any reason why bytearray isn't able to release/convert its buffe=
r
> to bytes? I see that it has a clear() method which... well... clears it. =
The
> former would be much more useful.
>
> I would also be content if there is some way of making memoryview
> artificially read-only to avoid the above error.

Python 3.3 has a C API function to create a memoryview for a char*,
that can be made read-only.

    http://docs.python.org/3/c-api/memoryview.html#PyMemoryView_FromMemory

I don't see a way to do what you want in pure Python, apart from
perhaps writing an elaborate proxy class that would just be a poor
man's memoryview.  Or you could bite the bullet and copy everything
once at the start to create a bytes object, and then never have to
worry about it again.