Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #67494

Re: how to get bytes from bytearray without copying

Show key headers only | View raw

On Sun, Mar 2, 2014 at 5:07 PM, Juraj Ivančić <juraj.ivancic@gmail.com> wrote:
> Is it possible to somehow 'steal' bytearray's buffer and make it a read-only
> bytes? I failed to find a way to do this, and would like to make sure.
>
> My use case is, I would expect, fairly common. I read a certain (potentially
> very large) amount of data from the network into a pre-allocated bytearray.
> From that point on, this data is logically read-only. To prevent making
> redundant copies, I wrap it in a memoryview, and then slice and dice it. The
> problem with this memoryview is that it, and its slices, are considered
> writable, and thus cannot be hashed:
>
> ValueError: cannot hash writable memoryview object
>
> The only way (AFAICT) to make this work is to first create a bytes object
> from bytearray, but this copies the data. I don't need this copy, so I'd
> like to avoid it, because of both principle and performance reasons.
>
> Is there any reason why bytearray isn't able to release/convert its buffer
> to bytes? I see that it has a clear() method which... well... clears it. The
> former would be much more useful.
>
> I would also be content if there is some way of making memoryview
> artificially read-only to avoid the above error.

Python 3.3 has a C API function to create a memoryview for a char*,
that can be made read-only.

    http://docs.python.org/3/c-api/memoryview.html#PyMemoryView_FromMemory

I don't see a way to do what you want in pure Python, apart from
perhaps writing an elaborate proxy class that would just be a poor
man's memoryview.  Or you could bite the bullet and copy everything
once at the start to create a bytes object, and then never have to
worry about it again.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: how to get bytes from bytearray without copying Ian Kelly <ian.g.kelly@gmail.com> - 2014-03-02 18:27 -0700

csiph-web