Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Sat, 18 Jan 2014 11:22:02 -0800
From: Mark Heieis <mheieis@alois.ca>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Porting c extension - PyBuffer_New() deprecated in python3. What's the replacement?
References: <52D0DAD5.3020803@alois.ca> <lar1qg$rro$1@ger.gmane.org>
In-Reply-To: <lar1qg$rro$1@ger.gmane.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.5688.1390077792.18130.python-list@python.org>
Lines: 87
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:64252

Stefan,

Thank-you for the reply. I hadn't considered cpython, unfortunately the 
extension is too large a project to port at the moment. I ended up 
replacing the PyBuffer_New() segment with malloc() and passing back an 
object from PyByteArray_FromStringAndSize(). It seems to work.

mrh.

On 2014-01-11 01:10, Stefan Behnel wrote:
> Mark Heieis, 11.01.2014 06:47:
>> I need to convert the following existing c extension code to support Python 3.
>>
>> // --- existing code ------
>>
>>      // PyBuffer_New() deprecated in python3
>>      if (!(pyBuf = PyBuffer_New(len)))
>>      {
>>          return NULL;
>>      }
>>
>>      // should use memoryview object in python3
>>      if (PyObject_AsWriteBuffer(pyBuf, &cbuf, &len))
>>      {
>>          Py_DECREF(pyBuf);
>>          return NULL ;
>>      }
>>
>> // fill in cbuf
>>      ...
>>
>>      return pyBuf ;
>>
>> //-----------
>>
>> I'm somewhat confounded in finding an equivalent (PyBuffer_New()) for
>> creating a buffer of size len that has continuous memory in the c extension
>> function for python3. cbuf is manipulated/filled in using c, after which
>> the created pyBuf is then returned. So far, I haven't found much in the way
>> of examples/doc for porting the deprecated Python-/C-level buffer API calls
>> to the new C-level buffer API/memoryview object model.
>>
>> Any guidance or direction to existing doc/example is much appreciated.
> If the extension isn't huge, you should consider rewriting it in Cython.
> That can usually be done quite quickly - the main thing is to figure out
> what the verbose C code actually does and write it down in much simpler
> Python code. And it will make it easy to make the code portable and fast.
> Also likely safer and more generic and versatile, because Cython covers
> away a lot of the annoying boilerplate, ref-counting issues, type
> conversions, etc.
>
> For your specific problem at hand, you could use Cython's memory views:
>
> http://docs.cython.org/src/userguide/memoryviews.html
>
> They allow you to convert the input value to a 1-dim char buffer (or
> whatever you need, but you mentioned the old Py2 buffer interface, which
> can't do much more) by saying
>
>      cdef char[:] my_memview = some_python_object
>
> If you need to pass the unpacked buffer into C code, you can get the
> address as "&my_memview[0]" (i.e. the address of the first item in the
> buffer). Memory views themselves support fast slicing and indexing, so you
> can efficiently work with them using the normal Python slicing/indexing syntax.
>
> In case what you actually receive are not arbitrary buffers but simple byte
> strings or bytearray instances, you can even use the normal byte string
> coercion in Cython and simply say
>
>      cdef char* c_string = some_python_byte_string_object
>
> and then use that pointer to pass it on into C.
>
> I've written a string processing tutorial for Cython here:
>
> http://docs.cython.org/src/tutorial/strings.html
>
> These things may take a moment to learn, especially if you are used to
> doing everything in excessive manual detail in C code, but once you are
> through that, you should get things done much more quickly than when trying
> to do them by hand.
>
> Stefan
>
>