Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #68516 > unrolled thread

Controlling buffer alignment in file.read()

Back to article view | Back to comp.lang.python

Contents

  Controlling buffer alignment in file.read() "Haralanov, Mitko" <mitko.haralanov@intel.com> - 2014-03-18 20:23 +0000
    Re: Controlling buffer alignment in file.read() Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-03-19 12:08 +1300
      RE: Controlling buffer alignment in file.read() "Haralanov, Mitko" <mitko.haralanov@intel.com> - 2014-03-18 23:13 +0000
        Re: Controlling buffer alignment in file.read() Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-03-19 13:07 +1300
        RE: Controlling buffer alignment in file.read() Laurent Pointal <laurent.pointal@free.fr> - 2014-03-22 19:28 +0100

#68516 — Controlling buffer alignment in file.read()

Hi all,

I am using Python to read from a binary device file which requires that all read sizes are in 8byte multiples and the user's buffer is 8byte aligned.

I am currently using a file object and the file.read() method. However, the issue is that the file.read() method allocates the buffer passed to C function under the covers and, therefore, the alignment is arbitrary.

Is there a way that I can get file.read() to use an 8byte aligned buffer?

Thanks,
- Mitko

[toc] | [next] | [standalone]

#68524

Haralanov, Mitko wrote:
> I am using Python to read from a binary device file which requires that all
> read sizes are in 8byte multiples and the user's buffer is 8byte aligned.
> 
> Is there a way that I can get file.read() to use an 8byte aligned buffer?

For control at that level you'd be better off using
direct system calls, i.e. os.open() and os.read(),
then you can read exacty the number of bytes you want.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#68526

> For control at that level you'd be better off using
> direct system calls, i.e. os.open() and os.read(),
> then you can read exacty the number of bytes you want.
> 

The problem is not controlling the number of bytes read. That part seems to be working.
The issue is that the buffer into which the data is placed needs to be of certain alignment (8byte-aligned). Python does not seem to have a way that allows me to control that.

Thanks,
- Mitko

[toc] | [prev] | [next] | [standalone]

#68527

Haralanov, Mitko wrote:

> The problem is not controlling the number of bytes read. That part seems to
> be working. The issue is that the buffer into which the data is placed needs
> to be of certain alignment (8byte-aligned). Python does not seem to have a
> way that allows me to control that.

Hmmm, that could be tricky. Have you tried using os.read()?
If you're lucky, Python will be using a malloc() call or
equivalent to create a str/bytes object to read the data
into, and that will return something platform-aligned.

If you're unlucky, there's probably no pure-Python
solution, and you might need to write a small C or
Cython module to accomplish this trick.


-- 
Greg

[toc] | [prev] | [next] | [standalone]

#68787

Haralanov, Mitko wrote:

>> For control at that level you'd be better off using
>> direct system calls, i.e. os.open() and os.read(),
>> then you can read exacty the number of bytes you want.
>> 
> 
> The problem is not controlling the number of bytes read. That part seems
> to be working. The issue is that the buffer into which the data is placed
> needs to be of certain alignment (8byte-aligned). Python does not seem to
> have a way that allows me to control that.
> 
> Thanks,
> - Mitko

Did you try to set buffering parameter to 0 (unbuffered) ?

Eventually going to os.open() which map tp low level (eventually followed by 
an os.fdopen())

http://docs.python.org/2/library/os.html#os.open

A+
Laurent.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web