Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!ecngs!feeder2.ecngs.de!novso.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=Xfk2hU70UlWa0DiySP8WzTDrZgfB0Me7zXMww6fx3gui0vh/LRm+Z2sOmGS5lgKTj2Iy+7bT94mdX1Ta2nt7pCrOMqseVMtV7AuPj+SkwYF6fsHGFVBq5ClEqu6ARBNv/3GU380gGZFhzC0JRDlgOi7DjRA5i3jbKzLOJFbGuDrYNRZ253i/jWC1vwAXQIi9mooCB3UBefrlXWuWvJC4SmUXy/w3Zx2OoDhX0W5knlzPFN1Lzb17lMZJ9IDlXq9gdk8RUaiKqnRz/pUZ2l+4EmGOxD5Ops/N1nNwxFNmyd0Nzo2R9dwj9FT2UtEEWJDhJPv+OAJtjSENo9cPL/mgCA==;
Date: Sun, 14 Dec 2014 14:31:33 +0000 (UTC)
From: Albert-Jan Roskam <fomcl@yahoo.com>
To: Steven D'Aprano <steve+comp.lang.python@pearwood.info>, "python-list@python.org" <python-list@python.org>
In-Reply-To: <548cd14c$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <548cd14c$0$13013$c3e8da3$5496439d@news.astraweb.com>
Subject: Re: numpy question (fairly basic, I think)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
Reply-To: Albert-Jan Roskam <fomcl@yahoo.com>
Newsgroups: comp.lang.python
Message-ID: <mailman.16946.1418567821.18130.python-list@python.org>
Lines: 37
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:82503



----- Original Message -----

> From: Steven D'Aprano <steve+comp.lang.python@pearwood.info>
> To: python-list@python.org
> Cc: 
> Sent: Sunday, December 14, 2014 12:52 AM
> Subject: Re: numpy question (fairly basic, I think)
> 
> Albert-Jan Roskam wrote:
> 
>>  Hi,
>> 
>>  I am new to numpy. I am reading binary data one record at a time (I have
>>  to) and I would like to store all the records in a numpy array which I
>>  pre-allocate. Below I try to fill the empty array with exactly one record,
>>  but it is filled with as many rows as there are columns. Why is this? It
>>  is probably something simple, but I am stuck! It is like the original
>>  record is not unpacked *as in tuple unpacking) into the array, so it
>>  remains one chunk, not an (nrows, ncols) structure.
> 
> Can you simplify the example to something shorter that focuses on the issue
> at hand? It isn't clear to me which bits of the code you show are behaving
> the way you expect and which bits are not.


Hi Steven,

Thanks for replying. My code was so elaborate because I did not know which part made it go wrong. I think I have got it already. Numpy arrays (ndarrays) must be homogeneous wrt their datatype (dtype). 

Probably to make vectorization work (?). However, a structured array (which I was using) *can* contain multiple dtypes, but it can only be one-dimensional. Its records are tuples (or arrays). In this sense, even a structured array is homogeneous. I was trying to change the one-dim array into a two-dim array so I could easily retrieve columns. I now use a pandas DataFrame to do that. If my sample data would have contained *only* floats (or ints, or ...), my original approach would have worked. 


Thanks!

Albert-Jan