Path: csiph.com!usenet.pasdenom.info!news.franciliens.net!fdn.fr!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <fomcl@yahoo.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.047
X-Spam-Evidence: '*H*': 0.91; '*S*': 0.00; "'',": 0.07; 'binary': 0.07; 'initialize': 0.07; "'')": 0.09; 'compact': 0.09; 'repeated': 0.09; 'rows': 0.09; 'to)': 0.09; 'subject:question': 0.10; '(1,': 0.16; '__future__': 0.16; 'advance!': 0.16; 'numpy': 0.16; 'record,': 0.16; 'tuple': 0.16; 'unpacked': 0.16; 'unpacking)': 0.16; "skip:' 30": 0.19; 'import': 0.22; 'this?': 0.23; 'record': 0.27; 'array': 0.29; 'skip:q 20': 0.31; 'probably': 0.32; 'subject: (': 0.35; 'something': 0.35; 'etc': 0.35; 'but': 0.35; 'there': 0.35; 'done': 0.36; 'hi,': 0.36; 'system,': 0.38; 'thank': 0.38; 'filled': 0.38; 'to:addr:python- list': 0.38; 'to:addr:python.org': 0.39; 'read': 0.60; 'received:67.195': 0.60; 'received:98.137': 0.60; 'simple,': 0.60; 'new': 0.61; 'skip:n 10': 0.64; 'more': 0.64; 'header:Reply-To:1': 0.67; 'received:gq1.yahoo.com': 0.68; 'received:mail.gq1.yahoo.com': 0.68; 'apart': 0.72; 'records': 0.73; 'skip:n 40': 0.81; '4.0,': 0.84; 'received:bullet.mail.gq1.yahoo.com': 0.84; 'subject:think': 0.84; 'us?': 0.84; 'medicine,': 0.91; 'education,': 0.96
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1418506751; bh=vzqfF/u3fx2AHYRFLtVKHabQ97y99fWikXwK8s55OnQ=; h=Date:From:Reply-To:To:Subject:From:Subject; b=CTEJVqCnDf81HZJsUc+GWVaaEMHvAnqFsi+7oxlN5P+XKT8WEw86LBGvAErKFrjayiCOyIezvFKET7aHoH2idW3xkLmXr2XkAQJ5x5SIWRsbq5SA+DjAnAcXPfjAo3sB2rua/RCFHRpgB+pJDYg9g6eByRHxh6IHXyh7JuNdUteNWm3I175YNNq1go2xqK91ZIAv9y9neuhAFofPcl6/Az9aQ9wy/HXZBXtZ/c6FX+tjhMkXWCN39QyKvwccPGUPvc431nAHNZDqx2V4S0cqaEcJQDCQZd6Oo+R4grx4Su/LZWCmGrSz4wZ0kqTGrsfa7zc0fs7FcYzUy4g30SSQLA==
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=i1z7tmcYv7HOA3blW0Srs0qMFCbrLApMqdVkxFee7Nm7egJp2qIqj8oe4bnx2jEoJOE54ysymDJRAeb5OXVUkJ1LX5/mTJlLr8VnBwXlhV7/lLvWl5fd9b5pozBroy/OmHZ97AhPFe1rHeT6tJFPoJo5puiiR1LyIhowzcYZML7bytSw1DgRqb3DdqwFL9bhBx3xapT7r4x6ZmXlmDMOUYktIhWDWFt0p+qQtCw73pfFjhYMdKK0ddWMBevw2btcUNa79DpYPtVwcXbxc54RTyNmv7CHxcH+xNpnFJU2XA9qKW179BNHJQhWdArqbRIbmu0gc7fhzv1OUhjbwb67kw==;
X-Yahoo-Newman-Property: ymail-4
X-Yahoo-Newman-Id: 101213.29426.bm@omp1002.mail.gq1.yahoo.com
X-YMail-OSG: 3r8XHzYVM1ld31OSSPBKOXyuOHSB15J8PeGK9orCL2ziRzFitkXl6pQ78sqhx89 Q0q7yTsfLwruIw_jH1.6TYnYw.Q29G_hgVsZYNPWCavFwVS3zwlqGZYHGNmNxrpI6DBKI9V9gt3Z oWg132MQTRC62bLTc6eLyHPQVugaosgf9e_1iTBEPUKbAL0OFwXQAfB_.rdUjxXFdcVFb5Wxrs7u 7w6xP_6vKNWaR.CafKrMFk0Sp68135fDIhwDHPVATeorBsezTzuhIJ6L3QinBG8zXknKKzJqnMXP hppPmkIjLOyMnFNyKMqdXFAkormRPCFfBrhnv6IuA_1IigGh7pnG2h3imGYmTDpWIGLt5i91yLso oz9n0.OP3j9Qx1_WSUkRKfo1oaoS4LIPt0Ry5sGAXZWq5vR65AWkJvlBoZ2AYx2fohPa8NasfwBb 4e_kRx8X.5xssfyS0WGjpXr4Ic5L5OiGJkmCRZsJ_7cHyOon1awFoCzHcPyGNtKF11GhYbKj_jmW _gfE_qC34MxKDwg--
Date: Sat, 13 Dec 2014 21:36:10 +0000 (UTC)
From: Albert-Jan Roskam <fomcl@yahoo.com>
To: Python    <python-list@python.org>
Subject: numpy question (fairly basic, I think)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: Albert-Jan Roskam <fomcl@yahoo.com>
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.16937.1418506920.18130.python-list@python.org>
Lines: 74
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1418506920 news.xs4all.nl 2854 [2001:888:2000:d::a6]:33879
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:82488

Hi,

I am new to numpy. I am reading binary data one record at a time (I have to) and I would like to store all the records in a numpy array which I pre-allocate. Below I try to fill the empty array with exactly one record, but it is filled with as many rows as there are columns. Why is this? It is probably something simple, but I am stuck! It is like the original record is not unpacked *as in tuple unpacking) into the array, so it remains one chunk, not an (nrows, ncols) structure.


from __future__ import print_function
import numpy as np


# one binary record
s = '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00;@\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x80U\xe1@\x00\x00\x00\x00\x80\xd9\xe4@\x00\x00\x00\x00@\xa7\xe3@\xab\xaa\xaa\xaajG\xe3@\x00\x00\x00\x00\x80\xd9\xe4@\x00\x00\x00\x00\x00\x00;@\x00\x00\x00\x00\x00\x00;@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\xa4DI\tBx       qwertyuiopasdfghjklzxcvbnm,./                                                                           \x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00p\x9f@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00(@DEC 2012'


# read it into a structured array
formats = ['<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', '<d', 'a8', 'a104', '<d', '<d', '<d', '<d', 'a8']
names = ["v%02d" % i for i in range(len(formats))]
dt = np.dtype({'formats': formats, "names": names})
record = np.fromstring(s, dtype=dt)


# make it more compact
trunc_formats = ['f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'f4', 'a1', 'a100', 'f4', 'f4', 'f4', 'f4', 'a8']
trunc_dt = np.dtype({'formats': trunc_formats, "names": names})
record = record.astype(trunc_dt)
print(record.shape)  # (1,), but it needs to be (1, 22)??
#record = np.asarray(*tuple(record.astype(trunc_dt)), dtype=np.object)
#record = np.expand_dims(record, axis=0)


# initialize an empty array and fill it with one record

nrows = 50  # arbitrary number
ncols = len(formats) # 22

array = np.zeros((nrows, ncols), trunc_dt)
array[0,:] = record
print(array)

# output: why is the record repeated ncols times?
[[ (1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')
(1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')
(1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')
...,
(1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')
(1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')
(1.0, 27.0, 1.0, 35500.0, 42700.0, 40250.0, 39483.33203125, 42700.0, 27.0, 27.0, 0.0, 1.0, 1.0, 1.0, 13575427072.0, 'x', 'qwertyuiopasdfghjklzxcvbnm,./                                                                       ', 1.0, 2012.0, 4.0, 12.0, 'DEC 2012')]
[ (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')
...,
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, '', '', 0.0, 0.0, 0.0, 0.0, '')]
etc
etc

 
Thank you in advance!


Regards,

Albert-Jan




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a 

fresh water system, and public health, what have the Romans ever done for us?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~