Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #99592

Re: read 4D binary data

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Laura Creighton <lac@openend.se>
Newsgroups comp.lang.python
Subject Re: read 4D binary data
Date Thu, 26 Nov 2015 19:00:35 +0100
Lines 47
Message-ID <mailman.150.1448560867.20593.python-list@python.org> (permalink)
References <4d15f12abb4c4781168ce05d3b450f63@cptec.inpe.br>
Mime-Version 1.0
Content-Type text/plain; charset="us-ascii"
Content-Transfer-Encoding quoted-printable
X-Trace news.uni-berlin.de f9zC4hl7+3K9D2LRSxSLKwmF1ms6fbO4Ljzl5uzirtwg==
Return-Path <lac@openend.se>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.010
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'binary': 0.05; 'cc:addr :python-list': 0.09; 'received:openend.se': 0.09; 'received:theraft.openend.se': 0.09; 'python': 0.10; 'jan': 0.11; 'question.': 0.13; 'thu,': 0.15; 'cc:addr:lac': 0.16; 'cc:addr:openend.se': 0.16; 'dimension': 0.16; 'from:addr:lac': 0.16; 'from:addr:openend.se': 0.16; 'from:name:laura creighton': 0.16; 'message-id:@fido.openend.se': 0.16; 'received:fido': 0.16; 'received:fido.openend.se': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'skip:> 20': 0.16; 'laura': 0.18; '2015': 0.20; 'cc:addr:python.org': 0.20; 'cc:2**1': 0.22; 'struct': 0.22; 'module': 0.25; 'points': 0.27; 'said,': 0.27; 'received:se': 0.29; 'array': 0.29; 'cc:no real name:2**1': 0.29; 'though,': 0.32; 'url:python': 0.33; 'i.e.': 0.35; 'nov': 0.35; 'but': 0.36; 'instead': 0.36; 'url:org': 0.36; 'tool': 0.36; 'data.': 0.36; 'url:library': 0.36; 'subject:: ': 0.37; 'charset:us-ascii': 0.37; 'data': 0.39; 'url:3': 0.60; 'your': 0.60; 'header:Message-Id:1': 0.61; 'levels': 0.70; 'analysis': 0.72; '>how': 0.84; 'header:In- reply-to:1': 0.84; 'latitude': 0.84; 'longitude': 0.84; 'subject:read': 0.84; 'url:array': 0.84; 'albert': 0.91; 'outlines': 0.91
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=openend.se; s=default; t=1448560850; bh=dTl5JrzwMIU8q2rrGstxu5nSv3BH/rOYR2iOfwBMEcI=; h=To:cc:From:Subject:In-reply-to:References:Date:From; b=fQdqn+LjrhvrkxkLfzGu92Hkrsq1bYHP50bLRePBxVrqj73VB6Q0ux7zzP7ykzdsf Vp2nwbaG7LFx22qZfSocKiqptON5QJ/TqMWfdlBd9oyZjYQU7c6/6o2+AcEvrmsl5B X3PcihDg2I9htTUgTL1wpH7mMw+JQ2rkNpRr2uIQ=
In-reply-to <4d15f12abb4c4781168ce05d3b450f63@cptec.inpe.br>
Comments In-reply-to jorge.conrado@cptec.inpe.br message dated "Thu, 26 Nov 2015 15:15:43 -0200."
Content-ID <11742.1448560835.1@fido>
X-Greylist Sender IP whitelisted, not delayed by milter-greylist-4.3.9 (theraft.openend.se [82.96.5.2]); Thu, 26 Nov 2015 19:00:50 +0100 (CET)
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:99592

Show key headers only | View raw


In a message of Thu, 26 Nov 2015 15:15:43 -0200, jorge.conrado@cptec.inpe.br wr
ites:
>Hi,
>
>
>I'm startig in Python and I have a 4D binary data. The dimension of my 
>data is:
>
>67 ----> longitude points
>41 ----> latitude points
>10 ----> pressure levels points
>33 ----> time points
>
>
>How can I read this data and what can I do to get a 2D array 
>(longitude,latitude) for a specific pressure  and time dimension.
>
>
>Conrado

If your binary data is all of the same type -- these are all floats,
or all ints -- you can use the python array module to read it.
https://docs.python.org/3/library/array.html

If they are heterogenous then you cannot use array, but can use struct
https://docs.python.org/3.5/library/struct.html

If I were you, I would want to get my data into pandas, which is
the best tool for data analysis
http://pandas.pydata.org/

And I would do this pretty much exactly as 
Albert Jan outlines in this stackoverflow question.
http://stackoverflow.com/questions/16573089/reading-binary-data-into-pandas
though, as I said, if your data is homogeneous you can use the array
module instead of using the struct one.  But struct will work on
homogenous data as well -- it is just slower.  You may not care.

Note that Albert Jan's answer is the one on the bottom, i.e. not the
one that got the checkmark from the original poster.

Laura

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: read 4D binary data Laura Creighton <lac@openend.se> - 2015-11-26 19:00 +0100

csiph-web