Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110703

Re: fastest way to read a text file in to a numpy array

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Cody Piersall <cody.piersall@gmail.com>
Newsgroups comp.lang.python
Subject Re: fastest way to read a text file in to a numpy array
Date Tue, 28 Jun 2016 09:37:55 -0500
Lines 19
Message-ID <mailman.84.1467125140.2358.python-list@python.org> (permalink)
References <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com> <CAFSbXtMu=ipkgi5u+buah7VnkYfz_vD+6ceQk-k_ui_uddfkBQ@mail.gmail.com>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
X-Trace news.uni-berlin.de TQb3u3l5WyAMmVsOOO1G5g0zzRBUf9nUuvmYuKqPfFQA==
Return-Path <cody.piersall@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.001
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'subject:text': 0.04; 'lines.': 0.07; 'subject:file': 0.07; 'cc:addr:python-list': 0.09; '[1]:': 0.09; '[2]:': 0.09; '[3]:': 0.09; 'files.': 0.13; '2016': 0.16; 'already,': 0.16; 'numpy': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'subject:array': 0.16; 'subject:fastest': 0.16; 'wrote:': 0.16; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'am,': 0.23; 'header:In-Reply-To:1': 0.24; 'wondering': 0.25; 'message-id:@mail.gmail.com': 0.27; 'fastest': 0.27; 'function': 0.28; 'robust': 0.29; 'array': 0.29; 'tue,': 0.34; 'file': 0.34; 'this?': 0.34; 'received:google.com': 0.35; 'text': 0.35; 'should': 0.36; 'there': 0.36; 'url:org': 0.36; 'subject:: ': 0.37; 'method': 0.37; 'hi,': 0.38; 'skip:n 10': 0.62; 'subject:read': 0.84; 'url:scipy': 0.84; 'url:reference': 0.91
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=shJSyGloa1pbJjkpj49GARguVBJ4O94uthnNLrtQ8wQ=; b=YJUtnPp9clTbRRhZ+1C6itaF+I3WdNNqbK+Z+QZFtDo7ChdL+YWh0H2TWB7xgOrB7K VFHIiEdYu21TcZjxBc2/2LUVsYBi6uYMg5xkA3e0cFL7Exar5Pw41ff0F2TNh23LvY8y wLvr/xJJsSZ+wVVciZFISFnFcQMFA/GHky8MjJigA9EpaIbbRcrVk8WIJZavjogcBfea 22aWplPdhlNF4pbBM7vtelocfkTRa4goBQzRLkY6IQQ8k0m9e9Wy8VIdIKS8hpawurOU HeDYyEdmbf1xQJ3+3ncwmHAjQGd0CljIAwcRjt4U17B52yDLdSxOdnY98RTT7rIpnFGf dUtg==
X-Google-DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=shJSyGloa1pbJjkpj49GARguVBJ4O94uthnNLrtQ8wQ=; b=HP1E747gMXRbxSV+4EJt7Vr5RQo30lT4A5kkiEwHFmWmumCly70qrHt1Pqwfr+k8BT A0bpMOmE4kVdX46NZEMPp2mKFWcdfUCUVUEQT8v7J0KIVCqqJVV1Gn/UsvqWxEgqngRw NCKbgho/3WDboRjyTcGmh/Tzpv3fylYG7cRKOjOYj8J3uynQUaqFIHrkXZ/9ptrzyf+G XkLK+/6ihhpnFK0Jo+ehgXzoB6g/nsys2y3MaOed6TVWtw947iTGfKgWVOYnVKMeBXWR Urgk4Za0+8vbsmmbUfS7DoMU43JZNMlf2TDf12i4ookp0ptQGjJrIJkeh5Cgd7/1+Qjw dVXw==
X-Gm-Message-State ALyK8tIaBlnoXgOelrv+AMxGM4tfwXctQSNMOrM2S6QIy4cZHM6XDmsoc6DI14g/TlvUV2RyjzK/AribNx5Vpg==
X-Received by 10.107.162.211 with SMTP id l202mr2565514ioe.138.1467124676181; Tue, 28 Jun 2016 07:37:56 -0700 (PDT)
In-Reply-To <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.22
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID <CAFSbXtMu=ipkgi5u+buah7VnkYfz_vD+6ceQk-k_ui_uddfkBQ@mail.gmail.com>
X-Mailman-Original-References <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com>
Xref csiph.com comp.lang.python:110703

Show key headers only | View raw

On Tue, Jun 28, 2016 at 8:45 AM, Heli <hemla21@gmail.com> wrote:
> Hi,
>
> I need to read a file in to a 2d numpy array containing many number of lines.
> I was wondering what is the fastest way to do this?
>
> Is even reading the file in to numpy array the best method or there are better approaches?
>

numpy.genfromtxt[1] is a pretty robust function for reading text files.

If you're generating the file from a numpy array already, you should
use arr.save()[2] and numpy.load()[3].

[1]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html
[2]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html
[3]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.load.html

Cody

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

fastest way to read a text file in to a numpy array Heli <hemla21@gmail.com> - 2016-06-28 06:45 -0700
  Re: fastest way to read a text file in to a numpy array Michael Selik <michael.selik@gmail.com> - 2016-06-28 14:00 +0000
  Re: fastest way to read a text file in to a numpy array Michael Selik <michael.selik@gmail.com> - 2016-06-28 14:29 +0000
  Re: fastest way to read a text file in to a numpy array Cody Piersall <cody.piersall@gmail.com> - 2016-06-28 09:37 -0500
    Re: fastest way to read a text file in to a numpy array Heli <hemla21@gmail.com> - 2016-06-30 08:49 -0700
      Re: fastest way to read a text file in to a numpy array Christian Gollwitzer <auriocus@gmx.de> - 2016-06-30 23:02 +0200

csiph-web