Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail From: Cody Piersall Newsgroups: comp.lang.python Subject: Re: fastest way to read a text file in to a numpy array Date: Tue, 28 Jun 2016 09:37:55 -0500 Lines: 19 Message-ID: References: <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: news.uni-berlin.de TQb3u3l5WyAMmVsOOO1G5g0zzRBUf9nUuvmYuKqPfFQA== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:text': 0.04; 'lines.': 0.07; 'subject:file': 0.07; 'cc:addr:python-list': 0.09; '[1]:': 0.09; '[2]:': 0.09; '[3]:': 0.09; 'files.': 0.13; '2016': 0.16; 'already,': 0.16; 'numpy': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'subject:array': 0.16; 'subject:fastest': 0.16; 'wrote:': 0.16; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'am,': 0.23; 'header:In-Reply-To:1': 0.24; 'wondering': 0.25; 'message-id:@mail.gmail.com': 0.27; 'fastest': 0.27; 'function': 0.28; 'robust': 0.29; 'array': 0.29; 'tue,': 0.34; 'file': 0.34; 'this?': 0.34; 'received:google.com': 0.35; 'text': 0.35; 'should': 0.36; 'there': 0.36; 'url:org': 0.36; 'subject:: ': 0.37; 'method': 0.37; 'hi,': 0.38; 'skip:n 10': 0.62; 'subject:read': 0.84; 'url:scipy': 0.84; 'url:reference': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=shJSyGloa1pbJjkpj49GARguVBJ4O94uthnNLrtQ8wQ=; b=YJUtnPp9clTbRRhZ+1C6itaF+I3WdNNqbK+Z+QZFtDo7ChdL+YWh0H2TWB7xgOrB7K VFHIiEdYu21TcZjxBc2/2LUVsYBi6uYMg5xkA3e0cFL7Exar5Pw41ff0F2TNh23LvY8y wLvr/xJJsSZ+wVVciZFISFnFcQMFA/GHky8MjJigA9EpaIbbRcrVk8WIJZavjogcBfea 22aWplPdhlNF4pbBM7vtelocfkTRa4goBQzRLkY6IQQ8k0m9e9Wy8VIdIKS8hpawurOU HeDYyEdmbf1xQJ3+3ncwmHAjQGd0CljIAwcRjt4U17B52yDLdSxOdnY98RTT7rIpnFGf dUtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=shJSyGloa1pbJjkpj49GARguVBJ4O94uthnNLrtQ8wQ=; b=HP1E747gMXRbxSV+4EJt7Vr5RQo30lT4A5kkiEwHFmWmumCly70qrHt1Pqwfr+k8BT A0bpMOmE4kVdX46NZEMPp2mKFWcdfUCUVUEQT8v7J0KIVCqqJVV1Gn/UsvqWxEgqngRw NCKbgho/3WDboRjyTcGmh/Tzpv3fylYG7cRKOjOYj8J3uynQUaqFIHrkXZ/9ptrzyf+G XkLK+/6ihhpnFK0Jo+ehgXzoB6g/nsys2y3MaOed6TVWtw947iTGfKgWVOYnVKMeBXWR Urgk4Za0+8vbsmmbUfS7DoMU43JZNMlf2TDf12i4ookp0ptQGjJrIJkeh5Cgd7/1+Qjw dVXw== X-Gm-Message-State: ALyK8tIaBlnoXgOelrv+AMxGM4tfwXctQSNMOrM2S6QIy4cZHM6XDmsoc6DI14g/TlvUV2RyjzK/AribNx5Vpg== X-Received: by 10.107.162.211 with SMTP id l202mr2565514ioe.138.1467124676181; Tue, 28 Jun 2016 07:37:56 -0700 (PDT) In-Reply-To: <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com> X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: X-Mailman-Original-References: <37b46ad8-8318-4d67-a65c-7dd7a50a3848@googlegroups.com> Xref: csiph.com comp.lang.python:110703 On Tue, Jun 28, 2016 at 8:45 AM, Heli wrote: > Hi, > > I need to read a file in to a 2d numpy array containing many number of lines. > I was wondering what is the fastest way to do this? > > Is even reading the file in to numpy array the best method or there are better approaches? > numpy.genfromtxt[1] is a pretty robust function for reading text files. If you're generating the file from a numpy array already, you should use arr.save()[2] and numpy.load()[3]. [1]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html [2]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html [3]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.load.html Cody