Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail From: Mark Lawrence Newsgroups: comp.lang.python Subject: Re: looping and searching in numpy array Date: Thu, 10 Mar 2016 13:22:14 +0000 Lines: 29 Message-ID: References: <77bd470b-cc05-4117-9ed1-6309d7a5633a@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Trace: news.uni-berlin.de Dbz3jBcqNR7YvB045hjamQdUx9su/Kic47XwdpzJ1HUg== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'from:addr:yahoo.co.uk': 0.05; 'below)': 0.07; 'before.': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'anyway': 0.11; 'size,': 0.13; 'suggest': 0.15; 'expert,': 0.16; 'numpy': 0.16; 'received:80.91.229.3': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'subject:array': 0.16; 'subject:looping': 0.16; 'wrote:': 0.16; 'language': 0.19; 'all,': 0.20; 'arrays': 0.22; 'lawrence': 0.22; 'seems': 0.23; 'bigger': 0.23; "haven't": 0.24; 'header:In-Reply-To:1': 0.24; 'script': 0.25; 'header:User- Agent:1': 0.26; 'example': 0.26; 'header:X-Complaints-To:1': 0.26; 'question': 0.27; 'search.': 0.29; 'array': 0.29; 'asked': 0.29; "i'm": 0.30; 'guess': 0.31; 'help,': 0.32; 'language.': 0.32; 'run': 0.33; 'values.': 0.33; 'list': 0.34; 'but': 0.36; 'there': 0.36; 'lines': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'two': 0.37; 'there,': 0.37; 'thanks': 0.37; 'received:org': 0.37; 'doing': 0.38; 'mailing': 0.38; 'to:addr:python.org': 0.40; 'mark': 0.40; 'your': 0.60; 'charset:windows-1252': 0.62; 'above,': 0.63; 'necessarily': 0.63; 'our': 0.64; 'dear': 0.67; 'pythonistas,': 0.84 X-Injected-Via-Gmane: http://gmane.org/ X-Gmane-NNTP-Posting-Host: 80.234.129.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 In-Reply-To: <77bd470b-cc05-4117-9ed1-6309d7a5633a@googlegroups.com> X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com comp.lang.python:104514 On 10/03/2016 11:43, Heli wrote: > Dear all, > > I need to loop over a numpy array and then do the following search. The following is taking almost 60(s) for an array (npArray1 and npArray2 in the example below) with around 300K values. > > > for id in np.nditer(npArray1): > > newId=(np.where(npArray2==id))[0][0] > > > Is there anyway I can make the above faster? I need to run the script above on much bigger arrays (50M). Please note that my two numpy arrays in the lines above, npArray1 and npArray2 are not necessarily the same size, but they are both 1d. > > > Thanks a lot for your help, > I'm no numpy expert, but if you're using a loop my guess is that you're doing it wrong. I suggest your first port of call is the numpy docs if you haven't all ready been there, then the specific numpy mailing list or stackoverflow, as it seems very likely that this type of question has been asked before. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence