Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #94733

Re: Optimizing if statement check over a numpy value

Show all headers | View raw

On Thursday, July 23, 2015 at 1:43:00 PM UTC+2, Jeremy Sanders wrote:
> Heli Nix wrote:
> 
> > Is there any way that I can optimize this if statement.
> 
> Array processing is much faster in numpy. Maybe this is close to what you 
> want
> 
> import numpy as N
> # input data
> vals = N.array([42, 1, 5, 3.14, 53, 1, 12, 11, 1])
> # list of items to exclude
> exclude = [1]
> # convert to a boolean array
> exclbool = N.zeros(vals.shape, dtype=bool)
> exclbool[exclude] = True
> # do replacement
> ones = vals==1.0
> # Note: ~ is numpy.logical_not
> vals[ones & (~exclbool)] = 1e-20
> 
> I think you'll have to convert your HDF array into a numpy array first, 
> using numpy.array().
> 
> Jeremy

Dear all, 

I tried the sorted python list, but this did not really help the runtime. 

I haven´t had time to check the sorted collections.  I solved my runtime problem by using the script from Jeremy up here. 

It was a life saviour and it is amazing how powerful numpy is. Thanks a lot Jeremy for this. By the way, I did not have to do any array conversion. The array read from hdf5 file using h5py is already a numpy array. 

The runtime over an array of around 16M reduced from around 12 hours (previous script) to 3 seconds using numpy on the same machine. 


Thanks alot for your help,

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Optimizing if statement  check over a numpy value Heli Nix <hemla21@gmail.com> - 2015-07-23 02:21 -0700
  Re: Optimizing if statement check over a numpy value MRAB <python@mrabarnett.plus.com> - 2015-07-23 10:55 +0100
  Re: Optimizing if statement check over a numpy value Laura Creighton <lac@openend.se> - 2015-07-23 12:13 +0200
  Re: Optimizing if statement  check over a numpy value Jeremy Sanders <jeremy@jeremysanders.net> - 2015-07-23 13:42 +0200
    Re: Optimizing if statement  check over a numpy value Heli Nix <hemla21@gmail.com> - 2015-07-29 07:23 -0700

csiph-web