Optimizing if statement check over a numpy value

Newsgroups	comp.lang.python
Date	2015-07-23 02:21 -0700
Message-ID	<65c45685-dee1-41f8-a16a-7a062f4e7b02@googlegroups.com> (permalink)
Subject	Optimizing if statement check over a numpy value
From	Heli Nix <hemla21@gmail.com>

Show all headers | View raw

Dear all, 

I have the following piece of code. I am reading a numpy dataset from an hdf5 file and I am changing values to a new value if they equal 1. 

 There is 90 percent chance that (if id not in myList:) is true and in 10 percent of time is false. 

with h5py.File(inputFile, 'r') as f1:
    with h5py.File(inputFile2, 'w') as f2:
        ds=f1["MyDataset"].value
        myList=[list of Indices that must not be given the new_value]

        new_value=1e-20
        for index,val in     np.ndenumerate(ds):
            if val==1.0 :
                id=index[0]+1
                if id not in myList:
                    ds[index]=new_value
           
        dset1 = f2.create_dataset("Cell Ids", data=cellID_ds)  
        dset2 = f2.create_dataset("Porosity", data=poros_ds) 

My numpy array has 16M data and it takes 9 hrs to run. If I comment my if statement (if id not in myList:) it only takes 5 minutes to run. 

Is there any way that I can optimize this if statement. 

Thank you very much in Advance for your help. 

Best Regards,

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Optimizing if statement  check over a numpy value Heli Nix <hemla21@gmail.com> - 2015-07-23 02:21 -0700
  Re: Optimizing if statement check over a numpy value MRAB <python@mrabarnett.plus.com> - 2015-07-23 10:55 +0100
  Re: Optimizing if statement check over a numpy value Laura Creighton <lac@openend.se> - 2015-07-23 12:13 +0200
  Re: Optimizing if statement  check over a numpy value Jeremy Sanders <jeremy@jeremysanders.net> - 2015-07-23 13:42 +0200
    Re: Optimizing if statement  check over a numpy value Heli Nix <hemla21@gmail.com> - 2015-07-29 07:23 -0700

csiph-web