Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #26001 > unrolled thread

no data exclution and unique combination.

Back to article view | Back to comp.lang.python

Contents

  no data exclution and unique combination. giuseppe.amatulli@gmail.com - 2012-07-24 11:27 -0700
    Re: no data exclution and unique combination. MRAB <python@mrabarnett.plus.com> - 2012-07-24 19:51 +0100
    Re: no data exclution and unique combination. MRAB <python@mrabarnett.plus.com> - 2012-07-24 20:08 +0100
    Re: no data exclution and unique combination. Terry Reedy <tjreedy@udel.edu> - 2012-07-24 15:32 -0400
    Re: no data exclution and unique combination. giuseppe.amatulli@gmail.com - 2012-08-09 13:06 -0700
      Re: no data exclution and unique combination. Dave Angel <d@davea.name> - 2012-08-09 17:23 -0400
      Re: no data exclution and unique combination. Terry Reedy <tjreedy@udel.edu> - 2012-08-09 17:33 -0400
        Re: no data exclution and unique combination. Hans Mulder <hansmu@xs4all.nl> - 2012-08-10 10:47 +0200

#26001 — no data exclution and unique combination.

Hi,
would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa. 

given 

a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])

no_data_a=1
no_data_b=2

a_clean=array([4,4,5,4,4,4])
b_clean=array([3,5,4,4,3,4])

after i need to calculate unique combination in pairs to count the observations 
and obtain
(4,3,2)
(4,5,1)
(5,4,1)
(4,4,2)

For the fist task i did

a_No_data_a = a[a != no_data_a]
b_No_data_a = b[a != no_data_a]

b_clean = b_No_data_a[b_No_data_a != no_data_b]
a_clean  = a_No_data_a[a_No_data_a != no_data_b]

but the results are not really stable. 

For the second task 
The np.unique would solve the problem if it can be apply to a two arrays.

Any idea?
thanks in advance 
Giuseppe

[toc] | [next] | [standalone]

#26002

On 24/07/2012 19:27, giuseppe.amatulli@gmail.com wrote:
> Hi,
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>
> given
>
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>
> no_data_a=1
> no_data_b=2
>
> a_clean=array([4,4,5,4,4,4])
> b_clean=array([3,5,4,4,3,4])
>
> after i need to calculate unique combination in pairs to count the observations
> and obtain
> (4,3,2)
> (4,5,1)
> (5,4,1)
> (4,4,2)
>
> For the fist task i did
>
> a_No_data_a = a[a != no_data_a]
> b_No_data_a = b[a != no_data_a]
>
> b_clean = b_No_data_a[b_No_data_a != no_data_b]
> a_clean  = a_No_data_a[a_No_data_a != no_data_b]
>
> but the results are not really stable.
>
mask = (a != no_data_a) & (b != no_data_b)
a_clean = a[mask]
b_clean = b[mask]

> For the second task
> The np.unique would solve the problem if it can be apply to a two arrays.
>

[toc] | [prev] | [next] | [standalone]

#26003

On 24/07/2012 19:51, MRAB wrote:
> On 24/07/2012 19:27, giuseppe.amatulli@gmail.com wrote:
>> Hi,
>> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>>
>> given
>>
>> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
>> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>>
>> no_data_a=1
>> no_data_b=2
>>
>> a_clean=array([4,4,5,4,4,4])
>> b_clean=array([3,5,4,4,3,4])
>>
>> after i need to calculate unique combination in pairs to count the observations
>> and obtain
>> (4,3,2)
>> (4,5,1)
>> (5,4,1)
>> (4,4,2)
>>
>> For the fist task i did
>>
>> a_No_data_a = a[a != no_data_a]
>> b_No_data_a = b[a != no_data_a]
>>
>> b_clean = b_No_data_a[b_No_data_a != no_data_b]
>> a_clean  = a_No_data_a[a_No_data_a != no_data_b]
>>
>> but the results are not really stable.
>>
> mask = (a != no_data_a) & (b != no_data_b)
> a_clean = a[mask]
> b_clean = b[mask]
>
>> For the second task
>> The np.unique would solve the problem if it can be apply to a two arrays.
>>
I couldn't figure out how to do the second part in numpy, so:

from collections import Counter
counts = Counter(zip(a_clean, b_clean))
counts = [pair + (count,) for pair, count in counts.items()]

[toc] | [prev] | [next] | [standalone]

#26005

On 7/24/2012 2:27 PM, giuseppe.amatulli@gmail.com wrote:
> Hi,
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>
> given
>
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>
> no_data_a=1
> no_data_b=2
>
> a_clean=array([4,4,5,4,4,4])
> b_clean=array([3,5,4,4,3,4])

As I discovered when running the solution before, your test data are 
wrong, leaving out 2,3 before the last pair (4,4). Anyway, for those 
interested in a plain Python solution, without numpy:

a=[1,2,4,4,5,4,1,4,1,1,2,4]
b=[1,2,3,5,4,4,1,3,2,1,3,4]

no_data_a=1
no_data_b=2

a_clean=(4,4,5,4,4,2,4)
b_clean=(3,5,4,4,3,3,4)

cleaned = list(zip(*(pair for pair in zip(a,b)
     if pair[0] != no_data_a and pair[1] != no_data_b)))
print(cleaned, cleaned == [a_clean, b_clean])
#
[(4, 4, 5, 4, 4, 2, 4), (3, 5, 4, 4, 3, 3, 4)] True


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#26805

Terry and MRAB,
thanks for yours suggestions, 
in the end i found this solution 


mask=( a != 0 ) & ( b != 0  )

a_mask=a[mask]
b_mask=b[mask]

array2D = np.array(zip(a_mask,b_mask))

unique=dict()
for row in  array2D :
    row = tuple(row)
    if row in unique:
        unique[row] += 1
    else:
        unique[row] = 1

print unique
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}

I choose this solution because i could not install "from collections import Counter". 
Anyway how i can print to a file the unique results without the brackets and obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2

Thanks in advance
Best regards.
Giuseppe








On Tuesday, 24 July 2012 13:27:05 UTC-5, giuseppe...@gmail.com  wrote:
> Hi,
> 
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa. 
> 
> 
> 
> given 
> 
> 
> 
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
> 
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
> 
> 
> 
> no_data_a=1
> 
> no_data_b=2
> 
> 
> 
> a_clean=array([4,4,5,4,4,4])
> 
> b_clean=array([3,5,4,4,3,4])
> 
> 
> 
> after i need to calculate unique combination in pairs to count the observations 
> 
> and obtain
> 
> (4,3,2)
> 
> (4,5,1)
> 
> (5,4,1)
> 
> (4,4,2)
> 
> 
> 
> For the fist task i did
> 
> 
> 
> a_No_data_a = a[a != no_data_a]
> 
> b_No_data_a = b[a != no_data_a]
> 
> 
> 
> b_clean = b_No_data_a[b_No_data_a != no_data_b]
> 
> a_clean  = a_No_data_a[a_No_data_a != no_data_b]
> 
> 
> 
> but the results are not really stable. 
> 
> 
> 
> For the second task 
> 
> The np.unique would solve the problem if it can be apply to a two arrays.
> 
> 
> 
> Any idea?
> 
> thanks in advance 
> 
> Giuseppe

[toc] | [prev] | [next] | [standalone]

#26815

On 08/09/2012 04:06 PM, giuseppe.amatulli@gmail.com wrote:
> <SNIP>
>
> print unique
> {(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
>
> I choose this solution because i could not install "from collections import Counter". 

Nothing to install, at least for Python 2.7. collections is in the
standard library, and Counter is in collections (new in version 2.7)

> Anyway how i can print to a file the unique results without the brackets and obtain something like this?
> 4 5 1
> 5 4 1
> 4 4 2
> 2 3 1
> 4 3 2
>
>

To print out a dict in an explicit format, you want a loop.

for key, val in unique.items():
     print key[0], key[1], val

Note that you cannot guarantee the order they will print out, once
stored in a dict.  You may want to sort them, or otherwise order them if
it matters.

<SNIP>

Note I added my responses after the parts of your message that I was
quoting.  To put the response first is called top-posting, and against
policy here.

-- 

DaveA

[toc] | [prev] | [next] | [standalone]

#26816

On 8/9/2012 4:06 PM, giuseppe.amatulli@gmail.com wrote:
> Terry and MRAB,
> thanks for yours suggestions,
> in the end i found this solution
>
>
> mask=( a != 0 ) & ( b != 0  )
>
> a_mask=a[mask]
> b_mask=b[mask]
>
> array2D = np.array(zip(a_mask,b_mask))
>
> unique=dict()
> for row in  array2D :
>      row = tuple(row)
>      if row in unique:
>          unique[row] += 1
>      else:
>          unique[row] = 1

I believe the 4 lines above are equivalent to
    unique[row] = unique.get(row, 0) + 1

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#26844

On 9/08/12 23:33:58, Terry Reedy wrote:
> On 8/9/2012 4:06 PM, giuseppe.amatulli@gmail.com wrote:
[...]
>> unique=dict()
>> for row in  array2D :
>>      row = tuple(row)
>>      if row in unique:
>>          unique[row] += 1
>>      else:
>>          unique[row] = 1
> 
> I believe the 4 lines above are equivalent to
>    unique[row] = unique.get(row, 0) + 1


I would write that bit as:

from collections import defaultdict

unique = defaultdict(int)
for row in  array2D:
    unique[row] += 1


Hope this helps,

-- HansM

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web