Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: Selecting unique values
Date: Tue, 26 Jul 2011 13:39:25 -0400
In-Reply-To: <j0lvt6$elc$1@dough.gmane.org>
Thread-Topic: Selecting unique values
Thread-Index: AcxLc3gqDhMlx7MCQna3Hrs6LAQa/wAR2Vng
References: <CABK368hMfEwm63OTjaHMy0DU7QCHt0=cwUPc42NL1aXL1GEUUg@mail.gmail.com> <j0lvt6$elc$1@dough.gmane.org>
From: "Sells, Fred" <fred.sells@adventistcare.org>
To: <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1503.1311701969.1164.python-list@python.org>
Lines: 115
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:10343

The set module or function (depends on which python version) will do
this if you make each record a tuple.

-----Original Message-----
From: python-list-bounces+frsells=3Dadventistcare.org@python.org
[mailto:python-list-bounces+frsells=3Dadventistcare.org@python.org] On
Behalf Of Peter Otten
Sent: Tuesday, July 26, 2011 5:04 AM
To: python-list@python.org
Subject: Re: Selecting unique values

Kumar Mainali wrote:

> I have a dataset with occurrence records of multiple species. I need
to
> get rid of multiple listings of the same occurrence point for a
species
> (as you see below in red and blue typeface). How do I create a dataset
> only with unique set of longitude and latitude for each species?
Thanks in
> advance.
>=20
> Species_name Longitude Latitude
> Abies concolor -106.601 35.868
> Abies concolor -106.493 35.9682
> Abies concolor -106.489 35.892
> Abies concolor -106.496 35.8542
> Accipiter cooperi -119.688 34.4339
> Accipiter cooperi -119.792 34.5069
> Accipiter cooperi -118.797 34.2581
> Accipiter cooperi -77.38333 39.68333
> Accipiter cooperi -77.38333 39.68333
> Accipiter cooperi -75.99153 40.633335
> Accipiter cooperi -75.99153 40.633335

>>> def uniquify(items):
...     seen =3D set()
...     for item in items:
...             if item not in seen:
...                     seen.add(item)
...                     yield item
...
>>> import sys
>>> sys.stdout.writelines(uniquify(open("species.txt")))
Species_name Longitude Latitude
Abies concolor -106.601 35.868
Abies concolor -106.493 35.9682
Abies concolor -106.489 35.892
Abies concolor -106.496 35.8542
Accipiter cooperi -119.688 34.4339
Accipiter cooperi -119.792 34.5069
Accipiter cooperi -118.797 34.2581
Accipiter cooperi -77.38333 39.68333
Accipiter cooperi -75.99153 40.633335

If you need to massage the lines a bit:

>>> def uniquify(items, key=3DNone):
...     seen =3D set()
...     for item in items:
...             if key is None:
...                     keyval =3D item
...             else:
...                     keyval =3D key(item)
...             if keyval not in seen:
...                     seen.add(keyval)
...                     yield item
...

Unique latitudes:

>>> sys.stdout.writelines(uniquify(open("species.txt"), key=3Dlambda s:=20
s.rsplit(None, 1)[-1]))
Species_name Longitude Latitude
Abies concolor -106.601 35.868
Abies concolor -106.493 35.9682
Abies concolor -106.489 35.892
Abies concolor -106.496 35.8542
Accipiter cooperi -119.688 34.4339
Accipiter cooperi -119.792 34.5069
Accipiter cooperi -118.797 34.2581
Accipiter cooperi -77.38333 39.68333
Accipiter cooperi -75.99153 40.633335

Unique species names:

>>> sys.stdout.writelines(uniquify(open("species.txt"), key=3Dlambda s:=20
s.rsplit(None, 2)[0]))
Species_name Longitude Latitude
Abies concolor -106.601 35.868
Accipiter cooperi -119.688 34.4339

Bonus: open() is not the built-in here:

>>> from StringIO import StringIO
>>> def open(filename):         =20
...     return StringIO("""Species_name Longitude Latitude
... Abies concolor -106.601 35.868                       =20
... Abies concolor -106.493 35.9682                      =20
... Abies concolor -106.489 35.892                       =20
... Abies concolor -106.496 35.8542                      =20
... Accipiter cooperi -119.688 34.4339                   =20
... Accipiter cooperi -119.792 34.5069                   =20
... Accipiter cooperi -118.797 34.2581                   =20
... Accipiter cooperi -77.38333 39.68333                 =20
... Accipiter cooperi -77.38333 39.68333                 =20
... Accipiter cooperi -75.99153 40.633335                =20
... Accipiter cooperi -75.99153 40.633335                =20
... """)                                                 =20
...                                                      =20


--=20
http://mail.python.org/mailman/listinfo/python-list