Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61704

Re: min max from tuples in list

Show all headers | View raw

Steven D'Aprano wrote:

> In any case, sorting in Python is amazingly fast. You may be pleasantly
> surprised that a version that sorts your data, while nominally
> O(N log N), may be much faster than an O(N) solution that doesn't require
> sorted data. If I were a betting man, I'd be willing to wager a shiny new
> dollar[1] that sorting works out faster for reasonable sized sets of data.

Well, that was my first reaction, too. But then

 
$ cat keyminmax.py
import operator
import itertools
import collections

def minmax_groupby(items):
    for key, group in itertools.groupby(sorted(items), 
key=operator.itemgetter(0)):
        minpair = maxpair = next(group)
        for maxpair in group:
            pass
        yield key, minpair[1], maxpair[1]

def minmax_dict(items):
    d = collections.defaultdict(list)
    for key, value in items:
        d[key].append(value)
    for key, values in d.items():
        yield key, min(values), max(values)

a = [(52, 193), (52, 193), (52, 192), (51, 193), (51, 191), (51, 190),
     (51, 189), (51, 188), (50, 194), (50, 187),(50, 186), (50, 185),
     (50, 184), (49, 194), (49, 183), (49, 182), (49, 181), (48, 194),
     (48, 180), (48, 179), (48, 178), (48, 177), (47, 194), (47, 176),
     (47, 175), (47, 174), (47, 173), (46, 195), (46, 172), (46, 171),
     (46, 170), (46, 169), (45, 195), (45, 168), (45, 167), (45, 166),
     (44, 195), (44, 165), (44, 164), (44, 163), (44, 162), (43, 195),
     (43, 161), (43, 160), (43, 159), (43, 158), (42, 196), (42, 157),
     (42, 156), (42, 155), (41, 196), (41, 154), (41, 153), (41, 152),
     (41, 151), (40, 196), (40, 150), (40, 149), (40, 148), (40, 147),
     (39, 196), (39, 146), (39, 145), (39, 144), (39, 143), (38, 196),
     (38, 142), (38, 141), (38, 140), (37, 197), (37, 139), (37, 138),
     (37, 137), (37, 136), (36, 197), (36, 135), (36, 134), (36, 133)]

from collections import deque
from itertools import groupby
from operator import itemgetter

def collect(data):
    data = sorted(data)
    groups = groupby(data, itemgetter(0))
    d = deque([], maxlen=1)
    for key, subiter in groups:
        smallest = largest = next(subiter)[1]
        d.extend(subiter)
        try:
            largest = d.pop()[1]
        except IndexError:
            pass
        yield (key, smallest, largest)


def time_dict():
    for item in minmax_dict(a):
        pass

def time_groupby():
    for item in minmax_groupby(a):
        pass

def time_daprano():
    for item in collect(a):
        pass

$ python -m timeit -s 'from keyminmax import time_groupby as t' 't()'
10000 loops, best of 3: 68.6 usec per loop
$ python -m timeit -s 'from keyminmax import time_dict as t' 't()'
10000 loops, best of 3: 53.3 usec per loop
$ python -m timeit -s 'from keyminmax import time_daprano as t' 't()'
10000 loops, best of 3: 75.7 usec per loop

So yes, sorting seems to be slower even for small datasets.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

min max from tuples in list Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-11 23:25 -0800
  Re: min max from tuples in list Chris Angelico <rosuav@gmail.com> - 2013-12-12 19:18 +1100
  Re: min max from tuples in list Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-12 00:34 -0800
    Re: min max from tuples in list Chris Angelico <rosuav@gmail.com> - 2013-12-12 19:43 +1100
  Re: min max from tuples in list Peter Otten <__peter__@web.de> - 2013-12-12 09:35 +0100
  Re: min max from tuples in list Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-12-12 10:52 +0200
  Re: min max from tuples in list Peter Otten <__peter__@web.de> - 2013-12-12 10:03 +0100
  Re: min max from tuples in list Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-12 11:44 +0000
    Re: min max from tuples in list Tim Chase <python.list@tim.thechases.com> - 2013-12-12 06:04 -0600
    Re: min max from tuples in list MRAB <python@mrabarnett.plus.com> - 2013-12-12 12:36 +0000
      Re: min max from tuples in list Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-12 23:25 +0000
    Re: min max from tuples in list Peter Otten <__peter__@web.de> - 2013-12-12 13:54 +0100
      Re: min max from tuples in list Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-12-13 02:36 +0000
    Re: min max from tuples in list Roy Smith <roy@panix.com> - 2013-12-12 10:02 -0500
      Re: min max from tuples in list Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-12 15:13 +0000
        Re: min max from tuples in list Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-12 22:28 -0800
          Re: min max from tuples in list rusi <rustompmody@gmail.com> - 2013-12-13 10:06 -0800
            Re: min max from tuples in list Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-12-13 19:30 -0500
              Re: min max from tuples in list Tim Roberts <timr@probo.com> - 2013-12-14 19:41 -0800
                Re: min max from tuples in list Chris Angelico <rosuav@gmail.com> - 2013-12-16 23:08 +1100
                Re: min max from tuples in list rusi <rustompmody@gmail.com> - 2013-12-16 07:49 -0800
                Re: min max from tuples in list Ned Batchelder <ned@nedbatchelder.com> - 2013-12-16 10:59 -0500
                Re: min max from tuples in list Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2013-12-17 11:49 +1300
  Re: min max from tuples in list Denis McMahon <denismfmcmahon@gmail.com> - 2013-12-13 01:33 +0000

csiph-web