Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #40823

Re: Interesting list() un-optimization

Show key headers only | View raw

> >>> Iterators do not generally have __len__ methods.
> >>>
> >>> >>> len(iter(range(10)))
> >>> Traceback (most recent call last):
> >>>   File "<stdin>", line 1, in <module>
> >>> TypeError: object of type 'range_iterator' has no len()
> >>
> >> But iterators have a length hint method that are used for some
> >> optimizations and preallocations, too.
> >>
> >> >>> i = iter(range(10))
> >> >>> i.__length_hint__()
> >> 10
> >>
> >> See http://www.python.org/dev/peps/pep-0424/
> >

very interesting (hadn't heard of it)! Just checked the PEP,
then tested list()'s behavior, and it is just as described:

class stupid(list):
	def __len__(self):
		print ('len() called')
		return NotImplemented
	
	def __length_hint__(self):
		print ('hint requested')
		l=iter(self).__length_hint__()
		print (l)
		return l

a=stupid((1,2,3))
len(d)
======>
  len() called

  Traceback (most recent call last):
    File "<pyshell#79>", line 1, in <module>
      len(d)
  TypeError: an integer is required

list(d)
======>
  len() called
  hint requested
  3
  [1, 2, 3]

so list() first tries to call the iterable's __len__ method. If that raises a
TypeError it falls back to __length_hint__ .
What I still don't know is how the listiterator object's __length_hint__ works.
Why, in this case, does it know that it has a length of 3 ? The PEP does not
provide any hint how a reasonable hint could be calculated.

> And how exactly would it do that, without either doing what __len__ does or
> reading the whole result set into memory?
> 
> Stefan
> 

a very good question.

Best,
Wolfgang

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Interesting list() un-optimization Roy Smith <roy@panix.com> - 2013-03-06 22:20 -0500
  Re: Interesting list() un-optimization Dave Angel <davea@davea.name> - 2013-03-06 22:38 -0500
  Re: Interesting list() un-optimization Tim Chase <python.list@tim.thechases.com> - 2013-03-06 21:57 -0600
  Re: Interesting list() un-optimization Kev Dwyer <kevin.p.dwyer@gmail.com> - 2013-03-07 07:31 +0000
  Re: Interesting list() un-optimization Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> - 2013-03-07 11:22 +0000
  Re: Interesting list() un-optimization Ian Kelly <ian.g.kelly@gmail.com> - 2013-03-07 09:00 -0700
  Re: Interesting list() un-optimization Christian Heimes <christian@python.org> - 2013-03-07 17:20 +0100
  Re: Interesting list() un-optimization Ian Kelly <ian.g.kelly@gmail.com> - 2013-03-07 10:31 -0700
  Re: Interesting list() un-optimization Stefan Behnel <stefan_ml@behnel.de> - 2013-03-07 20:19 +0100
  Re: Interesting list() un-optimization Ian Kelly <ian.g.kelly@gmail.com> - 2013-03-07 13:26 -0700
  Re: Interesting list() un-optimization Terry Reedy <tjreedy@udel.edu> - 2013-03-07 15:29 -0500
  Re: Interesting list() un-optimization Terry Reedy <tjreedy@udel.edu> - 2013-03-07 15:34 -0500
  Re: Interesting list() un-optimization Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> - 2013-03-07 20:41 +0000
  Re: Interesting list() un-optimization Terry Reedy <tjreedy@udel.edu> - 2013-03-07 17:53 -0500
  Re: Interesting list() un-optimization Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-03-08 17:59 +0000
    Re: Interesting list() un-optimization Roy Smith <roy@panix.com> - 2013-03-08 13:45 -0500
  Re: Interesting list() un-optimization Roy Smith <roy@panix.com> - 2013-03-10 09:05 -0400
    Re: Interesting list() un-optimization Terry Reedy <tjreedy@udel.edu> - 2013-03-10 17:39 -0400
      Re: Interesting list() un-optimization Roy Smith <roy@panix.com> - 2013-03-10 18:34 -0400
        Re: Interesting list() un-optimization Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-03-10 23:35 +0000
          Re: Interesting list() un-optimization Roy Smith <roy@panix.com> - 2013-03-10 19:50 -0400

csiph-web