Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #82263

Re: Nested loops is strangely slow, totally at a loss.

Show all headers | View raw

On Wed, Dec 10, 2014 at 4:20 PM, Shiyao Ma <i@introo.me> wrote:
> from itertools import product
> space_len = 580
> space = product(xrange(space_len), xrange(space_len), xrange(space_len))
>
> sparse_cloud = product(xrange(1000), xrange(1000), xrange(1000))
> for i, j, k in sparse_cloud:
>     ts = timeit.default_timer()
>     if (i, j, k) in space: pass
>     te = timeit.default_timer()

On Wed, Dec 10, 2014 at 4:23 PM, Shiyao Ma <i@introo.me> wrote:
> One thing to note, the logic of using "in" is not of concern here.
> This is a *contrived* example, the problem is the slowness of the first
> iteration.

Are you sure it isn't? Your 'space' is an iterable cubic
cross-product. Your first loop checks (0,0,0) which is the first
element returned, and is thus fast... but it also *consumes* that
first element. The next time you test it, the entire space gets
consumed, looking for another (0,0,0), which won't exist. That means
iterating over 580**3 == 195112000 (two hundred million) tuples, and
that *is* going to be slow.

Is product() really the right way to represent your space?

ChrisA

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Nested loops is strangely slow, totally at a loss. Chris Angelico <rosuav@gmail.com> - 2014-12-10 17:30 +1100

csiph-web