Path: csiph.com!usenet.pasdenom.info!gegeweb.org!newsfeed.kamp.net!newsfeed.kamp.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Mon, 22 Apr 2013 08:51:49 -0400
From: Dave Angel <davea@davea.name>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130308 Thunderbird/17.0.4
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: List Count
References: <sridnffI6YhxuOjMnZ2dnUVZ7tSdnZ2d@brightview.co.uk>
In-Reply-To: <sridnffI6YhxuOjMnZ2dnUVZ7tSdnZ2d@brightview.co.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.907.1366635127.3114.python-list@python.org>
Lines: 54
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:44057

On 04/22/2013 07:58 AM, Blind Anagram wrote:
> I would be grateful for any advice people can offer on the fastest way
> to count items in a sub-sequence of a large list.
>
> I have a list of boolean values that can contain many hundreds of
> millions of elements for which I want to count the number of True values
> in a sub-sequence, one from the start up to some value (say hi).
>
> I am currently using:
>
>     sieve[:hi].count(True)
>
> but I believe this may be costly because it copies a possibly large part
> of the sieve.
>
> Ideally I would like to be able to use:
>
>     sieve.count(True, hi)
>
> where 'hi' sets the end of the count but this function is, sadly, not
> available for lists.
>
> The use of a bytearray with a memoryview object instead of a list solves
> this particular problem but it is not a solution for me as it creates
> more problems than it solves in other aspects of the program.
>
> Can I assume that one possible solution would be to sub-class list and
> create a C based extension to provide list.count(value, limit)?
>
> Are there any other solutions that will avoid copying a large part of
> the list?
>

Instead of using the default slice notation, why not use 
itertools.islice() ?

Something like  (untested):

import itertools

it = itertools.islice(sieve, 0, hi)
sum(itertools.imap(bool, it))

I only broke it into two lines for clarity.  It could also be:

sum(itertools.imap(bool, itertools.islice(sieve, 0, hi)))

If you're using Python 3.x, say so, and I'm sure somebody can simplify 
these, since in Python 3, many functions already produce iterators 
instead of lists.


-- 
DaveA