Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder3.xlned.com!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Date: Wed, 27 Aug 2014 16:21:57 -0500
From: Tim Chase <python.list@tim.thechases.com>
To: Rodrick Brown <rodrick.brown@gmail.com>
Subject: Re: iterating over strings seems to be really slow?
In-Reply-To: <CABRP1o-X3wadeNh-7sf7XvEFGAus-QLQ=FTEab3hRT+y2EzyWA@mail.gmail.com>
References: <CABRP1o-X3wadeNh-7sf7XvEFGAus-QLQ=FTEab3hRT+y2EzyWA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: "python-list@python.org" <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.13525.1409174624.18130.python-list@python.org>
Lines: 56
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:77162

On 2014-08-27 16:53, Rodrick Brown wrote:
> *I'm confused why the former function runs significantly faster when
> wc1() builds the hash on a single pass and doesn't waste memory of
> returning an array of strings? *
> 
> *I would think wc2() to be slower what's going on here? *
> 
> 
> #!/usr/bin/env python
> 
> s = "The black cat jump over the bigger black cat"
> 
> 
> def wc1():
>     word=""
>     m={}
>     for c in s:
>         if c != " ":
>             word += c

String-building a character-at-a-time is slow.  Also, it doesn't
produce the same results as wc2() does.  Check

  if wc1() == wc2():
    print("Success")
  else:
    print("doh!")

> def wc2():
>     m={}
>     for c in s.split():
>         if m.has_key(c):
>             m[c] += 1
>         else:
>             m[c] = 1
>     return(m)

The thing that surprises me is that using collections.Counter() and
collections.defaultdict(int) are also slower than your wc2():

from collections import defaultdict, Counter

def wc3():
    return Counter(s.split())

def wc4():
    m = defaultdict(int)
    for c in s.split():
        m[c] += 1
    return m

-tkc