Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35246

Re: help with making my code more efficient

Path csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <rosuav@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.008
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'memory.': 0.05; '21,': 0.07; 'subject:code': 0.07; 'subject:help': 0.07; 'dec': 0.15; '11:19': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'run.': 0.16; 'subject:making': 0.16; 'wrote:': 0.17; 'typical': 0.17; 'elements': 0.23; 'header:In-Reply-To:1': 0.25; '(which': 0.26; 'am,': 0.27; 'message-id:@mail.gmail.com': 0.27; 'run': 0.28; 'e.g.': 0.30; 'fri,': 0.30; 'code': 0.31; 'could': 0.32; 'anyone': 0.33; 'to:addr:python-list': 0.33; 'received:google.com': 0.34; 'received:209.85': 0.35; 'but': 0.36; 'subject:with': 0.36; 'too': 0.36; 'enough': 0.36; 'execute': 0.37; 'received:209': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'some': 0.38; 'to:addr:python.org': 0.39; 'takes': 0.39; 'header:Received:5': 0.40; 'email addr:gmail.com': 0.63; 'hours': 0.66; 'sounds': 0.71
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0IG7p2MtGRIYwrEnETZMAhX/tu1KtDybRN67v/P3TzU=; b=lue1aFditIXtwjej+qFQ9ZvBW5xIeG25L40bAXN1IeClospPj+xbwRm4tGISIkvI1u Ffn2o/eKukq1MBsv3kYEIBYveGx+745xHx0KPfJrZ5/d9VRAOnAQ8Oz2az2sJN5NCJ66 KTS9PIraFDdLIsHUJLUI8PuhozPS8n/QxCNIEN5A8GSP41CfkYZxqjfKRsiaX02t3UGD oywGuqwPlumiWGT4+XU7jALyGtB3ekFV5j/4GIIVsyt+QVFhFWhgZfMTv3kP5nyGoiqk dDstcXAqORi7+raNkleWC/M2Vz4p7kCnkvvK/txj1yW6FaoN4YFSb4T97UspsoImABIX 5YRQ==
MIME-Version 1.0
In-Reply-To <bd5ae6b7-2440-42e4-a93c-eb877feebcfe@googlegroups.com>
References <bd5ae6b7-2440-42e4-a93c-eb877feebcfe@googlegroups.com>
Date Fri, 21 Dec 2012 11:38:03 +1100
Subject Re: help with making my code more efficient
From Chris Angelico <rosuav@gmail.com>
To python-list@python.org
Content-Type text/plain; charset=ISO-8859-1
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.1123.1356050292.29569.python-list@python.org> (permalink)
Lines 12
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1356050292 news.xs4all.nl 6853 [2001:888:2000:d::a6]:39992
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:35246

Show key headers only | View raw

On Fri, Dec 21, 2012 at 11:19 AM, Larry.Martell@gmail.com
<Larry.Martell@gmail.com> wrote:
> This code works, but it takes way too long to run - e.g. when cdata has 600,000 elements (which is typical for my app) it takes 2 hours for this to run.
>
> Can anyone give me some suggestions on speeding this up?
>

It sounds like you may have enough data to want to not keep it all in
memory. Have you considered switching to a database? You could then
execute SQL queries against it.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-20 16:19 -0800
  Re: help with making my code more efficient Chris Angelico <rosuav@gmail.com> - 2012-12-21 11:38 +1100
    Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-20 16:43 -0800
      Re: help with making my code more efficient Chris Angelico <rosuav@gmail.com> - 2012-12-21 13:56 +1100
      Re: help with making my code more efficient Roy Smith <roy@panix.com> - 2012-12-20 22:30 -0500
    Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-20 16:43 -0800
  Re: help with making my code more efficient Dave Angel <d@davea.name> - 2012-12-20 20:17 -0500
    Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-20 17:46 -0800
      Re: help with making my code more efficient Mitya Sirenef <msirenef@lightbird.net> - 2012-12-20 21:39 -0500
      Re: help with making my code more efficient Mitya Sirenef <msirenef@lightbird.net> - 2012-12-20 21:49 -0500
      Re: help with making my code more efficient Dave Angel <d@davea.name> - 2012-12-20 22:31 -0500
        Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 09:57 -0800
        Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 09:57 -0800
          Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 12:36 -0800
            Re: help with making my code more efficient Dave Angel <d@davea.name> - 2012-12-21 22:19 -0500
              Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 20:47 -0800
                Re: help with making my code more efficient Dave Angel <d@davea.name> - 2012-12-22 01:47 -0500
                Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-24 09:57 -0800
                Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-24 09:57 -0800
              Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 20:47 -0800
          Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-21 12:36 -0800
    Re: help with making my code more efficient "Larry.Martell@gmail.com" <Larry.Martell@gmail.com> - 2012-12-20 17:46 -0800
  Re: help with making my code more efficient MRAB <python@mrabarnett.plus.com> - 2012-12-21 02:08 +0000

csiph-web