Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.028 X-Spam-Evidence: '*H*': 0.94; '*S*': 0.00; 'python.': 0.02; 'problem?': 0.07; 'subject:code': 0.07; 'suddenly': 0.07; 'facts': 0.09; 'inherited': 0.09; 'logic': 0.09; 'optimizing': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'changes': 0.15; '"things': 0.16; 'big,': 0.16; 'chunks': 0.16; 'dig': 0.16; 'does,': 0.16; 'pep8': 0.16; 'styling': 0.16; 'subject:python': 0.16; 'appropriate': 0.16; 'code.': 0.18; 'all,': 0.19; 'small,': 0.19; 'thoughts': 0.19; 'tests': 0.22; 'cc:addr:python.org': 0.22; '(or': 0.24; '---': 0.24; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; "i've": 0.25; '15,': 0.26; 'changes,': 0.26; 'first,': 0.26; 'least': 0.26; 'header:In-Reply-To:1': 0.27; 'function': 0.29; 'wondering': 0.29; 'scanned': 0.29; "i'm": 0.30; 'url:mailman': 0.30; 'code': 0.31; 'app.': 0.31; 'loading': 0.31; 'please.': 0.31; 'ray': 0.31; 'anyone': 0.31; 'run': 0.32; 'says': 0.33; 'up.': 0.33; 'url:python': 0.33; 'running': 0.33; '-----': 0.33; 'style': 0.33; 'problem': 0.35; 'something': 0.35; 'definition': 0.35; 'add': 0.35; 'version': 0.36; 'really': 0.36; 'done': 0.36; 'url:listinfo': 0.36; 'useful': 0.36; 'similar': 0.36; 'url:org': 0.36; 'unit': 0.37; 'application': 0.37; 'email addr:python.org': 0.37; 'clear': 0.37; 'step': 0.37; 'being': 0.38; 'bringing': 0.38; 'solving': 0.38; 'stopped': 0.38; 'track': 0.38; 'subject:': 0.39; 'aside': 0.39; 'extremely': 0.39; 'functional': 0.39; 'sure': 0.39; 'url:mail': 0.40; 'how': 0.40; 'ensure': 0.60; 'even': 0.60; 'skip:u 10': 0.60; 'most': 0.60; 'helps': 0.61; 'profile': 0.61; "you're": 0.61; "you'll": 0.62; "you've": 0.63; 'making': 0.63; 'address': 0.63; 'personal': 0.63; 'such': 0.63; 'myself': 0.63; 'taking': 0.65; 'email name:python-list': 0.65; 'to:addr:gmail.com': 0.65; 'techniques': 0.66; 'here': 0.66; 'biggest': 0.67; 'header:Reply-To:1': 0.67; 'improvements': 0.68; 'carefully': 0.74; 'behavior': 0.77; 'longest': 0.84; 'missing.': 0.84; 'subject:base': 0.84; 'visually': 0.84; 'approached': 0.93 X-Virus-Scanned: amavisd-new at appropriatesolutions.com Date: Sat, 15 Feb 2014 13:04:43 -0500 (EST) From: Ray Cote To: Rita In-Reply-To: Subject: Re: inheriting a large python code base MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [71.168.95.54] X-Mailer: Zimbra 7.2.3_GA_2872 (Zimbra Desktop/7.2.3_11971_Mac) Cc: python-list@python.org X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ray Cote List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 98 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1392488049 news.xs4all.nl 2863 [2001:888:2000:d::a6]:59329 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:66467 Hi Rita: Some personal thoughts on how I've approached similar situations: Step 0: Put all that code under version control immediately.=20 Being able to track and back-out changes is extremely useful -- even in a s= mall, single-coder app.=20 With something this big, you'll find "things suddenly stopped working" as y= ou make changes, and being able to back-track your changes is invaluatable.=20 Step 2: Ask, "Why do you want to optimize the code (run faster)?" - what is slow? - starting? - loading data? - solving specific problem? - running specific functionality? - specific calculations? - network access? - database access? - small data sets? - large data sets? - who says it is slow?=20 and what is their definition of sufficiently quick? 3: You state you do not understand the internal logic.=20 I've also inherited applications where the internal logic was unclear.=20 Writing unit and functional tests (if they do not exist) will help to ensur= e you do not change the behavior of the application.=20 You may find that understanding the internal logic is the best way to speed= -up the application.=20 You'll likely find that the biggest speed improvements come from optimizing= the logic -- not the Python. 4: This is a personal prefernce when inheriting large code chunks -- and I'= m sure some will counter it is not optimum.=20 Go through the code and run pep8 and pyflakes (or flake8) on all the code a= nd address the issues that come up.=20 Aside from bringing the code into styling conformance, I find that this hel= ps me to familiarize myself with the code.=20 As you're making style changes, review individual functions and add documen= tation if it is missing.=20 If you're not clear what a function does, mark it as such so you know to lo= ok at it carefully in the future.=20 5: Now you have a clean set of code -- most of which you've at least visual= ly scanned -- and you're=20 ready to create some timing tests and dig into specifics. --Ray ----- Original Message ----- From: "Rita" To: python-list@python.org Sent: Saturday, February 15, 2014 12:10:33 PM Subject: inheriting a large python code base hi all,=20 i just inherited a large python code base and I would like to optimize the = code (run faster). The application is a scientific application so I really = don't understand the internal logic.=20 Here is what I have done so far, profile (cProfile) and was able to see whe= re the code was taking the longest time.=20 I was wondering if anyone else experienced something like this and how they= overcame a problem such a this=E2=80=A6what techniques have you used to "o= ptimize" the code.=20 --=20 --- Get your facts first, then you can distort them as you please. --=20 --=20 https://mail.python.org/mailman/listinfo/python-list --=20 Ray Cote, President Appropriate Solutions, Inc. We Build Software 603.924.6079