Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.005 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'subsequent': 0.05; '(python': 0.07; 'clause': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'second.': 0.09; 'seemed': 0.09; 'statements': 0.09; 'mostly': 0.14; '(apologies': 0.16; 'arrived': 0.16; 'delay,': 0.16; 'message-id:@4ax.com': 0.16; 'packets.': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'throughput': 0.16; 'exception': 0.16; 'proprietary': 0.16; 'bit': 0.19; 'thu,': 0.19; 'seems': 0.21; 'platforms': 0.22; 'print': 0.22; 'url:home': 0.24; 'sort': 0.25; 'logging': 0.26; 'pass': 0.26; 'header:X-Complaints-To:1': 0.27; "i'm": 0.30; 'went': 0.31; 'code': 0.31; 'handled': 0.32; 'candidate': 0.34; 'core': 0.34; 'could': 0.34; 'but': 0.35; 'collecting': 0.36; 'skip:> 10': 0.36; "didn't": 0.36; 'charset:us-ascii': 0.36; 'seconds': 0.37; 'application': 0.37; 'level': 0.37; 'to:addr :python-list': 0.38; 'anything': 0.39; 'to:addr:python.org': 0.39; 'enough': 0.39; 'received:org': 0.40; 'even': 0.60; 'removing': 0.60; 'hardware': 0.61; 'first': 0.61; 'information': 0.63; 'our': 0.64; 'more': 0.64; 'details': 0.65; 'effectively': 0.66; 'biggest': 0.67; 'delaying': 0.84; 'grew': 0.84; 'resulted': 0.84; 'vagueness,': 0.84; 'received:108': 0.93 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Dennis Lee Bieber Subject: Re: print statements and profiling a function slowed performance Date: Thu, 26 Jun 2014 22:11:42 -0400 Organization: IISS Elusive Unicorn References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Gmane-NNTP-Posting-Host: adsl-108-79-216-111.dsl.klmzmi.sbcglobal.net X-Newsreader: Forte Agent 6.00/32.1186 X-No-Archive: YES X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 35 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1403835113 news.xs4all.nl 2839 [2001:888:2000:d::a6]:58338 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:73646 On Thu, 26 Jun 2014 11:44:29 -0700 (PDT), CM declaimed the following: >1. print statements were slowing down my code enough to >really notice a particular transition. It went from about >2-3 seconds to a bit under 1 second. What at first seemed >unresponsive now seems almost snappy. The only difference >was removing a lot of print statements I had used for >debugging (Python 2.5, on a single core 1.97 Ghz machine). > Lots of I/O will slow anything down -- consider; every I/O operation will trigger an OS level process swap, since they are synchronous and handled by the OS. Last fall I was collecting data for a number of candidate hardware platforms (apologies for the vagueness, but I suspect the details would fall under some sort of proprietary information clause at my place of employment). The biggest finding was that the console (and even disk) logging of actions* resulted in a large latency in the throughput of those hardware platforms. * I'm stretching to reveal that what the platforms were logging were considered "exceptional" events by the providers, but for our application would be "normal"; hence we were effectively flooding them with lots of data that triggered exception processing and associated logging. Data that didn't trigger the exception could pass with less than 1mSec delay, but once a batch of exception data arrived the delays grew to 10mSec or more -- mostly as the log I/O was delaying processing of subsequent packets. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/