Re: Benchmarking some modules - strange result

Path	csiph.com!usenet.pasdenom.info!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<davea@davea.name>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.004
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'offline': 0.03; 'interpreter': 0.05; 'sufficient': 0.05; 'interpreter.': 0.07; 'linux,': 0.07; 'dan': 0.09; 'imported': 0.09; 'restart': 0.09; 'subject:modules': 0.09; 'weeks,': 0.09; 'windows,': 0.09; 'runs': 0.10; 'python': 0.11; 'jan': 0.12; '24,': 0.16; 'afterwards': 0.16; 'exiting': 0.16; 'guessing': 0.16; 'simplicity,': 0.16; 'sure.': 0.16; 'sys.modules': 0.16; 'sat,': 0.16; 'wrote:': 0.18; 'module': 0.19; 'pieces': 0.19; 'things.': 0.19; '>>>': 0.22; 'memory': 0.22; 'import': 0.22; 'reset': 0.22; 'tests': 0.22; 'header:User-Agent:1': 0.23; 'script.': 0.24; 'initial': 0.24; "haven't": 0.24; "i've": 0.25; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'url:code': 0.29; "i'm": 0.30; 'too.': 0.31; '25,': 0.31; 'linux.': 0.31; 'staying': 0.31; 'subject:some': 0.31; 'file': 0.32; 'stuff': 0.32; 'running': 0.33; 'ago': 0.33; 'beginning': 0.33; 'maybe': 0.34; "i'd": 0.34; 'could': 0.34; 'created': 0.35; 'test': 0.35; 'but': 0.35; 'there': 0.35; 'next': 0.36; 'url:microsoft': 0.37; 'two': 0.37; 'clear': 0.37; 'performance': 0.37; 'easiest': 0.38; 'needed': 0.38; 'to:addr :python-list': 0.38; 'pm,': 0.38; 'help,': 0.39; 'to:addr:python.org': 0.39; 'how': 0.40; 'truly': 0.60; 'affect': 0.61; 'first': 0.61; 'such': 0.63; 'url:p': 0.64; 'more': 0.64; 'charset:windows-1252': 0.65; 'within': 0.65; 'website:': 0.67; 'between': 0.67; 'believe': 0.68; 'received:74.208': 0.68; 'url :en-us': 0.68; 'allocation': 0.74; 'further,': 0.74; '2015': 0.84; '^_^': 0.84; 'received:74.208.4.194': 0.84; 'responding.': 0.84; 'routines': 0.84; 'utilities,': 0.84
Date	Tue, 03 Feb 2015 16:00:00 -0500
From	Dave Angel <davea@davea.name>
User-Agent	Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version	1.0
To	python-list@python.org
Subject	Re: Benchmarking some modules - strange result
References	<CAGGBd_rSazrC4yAt+CebeuvCJYjZWOXohoZHgTUtfGNZZtr5Ww@mail.gmail.com> <CAPTjJmr+9heLd44Ly=5oKrMcGBdBhWuDxmi_n1+izg=TxTB=sg@mail.gmail.com> <CAGGBd_q33YGH9ZwOv4oTE6LERxbf55HyOM07TpWznjMumz1yKw@mail.gmail.com>
In-Reply-To	<CAGGBd_q33YGH9ZwOv4oTE6LERxbf55HyOM07TpWznjMumz1yKw@mail.gmail.com>
Content-Type	text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding	7bit
X-Provags-ID	V02:K0:G7URmFXASrkdmCYcf8kRNv9DttMifkBvFKUqGdUtpBe OnxBpBp8Ul599Il2P53T+5oKGNI2Mh+B5hSg6dDNVIE1ppeVGr /aZFoIm905rjadyd5dTRkRd9IyBYbtnFIiW1rmYuKHd3eOaPbN 7eyBcs9Ynuf47Y/incqBsyQbP+joeQqStLZhS6h5FnMEB/Q68j BgeeNVc0gMy69iZXgC5rjSLyl9YIaOSeUGQf0hbburUmEc4uYT hWa2MxExEpmMk/mTtz49/+BZlUTXjMA9HcKD3wqAZpywfUmfbe neVdo80tGIwu2lGTIWUGmYbK+eyWE8jBzLIGCzlevaZ75VNdg= =
X-UI-Out-Filterresults	notjunk:1;
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.18440.1422997212.18130.python-list@python.org> (permalink)
Lines	46
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1422997212 news.xs4all.nl 2851 [2001:888:2000:d::a6]:43712
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:85170

Show key headers only | View raw

On 01/24/2015 09:36 PM, Dan Stromberg wrote:
> On Sat, Jan 24, 2015 at 6:24 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> On Sun, Jan 25, 2015 at 1:11 PM, Dan Stromberg <drsalists@gmail.com> wrote:
>>> For simplicity, let's say I've been running the suite of performance
>>> tests within a single interpreter - so I test one module thoroughly,
>>> then move on to the next without exiting the interpreter.
>>> [chomp more details]
>>
>> Do the modules import stuff that's staying imported? That could
>> majorly affect performance. What about .pyc files?
>
> The modules are all imported up at the top of the script.  I haven't
> verified this, but I'm guessing the .pyc's are created at the
> beginning too.
>
>> I don't know if you can truly reset the interpreter. Snapshotting
>> sys.modules and reinstating it afterwards would help, but there are
>> other concerns too. I would personally just restart the interpreter
>> each time, frankly; it's the easiest way to be sure.
>
> Maybe so ^_^
>

I've been offline for two weeks, so sorry about the delay in responding.

There are many pieces of the Python system that do not reset.  But just 
because the C memory allocation routines do not reset to their initial 
state, running the gc between runs is not sufficient to normalize 
things.  I believe you MUST restart the interpreter between runs.

Further, the OS buffer system and the paging system are not going to be 
in the same state between runs.  years ago I knew how to clear those 
buffers in Windows, but I've never needed to look it up for Linux.

in Windows, I'd look first at one of the  sysinternalstools, such as
https://technet.microsoft.com/en-us/sysinternals/ff700229

On Linux, i'd look at the following utilities,  swapon/swapoff,

and at file  /proc/sys/vm/drop_caches
and maybe see website:  https://code.google.com/p/linux-ftools/

-- 
DaveA

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Benchmarking some modules - strange result Dave Angel <davea@davea.name> - 2015-02-03 16:00 -0500

csiph-web