Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15833

Re: unit-profiling, similar to unit-testing

Show all headers | View raw

On Wed, Nov 16, 2011 at 09:36:40AM -0500, Roy Smith wrote:
> In article <95bcp8-bft.ln1@satorlaser.homedns.org>,
>  Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> wrote:
> 
> > Hi!
> > 
> > I'm currently trying to establish a few tests here that evaluate certain 
> > performance characteristics of our systems. As part of this, I found 
> > that these tests are rather similar to unit-tests, only that they are 
> > much more fuzzy and obviously dependent on the systems involved, CPU 
> > load, network load, day of the week (Tuesday is virus scan day) etc.
> > 
> > What I'd just like to ask is how you do such things. Are there tools 
> > available that help? I was considering using the unit testing framework, 
> > but the problem with that is that the results are too hard to interpret 
> > programmatically and too easy to misinterpret manually. Any suggestions?
> 
> It's really, really, really hard to either control for, or accurately 
> measure, things like CPU or network load.  There's so much stuff you 
> can't even begin to see.  The state of your main memory cache.  Disk 
> fragmentation.  What I/O is happening directly out of kernel buffers vs 
> having to do a physical disk read.  How slow your DNS server is today.

While I agree there's a lot of things you can't control for, you can
get a more accurate picture by using CPU time instead of wall time
(e.g. the clock() system call). If what you care about is mostly CPU
time, you can control for the "your disk is fragmented", "your DNS
server died", or "my cow-orker was banging on the test machine" this
way.

\t

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

unit-profiling, similar to unit-testing Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-11-16 10:08 +0100
  Re: unit-profiling, similar to unit-testing Roy Smith <roy@panix.com> - 2011-11-16 09:36 -0500
    Re: unit-profiling, similar to unit-testing Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-11-17 09:53 +0100
      Re: unit-profiling, similar to unit-testing Roy Smith <roy@panix.com> - 2011-11-17 09:03 -0500
        Re: unit-profiling, similar to unit-testing "spartan.the" <spartan.the@gmail.com> - 2011-11-17 13:28 -0800
    Re: unit-profiling, similar to unit-testing Tycho Andersen <tycho@tycho.ws> - 2011-11-17 14:45 -0600
      Re: unit-profiling, similar to unit-testing Roy Smith <roy@panix.com> - 2011-11-17 21:00 -0500

csiph-web