Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #18725

Re: replacing __dict__ with an OrderedDict

Show all headers | View raw

In article <11jrt8-l32.ln1@satorlaser.homedns.org>,
 Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> wrote:

> > Some people advocate that the test framework should
> > intentionally randomize the order, to flush out inter-test dependencies
> > that the author didn't realize existed (or intend).
> 
> If you now 
> happen to influence one test with another and the next run randomizes 
> the tests differently, you will never see the fault again. Without this 
> reproducability, you don't gain anything but the bad stomach feeling 
> that something is wrong.

The standard solution to that is to print out the PRNG initialization 
state and provide a way in your test harness to re-initialize it to that 
state.  I've done things like that in test scenarios where it is 
difficult or impossible to cover the problem space deterministically.

> Your unfortunate case is where test X creates persistent state that must 
> be present in order for test X+1 to produce meaningful results. This 
> kind of dependency obviously blows, as it means you can't debug test X+1 
> separately. I'd call this operational dependency.
> 
> This kind of dependency is IMHO a bug in the tests themselves.

For the most part, I'm inclined to agree.  However, there are scenarios 
where having each test build the required state from scratch is 
prohibitively expensive.  Imagine if you worked at NASA wanted to run 
test_booster_ignition(), test_booster_cutoff(), 
test_second_stage_ignition(), and test_self_destruct().  I suppose you 
could run them in random order, but you'd use up a lot of rockets that 
way.

Somewhat more seriously, let's say you wanted to do test queries against 
a database with 100 million records in it.  You could rebuild the 
database from scratch for each test, but doing so might take hours per 
test.  Sometimes, real life is just *so* inconvenient.

> There is another dependency and that I'd call a logical dependency. This 
> occurs when e.g. test X tests for an API presence and test Y tests the 
> API behaviour. In other words, Y has no chance to succeed if X already 
> failed.

Sure.  I run into that all the time.  A trivial example would be the 
project I'm working on now.  I've come to realize that a long unbroken 
string of E's means, "Dummy, you forgot to bring the application server 
up before you ran the tests".  It would be nicer if the test suite could 
have run a single test which proved it could create a TCP connection and 
when that failed, just stop.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-06 10:48 +0100
  Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-06 22:43 +1100
    Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-06 14:36 +0100
      Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-07 04:01 +1100
        Re: replacing __dict__ with an OrderedDict Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-01-07 00:45 +0000
      Re: replacing __dict__ with an OrderedDict Ian Kelly <ian.g.kelly@gmail.com> - 2012-01-06 10:20 -0700
        Re: replacing __dict__ with an OrderedDict Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-01-07 00:49 +0000
          Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-07 12:21 +1100
          Re: replacing __dict__ with an OrderedDict Ian Kelly <ian.g.kelly@gmail.com> - 2012-01-06 18:24 -0700
            Re: replacing __dict__ with an OrderedDict Eelco <hoogendoorn.eelco@gmail.com> - 2012-01-08 14:03 -0800
              Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-09 23:10 +1100
                Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-09 14:16 +0100
                Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-10 23:31 +1100
                Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-10 18:21 +0100
        Re: replacing __dict__ with an OrderedDict Roy Smith <roy@panix.com> - 2012-01-09 09:35 -0500
          Re: replacing __dict__ with an OrderedDict Neil Cerutti <neilc@norwich.edu> - 2012-01-09 14:52 +0000
          Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-09 17:59 +0100
            Re: replacing __dict__ with an OrderedDict Ian Kelly <ian.g.kelly@gmail.com> - 2012-01-09 10:30 -0700
            Re: replacing __dict__ with an OrderedDict Roy Smith <roy@panix.com> - 2012-01-09 20:05 -0500
              Re: replacing __dict__ with an OrderedDict Terry Reedy <tjreedy@udel.edu> - 2012-01-09 23:21 -0500
              Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-10 23:22 +1100
                Re: replacing __dict__ with an OrderedDict Roy Smith <roy@panix.com> - 2012-01-10 09:05 -0500
                Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-11 02:57 +1100
                Re: replacing __dict__ with an OrderedDict Roy Smith <roy@panix.com> - 2012-01-10 21:47 -0500
                Re: replacing __dict__ with an OrderedDict Tim Wintle <tim.wintle@teamrubber.com> - 2012-01-10 15:45 +0000
            Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-10 23:46 +1100
      Re: replacing __dict__ with an OrderedDict Lie Ryan <lie.1296@gmail.com> - 2012-01-07 04:42 +1100
  Re: replacing __dict__ with an OrderedDict Peter Otten <__peter__@web.de> - 2012-01-06 12:44 +0100
    Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-06 14:40 +0100
      Re: replacing __dict__ with an OrderedDict Ian Kelly <ian.g.kelly@gmail.com> - 2012-01-06 09:06 -0700
        Re: replacing __dict__ with an OrderedDict alex23 <wuwei23@gmail.com> - 2012-01-08 20:58 -0800
      Re: replacing __dict__ with an OrderedDict Ian Kelly <ian.g.kelly@gmail.com> - 2012-01-06 09:07 -0700
      Re: replacing __dict__ with an OrderedDict Arnaud Delobelle <arnodel@gmail.com> - 2012-01-06 21:38 +0000
    Re: replacing __dict__ with an OrderedDict Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-01-18 11:29 +0100
  Re: replacing __dict__ with an OrderedDict Ethan Furman <ethan@stoneleaf.us> - 2012-01-06 06:13 -0800
  Re: replacing __dict__ with an OrderedDict Arnaud Delobelle <arnodel@gmail.com> - 2012-01-06 21:32 +0000

csiph-web