Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #97964 > unrolled thread

Unittests and serial workflows

Back to article view | Back to comp.lang.python

Contents

  Unittests and serial workflows Fabien <fabien.maussion@gmail.com> - 2015-10-27 05:10 +0100
    Re: Unittests and serial workflows Paul Rubin <no.email@nospam.invalid> - 2015-10-26 23:18 -0700
      Re: Unittests and serial workflows Fabien <fabien.maussion@gmail.com> - 2015-10-30 16:05 +0100
        Re: Unittests and serial workflows Paul Rubin <no.email@nospam.invalid> - 2015-10-30 09:23 -0700
    Re: Unittests and serial workflows Steven D'Aprano <steve@pearwood.info> - 2015-10-31 12:28 +1100

#97964 — Unittests and serial workflows

Hi all,

I am not trained as a software developer and have only superficial 
knowledge about the theories and good practices of software testing.

Say I have several tasks (functions) A(), B(), C(), etc., each of them 
depending on the previous one. My understanding of unit testing is that 
each function should be tested alone and each test should be 
independent, so that currently my tests look like this:

class TestTasks(unittest.TestCase):

     def test_a(self):
         out = A()
         self.assertStuffs(out)

     def test_b(self):
         tmp = A()
         out = B(tmp)
         self.assertStuffs(out)

And so forth. The order in which unittest is running the test is not the 
one given by the code (it seems to be alphabetical?) so that I am not 
guaranteed that my functions are tested in the right order (my real 
tasks of course are not sorted alphabetically).

Furthermore, it is inefficient since task A() is going to be run many 
times. Of course I could run all the tasks once, store the results and 
run my tests afterwards one by one but this contradicts somehow the idea 
of unit testing. And I like to be able to run each test alone (in 
PyCharm a mouse click alone starts a single test).

My question(s): is Unittest the "right" framework for my needs? Should I 
move to Nosetests? Any hint to a useful, not too long document about 
testing framework and good practices?

Thanks a lot,

Fabien

[toc] | [next] | [standalone]

#97965

Fabien <fabien.maussion@gmail.com> writes:
> Say I have several tasks (functions) A(), B(), C(), etc., each of them
> depending on the previous one. My understanding of unit testing is
> that each function should be tested alone and each test should be
> independent, so that currently my tests look like this:

Part of the trick is to organize the code so the functions are
independent.

>     def test_b(self):
>         tmp = A()
>         out = B(tmp)
>         self.assertStuffs(out)

It's better to avoid that.  Write B so that it doesn't require input to
actually come from A, but rather, write or generate some other sample
input.  Then test B separately from A.  Then afterwards you can have
function C that calls both, and put a test case through that.

[toc] | [prev] | [next] | [standalone]

#98003

On 10/27/2015 07:18 AM, Paul Rubin wrote:
>> >     def test_b(self):
>> >         tmp = A()
>> >         out = B(tmp)
>> >         self.assertStuffs(out)
> It's better to avoid that.  Write B so that it doesn't require input to
> actually come from A, but rather, write or generate some other sample
> input.  Then test B separately from A.  Then afterwards you can have
> function C that calls both, and put a test case through that.

Thanks for your thoughts. I am not sure that this is always possible in 
my case, but I see what you mean.

Cheers,

Fabien

[toc] | [prev] | [next] | [standalone]

#98004

Fabien <fabien.maussion@gmail.com> writes:
>> Write B so that it doesn't require input to actually come from A, but
>> rather, write or generate some other sample input.  Then test B
>> separately from A.

> Thanks for your thoughts. I am not sure that this is always possible
> in my case, but I see what you mean.

It's generally difficult to take an existing program and add tests to
it, if that's what you're trying to do.  If you want a program with a
suite of tests, it's best to write the tests at the same time that you
write the code (so-called test-driven development or TDD).  Writing the
tests then steers the organization of the code, so that the functions
are independently testable.  You should also know about "mock objects"
which can supply interfaces that would normally expect to do (e.g.)
database queries or operations on a network.  I've generally written
those by hand, but I think there's a Python library module for them now.

[toc] | [prev] | [next] | [standalone]

#98015

On Tue, 27 Oct 2015 03:10 pm, Fabien wrote:

> Hi all,
> 
> I am not trained as a software developer and have only superficial
> knowledge about the theories and good practices of software testing.
> 
> Say I have several tasks (functions) A(), B(), C(), etc., each of them
> depending on the previous one. My understanding of unit testing is that
> each function should be tested alone and each test should be
> independent, 

Yes, that's the standard definition of unit testing.

However, unit testing frameworks are often perfectly well suited to other
forms of tests, like regression tests, integration tests, and others.
Sometimes it is perfectly reasonable to write a test which, strictly
speaking, doesn't fall under the unit test definition.



> so that currently my tests look like this: 
> 
> class TestTasks(unittest.TestCase):
> 
>      def test_a(self):
>          out = A()
>          self.assertStuffs(out)
> 
>      def test_b(self):
>          tmp = A()
>          out = B(tmp)
>          self.assertStuffs(out)

That's fine and meets the definition of a unit tests. Unit tests are allowed
to assume that other components of the module are working.

In test_b, you assume that A() is working correctly, and test B(). If A() is
not in fact working correctly, then the test may be invalid, or a bug in
A() may cause cascading failures: test_a will hopefully fail, and test_b
then fail as well.

Sometimes you want to avoid such cascading failures. To do that, rather than
call A() inside test_b, you create a simulation of A(), or hard-code a
value that could have been generated by A(). Such things are often called
mocks and stubs. Something like this perhaps:

    def test_a(self):
        out = A()
        self.assertEqual(out, (1, 2, 3, 4))

    def test_b(self):
        tmp = (1, 2, 3, 4)
        out = B(tmp)
        self.assertStuffs(out)


Now, even if A() is buggy and returns something invalid, the argument to B()
is fine and you can test B() in isolation to A().

 
> And so forth. The order in which unittest is running the test is not the
> one given by the code (it seems to be alphabetical?) so that I am not
> guaranteed that my functions are tested in the right order (my real
> tasks of course are not sorted alphabetically).

In my opinion, if your unit tests need to be run in a specific order, they
aren't sufficiently isolated. But, I accept that sometimes reality bites,
and you have to deal with tests which aren't sufficiently isolated. You
don't have time to fix the tests, and you can't fix the code you're
testing. Sucks to be you... 

It's times like that you want to use a *subclass* of unittest which gives
you a predictable order. Have fun writing one :-)


> Furthermore, it is inefficient since task A() is going to be run many
> times. 

You shouldn't care (too much) about the efficiency of tests. You don't want
them to be *horribly* inefficient, of course, but it it takes 30 seconds to
run the tests, who cares? Even a minute or so is no big deal.

A big test suite like that for Python itself, with dozens of modules to be
tested, might take ten minutes to run, just long enough for you to stretch
your legs, get the cobwebs out of your brain, and make a coffee.

Of course you need the ability to run individual test suites, for those
rapid debug/run tests/debug cycles. But don't worry about optimizing each
test. That's a waste of time, and likely to introduce bugs into the tests
themselves. Write your tests in the simplest, most fool-proof way you can,
and don't be too clever.


> Of course I could run all the tasks once, store the results and 
> run my tests afterwards one by one but this contradicts somehow the idea
> of unit testing. And I like to be able to run each test alone (in
> PyCharm a mouse click alone starts a single test).
> 
> My question(s): is Unittest the "right" framework for my needs? Should I
> move to Nosetests? Any hint to a useful, not too long document about
> testing framework and good practices?

http://thedailywtf.com/articles/Testing-Done-Right

And of course, read the Daily WTF for plenty of examples of what not to do.
They have plenty of examples of bad testing.




-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web