Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #30631 > unrolled thread

unit testing class hierarchies

Back to article view | Back to comp.lang.python

Contents

  unit testing class hierarchies Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-10-02 14:27 +0200
    Re: unit testing class hierarchies Demian Brecht <demianbrecht@gmail.com> - 2012-10-02 07:05 -0700
    Re: unit testing class hierarchies Thomas Bach <thbach@students.uni-mainz.de> - 2012-10-02 16:06 +0200
      Re: unit testing class hierarchies Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-10-02 17:24 +0200
      Re: unit testing class hierarchies Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-10-02 18:54 +0200
        Re: unit testing class hierarchies Peter Otten <__peter__@web.de> - 2012-10-02 19:38 +0200
        Re: unit testing class hierarchies Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-02 19:41 +0100
        Re: unit testing class hierarchies Ben Finney <ben+python@benfinney.id.au> - 2012-10-03 08:30 +1000
          Re: unit testing class hierarchies Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-10-03 01:20 +0000
            Re: unit testing class hierarchies Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-10-03 10:33 +0100
            Re: unit testing class hierarchies Terry Reedy <tjreedy@udel.edu> - 2012-10-03 16:14 -0400
    Re: unit testing class hierarchies Peter Otten <__peter__@web.de> - 2012-10-02 16:32 +0200
    Re: unit testing class hierarchies Fayaz Yusuf Khan <fayaz@dexetra.com> - 2012-10-02 20:35 +0530
    Re: unit testing class hierarchies Peter Otten <__peter__@web.de> - 2012-10-02 19:40 +0200
      Re: unit testing class hierarchies Roy Smith <roy@panix.com> - 2012-10-02 19:46 -0400

#30631 — unit testing class hierarchies

Greetings!

I'm trying to unittest a class hierachy using Python 2.7. I have a 
common baseclass Base and derived classes D1 and D2 that I want to test. 
The baseclass in not instantiatable on its own. Now, the first approach 
is to have test cases TestD1 and TestD2, both derived from class TestCase:

class TestD1(unittest.TestCase):
     def test_base(self):
         ...
     def test_r(self):
         ...
     def test_s(self):
         ...

class TestD2(unittest.TestCase):
     def test_base(self):
         # same as above
         ...
     def test_x(self):
         ...
     def test_y(self):
         ...

As you see, the code for test_base() is redundant, so the idea is to 
move it to a baseclass:

class TestBase(unittest.TestCase):
     def test_base(self):
         ...

class TestD1(TestBase):
     def test_r(self):
         ...
     def test_s(self):
         ...

class TestD2(TestBase):
     def test_x(self):
         ...
     def test_y(self):
         ...

The problem here is that TestBase is not a complete test case (just as 
class Base is not complete), but the unittest framework will still try 
to run it on its own. One way around this is to not derive class 
TestBase from unittest.TestCase but instead use multiple inheritance in 
the derived classes [1]. Maybe it's just my personal gut feeling, but I 
don't like that solution, because it is not obvious that this class 
actually needs to be combined with a TestCase class in order to 
function. I would rather tell the unittest framework directly that it's 
not supposed to consider this intermediate class as a test case, but 
couldn't find a way to express that clearly.

How would you do this?

Uli


[1] in C++ I would call that a "mixin"

[toc] | [next] | [standalone]

#30633

> [1] in C++ I would call that a "mixin"

Mixins are perfectly valid Python constructs as well and are perfectly 
valid (imho) for this use case.

On a side note, I usually append a "Mixin" suffix to my mixin classes in 
order to make it obvious to the reader.



-- 
Demian Brecht
@demianbrecht
http://demianbrecht.github.com

[toc] | [prev] | [next] | [standalone]

#30634

On Tue, Oct 02, 2012 at 02:27:11PM +0200, Ulrich Eckhardt wrote:
> As you see, the code for test_base() is redundant, so the idea is to
> move it to a baseclass:
> 
> class TestBase(unittest.TestCase):
>     def test_base(self):
>         ...
> 
> class TestD1(TestBase):
>     def test_r(self):
>         ...
>     def test_s(self):
>         ...
> 
> class TestD2(TestBase):
>     def test_x(self):
>         ...
>     def test_y(self):
>         ...

Could you provide more background? How do you avoid that test_base()
runs in TestD1 or TestD2?

To me it sounds like test_base() is actually no test. Hence, I would
rather give it a catchy name like _build_base_cls().  If a method name
does not start with 'test' it is not considered a test to run
automatically.

Does this help?

Regards,
	Thomas Bach.

[toc] | [prev] | [next] | [standalone]

#30641

Am 02.10.2012 16:06, schrieb Thomas Bach:
> On Tue, Oct 02, 2012 at 02:27:11PM +0200, Ulrich Eckhardt wrote:
>> As you see, the code for test_base() is redundant, so the idea is to
>> move it to a baseclass:
>>
>> class TestBase(unittest.TestCase):
>>      def test_base(self):
>>          ...
>>
>> class TestD1(TestBase):
>>      def test_r(self):
>>          ...
>>      def test_s(self):
>>          ...
>>
>> class TestD2(TestBase):
>>      def test_x(self):
>>          ...
>>      def test_y(self):
>>          ...
>
> Could you provide more background? How do you avoid that test_base()
> runs in TestD1 or TestD2?

Sorry, there's a misunderstanding: I want test_base() to be run as part 
of both TestD1 and TestD2, because it tests basic functions provided by 
both class D1 and D2.

Uli

[toc] | [prev] | [next] | [standalone]

#30646

Am 02.10.2012 16:06, schrieb Thomas Bach:
> On Tue, Oct 02, 2012 at 02:27:11PM +0200, Ulrich Eckhardt wrote:
>> As you see, the code for test_base() is redundant, so the idea is to
>> move it to a baseclass:
>>
>> class TestBase(unittest.TestCase):
>>      def test_base(self):
>>          ...
>>
>> class TestD1(TestBase):
>>      def test_r(self):
>>          ...
>>      def test_s(self):
>>          ...
>>
>> class TestD2(TestBase):
>>      def test_x(self):
>>          ...
>>      def test_y(self):
>>          ...
>
> Could you provide more background? How do you avoid that test_base()
> runs in TestD1 or TestD2?

Sorry, there's a misunderstanding: I want test_base() to be run as part 
of both TestD1 and TestD2, because it tests basic functions provided by 
both classes D1 and D2. The instances of D1 and D2 are created in 
TestD1.setUp and TestD2.setUp and then used by all tests. There is no 
possible implementation creating such an instance for TestBase, since 
the baseclass is abstract.

Last edit for today, I hope that makes my intentions clear...

;)

Uli

[toc] | [prev] | [next] | [standalone]

#30647

Ulrich Eckhardt wrote:

> Am 02.10.2012 16:06, schrieb Thomas Bach:
>> On Tue, Oct 02, 2012 at 02:27:11PM +0200, Ulrich Eckhardt wrote:
>>> As you see, the code for test_base() is redundant, so the idea is to
>>> move it to a baseclass:
>>>
>>> class TestBase(unittest.TestCase):
>>>      def test_base(self):
>>>          ...
>>>
>>> class TestD1(TestBase):
>>>      def test_r(self):
>>>          ...
>>>      def test_s(self):
>>>          ...
>>>
>>> class TestD2(TestBase):
>>>      def test_x(self):
>>>          ...
>>>      def test_y(self):
>>>          ...
>>
>> Could you provide more background? How do you avoid that test_base()
>> runs in TestD1 or TestD2?
> 
> Sorry, there's a misunderstanding: I want test_base() to be run as part
> of both TestD1 and TestD2, because it tests basic functions provided by
> both classes D1 and D2. The instances of D1 and D2 are created in
> TestD1.setUp and TestD2.setUp and then used by all tests. There is no
> possible implementation creating such an instance for TestBase, since
> the baseclass is abstract.
> 
> Last edit for today, I hope that makes my intentions clear...
> 
> ;)

Ceterum censeo baseclassinem esse delendam ;)

$ cat test_shared.py
import unittest

class Shared(unittest.TestCase):
    def test_shared(self):
        pass

class D1(Shared):
    def test_d1_only(self):
        pass

class D2(Shared):
    def test_d2_only(self):
        pass

del Shared

unittest.main()

$ python test_shared.py -v
test_d1_only (__main__.D1) ... ok
test_shared (__main__.D1) ... ok
test_d2_only (__main__.D2) ... ok
test_shared (__main__.D2) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK
$ 

[toc] | [prev] | [next] | [standalone]

#30651

On 02/10/2012 19:06, Demian Brecht wrote:
> Am I missing something? Is there something that wasn't answered by my reply
> about using mixins?
>
> from unittest import TestCase
>
> class SharedTestMixin(object):
>      def test_shared(self):
>          self.assertNotEquals('foo', 'bar')
>
> class TestA(TestCase, SharedTestMixin):
>      def test_a(self):
>          self.assertEquals('a', 'a')
>
> class TestB(TestCase, SharedTestMixin):
>      def test_b(self):
>          self.assertEquals('b', 'b')
>
> $ nosetests test.py -v
> test_a (test.TestA) ... ok
> test_shared (test.TestA) ... ok
> test_b (test.TestB) ... ok
> test_shared (test.TestB) ... ok
>
> ----------------------------------------------------------------------
> Ran 4 tests in 0.001s
>
> OK
>
> This seems to be a clear answer to the problem that solves the original
> requirements without introducing error-prone, non-obvious solutions.
>
>
>

Peter Otten's response is obviously vastly superior to yours, 4 tests in 
0.000s compared to your highly inefficient 4 tests in 0.001s :)

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30660

Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> writes:

> I want test_base() to be run as part of both TestD1 and TestD2,
> because it tests basic functions provided by both classes D1 and D2.

It sounds, from your description so far, that you have identified a
design flaw in D1 and D2.

The common functionality should be moved to a common code point (maybe a
base class of D1 and D2; maybe a function without need of a class). Then
you'll have only one occurrence of that functionality to test, which is
good design as well as easier test code :-)

-- 
 \      “When I was a little kid we had a sand box. It was a quicksand |
  `\           box. I was an only child... eventually.” —Steven Wright |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#30666

On Wed, 03 Oct 2012 08:30:19 +1000, Ben Finney wrote:

> Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> writes:
> 
>> I want test_base() to be run as part of both TestD1 and TestD2, because
>> it tests basic functions provided by both classes D1 and D2.
> 
> It sounds, from your description so far, that you have identified a
> design flaw in D1 and D2.
> 
> The common functionality should be moved to a common code point (maybe a
> base class of D1 and D2; maybe a function without need of a class). Then
> you'll have only one occurrence of that functionality to test, which is
> good design as well as easier test code :-)

But surely, regardless of where that functionality is defined, you still 
need to test that both D1 and D2 exhibit the correct behaviour? Otherwise 
D2 (say) may break that functionality and your tests won't notice.

Given a class hierarchy like this:

class AbstractBaseClass:
    spam = "spam"

class D1(AbstractBaseClass): pass
class D2(D1): pass


I write tests like this:

class TestD1CommonBehaviour(unittest.TestCase):
    cls = D1
    def testSpam(self):
         self.assertTrue(self.cls.spam == "spam")
    def testHam(self):
         self.assertFalse(hasattr(self.cls, 'ham'))
    
class TestD2CommonBehaviour(TestD1CommonBehaviour):
    cls = D2

class TestD1SpecialBehaviour(unittest.TestCase):
    # D1 specific tests here

class TestD2SpecialBehaviour(unittest.TestCase):
    # D2 specific tests here



If D2 doesn't inherit from D1, but both from AbstractBaseClass, I need to 
do a little more work. First, in the test suite I create a subclass 
specifically for testing the common behaviour, write tests for that, then 
subclass from that:


class MyD(AbstractBaseClass):
    # Defeat the prohibition on instantiating the base class
    pass

class TestCommonBehaviour(unittest.TestCase):
    cls = MyD
    def testSpam(self):
         self.assertTrue(self.cls.spam == "spam")
    def testHam(self):
         self.assertFalse(hasattr(self.cls, 'ham'))

class TestD1CommonBehaviour(unittest.TestCase):
    cls = D1

class TestD2CommonBehaviour(unittest.TestCase):
    cls = D2

D1 and D2 specific tests remain the same.


Either way, each class gets tested for the full set of expected 
functionality.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#30678

On 3 October 2012 02:20, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>
> But surely, regardless of where that functionality is defined, you still
> need to test that both D1 and D2 exhibit the correct behaviour? Otherwise
> D2 (say) may break that functionality and your tests won't notice.
>
> Given a class hierarchy like this:
>
> class AbstractBaseClass:
>     spam = "spam"
>
> class D1(AbstractBaseClass): pass
> class D2(D1): pass
>
>
> I write tests like this:
>
> class TestD1CommonBehaviour(unittest.TestCase):
>     cls = D1
>     def testSpam(self):
>          self.assertTrue(self.cls.spam == "spam")
>     def testHam(self):
>          self.assertFalse(hasattr(self.cls, 'ham'))
>
> class TestD2CommonBehaviour(TestD1CommonBehaviour):
>     cls = D2

That's an excellent idea. I wanted a convenient way to run the same
tests on two classes in order to test both a pure python and a
cython-accelerator module implementation of the same class. I find it
difficult to work out how to do such simple things with unittest
because of its Java-like insistence on organising all tests into
classes. I can't immediately remember what solution I came up with but
yours is definitely better.


Oscar

[toc] | [prev] | [next] | [standalone]

#30772

On 10/3/2012 5:33 AM, Oscar Benjamin wrote:
> On 3 October 2012 02:20, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>>
>> But surely, regardless of where that functionality is defined, you still
>> need to test that both D1 and D2 exhibit the correct behaviour? Otherwise
>> D2 (say) may break that functionality and your tests won't notice.
>>
>> Given a class hierarchy like this:
>>
>> class AbstractBaseClass:
>>      spam = "spam"
>>
>> class D1(AbstractBaseClass): pass
>> class D2(D1): pass
>>
>>
>> I write tests like this:
>>
>> class TestD1CommonBehaviour(unittest.TestCase):
>>      cls = D1
>>      def testSpam(self):
>>           self.assertTrue(self.cls.spam == "spam")
>>      def testHam(self):
>>           self.assertFalse(hasattr(self.cls, 'ham'))
>>
>> class TestD2CommonBehaviour(TestD1CommonBehaviour):
>>      cls = D2
>
> That's an excellent idea. I wanted a convenient way to run the same
> tests on two classes in order to test both a pure python and a
> cython-accelerator module implementation of the same class.

Python itself has same issue with testing Python and C coded modules. It 
has the additional issue that the Python class by default import the C 
version, so additional work is needed to avoid that and actually test 
the python code.

For instance, heapq.test_heapq.py has

...
py_heapq = support.import_fresh_module('heapq', blocked=['_heapq'])
c_heapq = support.import_fresh_module('heapq', fresh=['_heapq'])
...
class TestHeap(TestCase):
     module = None
... <multiple test methods for functions module.xxx>

class TestHeapPython(TestHeap):
     module = py_heapq

@skipUnless(c_heapq, 'requires _heapq')
class TestHeapC(TestHeap):
     module = c_heapq
...
def test_main(verbose=None):
     test_classes = [TestModules, TestHeapPython, TestHeapC,

# TestHeap is omitted from the list and not run directly

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#30636

Ulrich Eckhardt wrote:

> As you see, the code for test_base() is redundant, so the idea is to
> move it to a baseclass:
> 
> class TestBase(unittest.TestCase):
>      def test_base(self):
>          ...
> 
> class TestD1(TestBase):
>      def test_r(self):
>          ...
>      def test_s(self):
>          ...
> 
> class TestD2(TestBase):
>      def test_x(self):
>          ...
>      def test_y(self):
>          ...
> 
> The problem here is that TestBase is not a complete test case (just as
> class Base is not complete), but the unittest framework will still try
> to run it on its own. One way around this is to not derive class
> TestBase from unittest.

Another is to remove it from the global namespace with 

del TestBase

[toc] | [prev] | [next] | [standalone]

#30637

Peter Otten wrote:

> Ulrich Eckhardt wrote:
>> The problem here is that TestBase is not a complete test case (just 
as
>> class Base is not complete), but the unittest framework will still 
try
>> to run it on its own.
How exactly are you invoking the test runner? unittest? nose? You can 
tell the test discoverer which classes you want it to run and which 
ones you don't. For the unittest library, I use my own custom 
load_tests methods:
def load_tests(loader, tests, pattern):
    testcases = [TestD1, TestD2]
    return TestSuite([loader.loadTestsFromTestCase(testcase)
                      for testcase in testcases])
http://docs.python.org/library/unittest.html#load-tests-protocol

>> One way around this is to not derive class
>> TestBase from unittest.
> 
> Another is to remove it from the global namespace with
> 
> del TestBase
Removing the class from namespace may or may not help. Consider a 
scenario where someone decided to be creative with the cls.__bases__ 
attribute.

-- 
Fayaz Yusuf Khan
Cloud architect, Dexetra SS, India
fayaz.yusuf.khan_AT_gmail_DOT_com, fayaz_AT_dexetra_DOT_com
+91-9746-830-823

[toc] | [prev] | [next] | [standalone]

#30648

Fayaz Yusuf Khan wrote:

> Peter Otten wrote:
> 
>> Ulrich Eckhardt wrote:
>>> The problem here is that TestBase is not a complete test case (just
> as
>>> class Base is not complete), but the unittest framework will still
> try
>>> to run it on its own.
> How exactly are you invoking the test runner? unittest? nose? You can
> tell the test discoverer which classes you want it to run and which
> ones you don't. For the unittest library, I use my own custom
> load_tests methods:
> def load_tests(loader, tests, pattern):
>     testcases = [TestD1, TestD2]
>     return TestSuite([loader.loadTestsFromTestCase(testcase)
>                       for testcase in testcases])
> http://docs.python.org/library/unittest.html#load-tests-protocol
> 
>>> One way around this is to not derive class
>>> TestBase from unittest.
>> 
>> Another is to remove it from the global namespace with
>> 
>> del TestBase
> Removing the class from namespace may or may not help. Consider a
> scenario where someone decided to be creative with the cls.__bases__
> attribute.

Isn't that a bit far-fetched? I'd rather start simple and fix problems as 
they arise... 

[toc] | [prev] | [next] | [standalone]

#30663

In article <mailman.1734.1349199947.27098.python-list@python.org>,
 Peter Otten <__peter__@web.de> wrote:

> >> Another is to remove it from the global namespace with
> >> 
> >> del TestBase

When I had this problem, that's the solution I used.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web