Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #106999 > unrolled thread

How to parameterize unittests

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-14 16:08 +0200
    Re: How to parameterize unittests Steven D'Aprano <steve@pearwood.info> - 2016-04-15 01:05 +1000
      Re: How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-15 08:52 +0200
      Re: How to parameterize unittests Chris Angelico <rosuav@gmail.com> - 2016-04-15 17:42 +1000
      Re: How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-15 10:20 +0200
        Re: How to parameterize unittests Steven D'Aprano <steve@pearwood.info> - 2016-04-15 19:10 +1000
          Re: How to parameterize unittests Michael Selik <michael.selik@gmail.com> - 2016-04-15 10:05 +0000
          Re: How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-15 13:43 +0200
          Re: How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-15 14:48 +0200
            Re: How to parameterize unittests Steven D'Aprano <steve@pearwood.info> - 2016-04-16 02:47 +1000
              Re: How to parameterize unittests Chris Angelico <rosuav@gmail.com> - 2016-04-16 03:51 +1000
              Re: How to parameterize unittests Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-04-16 22:37 +0200
      Re: How to parameterize unittests Serhiy Storchaka <storchaka@gmail.com> - 2016-04-15 11:24 +0300
      Re: How to parameterize unittests Chris Angelico <rosuav@gmail.com> - 2016-04-15 18:28 +1000
      Re: How to parameterize unittests Serhiy Storchaka <storchaka@gmail.com> - 2016-04-15 11:31 +0300
      Re: How to parameterize unittests Ben Finney <ben+python@benfinney.id.au> - 2016-04-15 18:35 +1000

#106999 — How to parameterize unittests

I have a unittest for my avltree module.

Now I want this unittest to also run on a subclass of avltree.
How can I organise this, so that I can largely reuse the
original TestCase?

-- 
Antoon Pardon

[toc] | [next] | [standalone]

#107001

On Fri, 15 Apr 2016 12:08 am, Antoon Pardon wrote:

> 
> I have a unittest for my avltree module.
> 
> Now I want this unittest to also run on a subclass of avltree.
> How can I organise this, so that I can largely reuse the
> original TestCase?


class Test_AVLTree(unittest.TestCase):
    tree = avltree

    def test_empty_tree_is_false(self):
        instance = self.tree()
        self.assertFalse(instance)


class Test_MySubclassTree(Test_AVLTree):
    tree = My_Subclass_Tree



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#107026

Op 14-04-16 om 17:05 schreef Steven D'Aprano:
> On Fri, 15 Apr 2016 12:08 am, Antoon Pardon wrote:
>
>> I have a unittest for my avltree module.
>>
>> Now I want this unittest to also run on a subclass of avltree.
>> How can I organise this, so that I can largely reuse the
>> original TestCase?
>
> class Test_AVLTree(unittest.TestCase):
>     tree = avltree
>
>     def test_empty_tree_is_false(self):
>         instance = self.tree()
>         self.assertFalse(instance)
>
>
> class Test_MySubclassTree(Test_AVLTree):
>     tree = My_Subclass_Tree

I see, that's going to be a lot of cut & pastes.
Thanks.

-- 
Antoon.

[toc] | [prev] | [next] | [standalone]

#107028

On Fri, Apr 15, 2016 at 4:52 PM, Antoon Pardon
<antoon.pardon@rece.vub.ac.be> wrote:
> Op 14-04-16 om 17:05 schreef Steven D'Aprano:
>> On Fri, 15 Apr 2016 12:08 am, Antoon Pardon wrote:
>>
>>> I have a unittest for my avltree module.
>>>
>>> Now I want this unittest to also run on a subclass of avltree.
>>> How can I organise this, so that I can largely reuse the
>>> original TestCase?
>>
>> class Test_AVLTree(unittest.TestCase):
>>     tree = avltree
>>
>>     def test_empty_tree_is_false(self):
>>         instance = self.tree()
>>         self.assertFalse(instance)
>>
>>
>> class Test_MySubclassTree(Test_AVLTree):
>>     tree = My_Subclass_Tree
>
> I see, that's going to be a lot of cut & pastes.
> Thanks.

Not really; the first class has all the tests, and the second one is
literally just those two lines. It overrides 'tree' (accessed inside
methods as 'self.tree'), and since all the tests are written to
instantiate self.tree, they are effectively parameterized.

ChrisA

[toc] | [prev] | [next] | [standalone]

#107029

Op 15-04-16 om 09:42 schreef Chris Angelico:
> On Fri, Apr 15, 2016 at 4:52 PM, Antoon Pardon
> <antoon.pardon@rece.vub.ac.be> wrote:
>> Op 14-04-16 om 17:05 schreef Steven D'Aprano:
>>> On Fri, 15 Apr 2016 12:08 am, Antoon Pardon wrote:
>>>
>>>> I have a unittest for my avltree module.
>>>>
>>>> Now I want this unittest to also run on a subclass of avltree.
>>>> How can I organise this, so that I can largely reuse the
>>>> original TestCase?
>>> class Test_AVLTree(unittest.TestCase):
>>>     tree = avltree
>>>
>>>     def test_empty_tree_is_false(self):
>>>         instance = self.tree()
>>>         self.assertFalse(instance)
>>>
>>>
>>> class Test_MySubclassTree(Test_AVLTree):
>>>     tree = My_Subclass_Tree
>> I see, that's going to be a lot of cut & pastes.
>> Thanks.
> Not really; the first class has all the tests, and the second one is
> literally just those two lines. It overrides 'tree' (accessed inside
> methods as 'self.tree'), and since all the tests are written to
> instantiate self.tree, they are effectively parameterized.

But the tests, at this moment, are not written to instantiate self.tree
but to call avltree directly. So I have to rewrite these tests. That
will IMO involve a lot of cut and paste.

-- 
Antoon.

[toc] | [prev] | [next] | [standalone]

#107035

On Fri, 15 Apr 2016 06:20 pm, Antoon Pardon wrote:

>>>> class Test_MySubclassTree(Test_AVLTree):
>>>> tree = My_Subclass_Tree
>>> I see, that's going to be a lot of cut & pastes.
>>> Thanks.
>> Not really; the first class has all the tests, and the second one is
>> literally just those two lines. It overrides 'tree' (accessed inside
>> methods as 'self.tree'), and since all the tests are written to
>> instantiate self.tree, they are effectively parameterized.
> 
> But the tests, at this moment, are not written to instantiate self.tree
> but to call avltree directly. So I have to rewrite these tests. That
> will IMO involve a lot of cut and paste.


*shrug*

I feel your pain, because I've had to go through exactly the same process.

If you have code which is not parameterized, and you want to parameterize
it, you have to refactor. Unit tests are no different from anything else.

I suggest that you handle it this way:

(1) Start with all your tests passing. If they're not passing, that will
make the process a lot harder. If there are any failing tests, temporarily
re-name them so that they don't run (test_foo to FIXME_test_foo, say), or
use the @unittest.skip decorator.

(2) Go through each TestCase class, and add a simple "tree = avltree" class
attribute to the class. Confirm that the tests still pass.

(3) In your editor, run a global Find and Replace "avltree -> self.tree".
You will need to inspect each one rather than do it automatically.
Obviously you need to avoid changing the "tree = avltree" class attribute,
and any import lines, but probably everything else should change.

(4) Run the tests again. Hopefully everything will work perfectly, and all
the tests will pass, but if not, fix any that have been broken by the
refactoring.

(5) Once all the tests pass again, the refactoring is complete, and you can
start subclassing your test classes with the new parameterized tree.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#107038

On Fri, Apr 15, 2016, 11:16 AM Steven D'Aprano <steve@pearwood.info> wrote:

> On Fri, 15 Apr 2016 06:20 pm, Antoon Pardon wrote:
>
> >>> I see, that's going to be a lot of cut & pastes.
>
> (3) In your editor, run a global Find and Replace "avltree -> self.tree".
> You will need to inspect each one rather than do it automatically.
> Obviously you need to avoid changing the "tree = avltree" class attribute,
> and any import lines, but probably everything else should change.
>

There's probably some context around uses of avltree that should be
replaced. Perhaps a trailing dot: "avltree." replace with "self.tree." so
imports and assignments are ignored.

Also, your other, very helpful steps should minimize the trial and error
aspects of global find-replace. I wouldn't feel obligated to visually
inspect each one -- not on my first try, anyway.

>

[toc] | [prev] | [next] | [standalone]

#107044

Op 15-04-16 om 11:10 schreef Steven D'Aprano:
> If you have code which is not parameterized, and you want to parameterize
> it, you have to refactor. Unit tests are no different from anything else.

I don't agree with that. If I have a piece of code that I want to parameterize,
Often enough all I need to do is shift the code to the right. Prepend a def
line and use a parameter with the same name as the more global variable I
was using before. I don't need to change access to the variable in the code.

If python would treat class variable as a scope between methods and the
rest, something equally simple might have worked here. But that is another
discussions.

[Sensible suggestions removed.]

-- 
Antoon Pardon

[toc] | [prev] | [next] | [standalone]

#107054

Op 15-04-16 om 13:43 schreef Antoon Pardon:
> Op 15-04-16 om 11:10 schreef Steven D'Aprano:
>> If you have code which is not parameterized, and you want to parameterize
>> it, you have to refactor. Unit tests are no different from anything else.
> I don't agree with that. If I have a piece of code that I want to parameterize,
> Often enough all I need to do is shift the code to the right. Prepend a def
> line and use a parameter with the same name as the more global variable I
> was using before. I don't need to change access to the variable in the code.

Some prelimary tests seems to suggest this idea might work here too.

Starting from this:

    class Test_AVLTree(unittest.TestCase):
 
        def test_empty_tree_is_false(self):
            instance = avltree()
            self.assertFalse(instance)

Changing it into this:

    def MakeAVLTest(avltree):
        class Test_AVLTree(unittest.TestCase):

            def test_empty_tree_is_false(self):
                instance = avltree()
                self.assertFalse(instance)

        return Test_AVLTree

    AVLTest = MakeAVLTest(avltree)
    MyTreeTest = MakeAVLTest(mytree)

Seems to work

[toc] | [prev] | [next] | [standalone]

#107064

On Fri, 15 Apr 2016 10:48 pm, Antoon Pardon wrote:

> Starting from this:
> 
>     class Test_AVLTree(unittest.TestCase):
>  
>         def test_empty_tree_is_false(self):
>             instance = avltree()
>             self.assertFalse(instance)
> 
> Changing it into this:
> 
>     def MakeAVLTest(avltree):
>         class Test_AVLTree(unittest.TestCase):
> 
>             def test_empty_tree_is_false(self):
>                 instance = avltree()
>                 self.assertFalse(instance)
> 
>         return Test_AVLTree
> 
>     AVLTest = MakeAVLTest(avltree)
>     MyTreeTest = MakeAVLTest(mytree)
> 
> Seems to work

Right up to the moment that you realise that you need different tests for a
subclass, and now you can't using subclassing because they aren't
subclasses, they're completely independent classes that happen to duplicate
the same methods.

If the tests for your AVL tree and it's subclasses are *identical*, then
what's the point of the subclasses?




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#107067

On Sat, Apr 16, 2016 at 2:47 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> If the tests for your AVL tree and it's subclasses are *identical*, then
> what's the point of the subclasses?

If the tree classes all have the same API but different performance
trade-offs, it would make sense to use all the same tests to make sure
their behaviours are correct. But if your module's purpose is to offer
a variety of tree classes, its tests should be designed around
parameterization. Refactoring the code would be the correct behaviour.

ChrisA

[toc] | [prev] | [next] | [standalone]

#107114

Op 15-04-16 om 18:47 schreef Steven D'Aprano:
> On Fri, 15 Apr 2016 10:48 pm, Antoon Pardon wrote:
> 
>> Starting from this:
>>
>>     class Test_AVLTree(unittest.TestCase):
>>  
>>         def test_empty_tree_is_false(self):
>>             instance = avltree()
>>             self.assertFalse(instance)
>>
>> Changing it into this:
>>
>>     def MakeAVLTest(avltree):
>>         class Test_AVLTree(unittest.TestCase):
>>
>>             def test_empty_tree_is_false(self):
>>                 instance = avltree()
>>                 self.assertFalse(instance)
>>
>>         return Test_AVLTree
>>
>>     AVLTest = MakeAVLTest(avltree)
>>     MyTreeTest = MakeAVLTest(mytree)
>>
>> Seems to work
> 
> Right up to the moment that you realise that you need different tests for a
> subclass, and now you can't using subclassing because they aren't
> subclasses, they're completely independent classes that happen to duplicate
> the same methods.
> 
> If the tests for your AVL tree and it's subclasses are *identical*, then
> what's the point of the subclasses?

The subclass has a key-function as a static method similar to the key argument
for sort. This way I can have a tree with strings as keys, but with the
keys sorted according to the locale with the locale.strxfrm function.

This means the only tests that need changing are the tests that test for
iterators to deliver the items in sorted order. But that can be done
by letting the test check for the presence of the keyfunction and in
that case, use if to check if things are in order.

-- 
Antoon

[toc] | [prev] | [next] | [standalone]

#107030

On 14.04.16 18:05, Steven D'Aprano wrote:
> On Fri, 15 Apr 2016 12:08 am, Antoon Pardon wrote:
>> I have a unittest for my avltree module.
>>
>> Now I want this unittest to also run on a subclass of avltree.
>> How can I organise this, so that I can largely reuse the
>> original TestCase?
>
>
> class Test_AVLTree(unittest.TestCase):
>      tree = avltree
>
>      def test_empty_tree_is_false(self):
>          instance = self.tree()
>          self.assertFalse(instance)
>
>
> class Test_MySubclassTree(Test_AVLTree):
>      tree = My_Subclass_Tree

Yes, this is common approach.

If there tests specific for original class or tests that there is no 
need to run for subclasses, you should define common tests in mixin 
class that is not test class itself:


class AbstractTest_AVLTree: # note, there is no TestCase!

     def test_common(self):
         ...


class Test_AVLTree(AbstractTest_AVLTree, unittest.TestCase):
      tree = avltree

      def test_base_class_specific(self):
          ...


class Test_MySubclassTree(AbstractTest_AVLTree, unittest.TestCase):
     tree = My_Subclass_Tree

     def test_sub_class_specific(self):
         ...

Complex tests can have a DAG of test classes.

[toc] | [prev] | [next] | [standalone]

#107031

On Fri, Apr 15, 2016 at 6:20 PM, Antoon Pardon
<antoon.pardon@rece.vub.ac.be> wrote:
> But the tests, at this moment, are not written to instantiate self.tree
> but to call avltree directly. So I have to rewrite these tests. That
> will IMO involve a lot of cut and paste.

Ah. In that case, it either involves a lot of editing (to make them
call self.tree), or monkey-patching the name 'avltree'.

ChrisA

[toc] | [prev] | [next] | [standalone]

#107032

On 15.04.16 11:20, Antoon Pardon wrote:
> But the tests, at this moment, are not written to instantiate self.tree
> but to call avltree directly. So I have to rewrite these tests. That
> will IMO involve a lot of cut and paste.

There is yet one approach. Import your original test file, patch it by 
setting it's global avltree to your subclass, run tests. Don't forget to 
restore original values. This approach is more fragile and less 
flexible, but doesn't need modifying the original test.

[toc] | [prev] | [next] | [standalone]

#107033

Antoon Pardon <antoon.pardon@rece.vub.ac.be> writes:

> But the tests, at this moment, are not written to instantiate
> self.tree but to call avltree directly.

That is exactly what the ‘TestCase.setUp’ method is for: to have the
test case class specify how its test cases will customise themselves.

    class AVLTree_TestCase(unittest.TestCase):
        tree = AVLTree

        def setUp(self):
            super().setUp()
            self.test_instance = self.tree()

        def test_empty_tree_is_false(self):
            self.assertFalse(self.test_instance)


    class LoremIpsumAVLTree_TestCase(AVLTree_TestCase):
        tree = LoremIpsumAVLTree


    class DolorSitAmetAVLTree_TestCase(AVLTree_TestCase):
        tree = DolorSitAmetAVLTree


By not overriding ‘setUp’, the same routine will be called in the
subclass's instance also.

> So I have to rewrite these tests.

Yes, you'll need to consider inheritance and how the ‘unittest’ API
supports it.

-- 
 \       “Faith, n. Belief without evidence in what is told by one who |
  `\   speaks without knowledge, of things without parallel.” —Ambrose |
_o__)                           Bierce, _The Devil's Dictionary_, 1906 |
Ben Finney

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web