Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110221 > unrolled thread

Marking a subtest as an expected failure

Back to article view | Back to comp.lang.python

Contents

  Marking a subtest as an expected failure Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-06-21 18:28 +1000
    Re: Marking a subtest as an expected failure Ben Finney <ben+python@benfinney.id.au> - 2016-06-22 05:03 +1000

#110221 — Marking a subtest as an expected failure

I'm using unittest and the subtest context manager to run some tests:


def test_spam(self):
    for i in range(100):
        for j in range(100):
            with self.subtest(i=i, j=j):
                self.assertEqual(spam(i, j), 999)


Now, it turns out that spam(i, j) == 999 for all i, j *except* one or two
exceptional cases. E.g. let's say spam(97, 83) != 999. And further, this
is not a bug to be fixed, but an expected failure. I can write:

def test_spam(self):
    for i in range(100):
        for j in range(100):
            with self.subtest(i=i, j=j):
                if (i, j) != (97, 83):
                    self.assertEqual(spam(i, j), 999)


but is there a nicer way to mark a specific subtest as an expected failure?
I don't want the entire test_spam test to be counted as an expected failure, 
just that one subtest.

https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests





-- 
Steve

[toc] | [next] | [standalone]

#110263

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> def test_spam(self):
>     for i in range(100):
>         for j in range(100):
>             with self.subtest(i=i, j=j):
>                 if (i, j) != (97, 83):
>                     self.assertEqual(spam(i, j), 999)
>
>
> but is there a nicer way to mark a specific subtest as an expected
> failure?

Expected failures and subtests are apparently not easily mixed using the
current API.

Digging into the ‘unittest.case’ module, the ‘expectedFailure’ decorator
does its job by setting the test case's ‘__unittest_expecting_failure__’
attribute to ‘True’.

Perhaps you can get the subtest object from the context manager, and
decide to set that magic attribute::

    # …
        with self.subtest(i=i, j=j) as subtest:
            if (i, j) == (97, 83):
                # The inhabitants of (97, 83) have always been trouble.
                subtest.__unittest_expecting_failure__ = True
            self.assertEqual(spam(i, j), 999)

That attribute is undocumented though, so I don't know whether to
consider that a nasty hack or a clever one. (Nor do I know whether it
works as I expect.)

You would be justified to report a bug that this interaction of useful
features should really be easier to access.

-- 
 \     “Try adding “as long as you don't breach the terms of service – |
  `\          according to our sole judgement” to the end of any cloud |
_o__)                      computing pitch.” —Simon Phipps, 2010-12-11 |
Ben Finney

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web