Info Catalog ert: The should Macro ert: How to Write Tests ert: Tests and Their Environment

ert: Expected Failures

 
 3.2 Expected Failures
 =====================
 
 Some bugs are complicated to fix, or not very important, and are left as
 _known bugs_.  If there is a test case that triggers the bug and fails,
 ERT will alert you of this failure every time you run all tests.  For
 known bugs, this alert is a distraction.  The way to suppress it is to
 add ‘:expected-result :failed’ to the test definition:
 
      (ert-deftest future-bug ()
        "Test `time-forward' with negative arguments.
      Since this functionality isn't implemented, the test is known to fail."
        :expected-result :failed
        (time-forward -1))
 
    ERT will still display a small ‘f’ in the progress bar as a reminder
 that there is a known bug, and will count the test as failed, but it
 will be quiet about it otherwise.
 
    An alternative to marking the test as a known failure this way is to
 delete the test.  This is a good idea if there is no intent to fix it,
 i.e., if the behavior that was formerly considered a bug has become an
 accepted feature.
 
    In general, however, it can be useful to keep tests that are known to
 fail.  If someone wants to fix the bug, they will have a very good
 starting point: an automated test case that reproduces the bug.  This
 makes it much easier to fix the bug, demonstrate that it is fixed, and
 prevent future regressions.
 
    ERT displays the same kind of alerts for tests that pass unexpectedly
 as it displays for unexpected failures.  This way, if you make code
 changes that happen to fix a bug that you weren’t aware of, you will
 know to remove the ‘:expected-result’ clause of that test and close the
 corresponding bug report, if any.
 
    Since ‘:expected-result’ evaluates its argument when the test is
 loaded, tests can be marked as known failures only on certain Emacs
 versions, specific architectures, etc.:
 
      (ert-deftest foo ()
        "A test that is expected to fail on Emacs 23 but succeed elsewhere."
        :expected-result (if (string-match "GNU Emacs 23[.]" (emacs-version))
                             :failed
                           :passed)
        ...)
 

Info Catalog ert: The should Macro ert: How to Write Tests ert: Tests and Their Environment