Info Catalog ert: Expected Failures ert: How to Write Tests ert: Useful Techniques

ert: Tests and Their Environment

 
 3.3 Tests and Their Environment
 ===============================
 
 Sometimes, it doesn’t make sense to run a test due to missing
 preconditions.  A required Emacs feature might not be compiled in, the
 function to be tested could call an external binary which might not be
 available on the test machine, you name it.  In this case, the macro
 ‘skip-unless’ could be used to skip the test:
 
      (ert-deftest test-dbus ()
        "A test that checks D-BUS functionality."
        (skip-unless (featurep 'dbusbind))
        ...)
 
    The outcome of running a test should not depend on the current state
 of the environment, and each test should leave its environment in the
 same state it found it in.  In particular, a test should not depend on
 any Emacs customization variables or hooks, and if it has to make any
 changes to Emacs’s state or state external to Emacs (such as the file
 system), it should undo these changes before it returns, regardless of
 whether it passed or failed.
 
    Tests should not depend on the environment because any such
 dependencies can make the test brittle or lead to failures that occur
 only under certain circumstances and are hard to reproduce.  Of course,
 the code under test may have settings that affect its behavior.  In that
 case, it is best to make the test ‘let’-bind all such setting variables
 to set up a specific configuration for the duration of the test.  The
 test can also set up a number of different configurations and run the
 code under test with each.
 
    Tests that have side effects on their environment should restore it
 to its original state because any side effects that persist after the
 test can disrupt the workflow of the programmer running the tests.  If
 the code under test has side effects on Emacs’s current state, such as
 on the current buffer or window configuration, the test should create a
 temporary buffer for the code to manipulate (using ‘with-temp-buffer’),
 or save and restore the window configuration (using
 ‘save-window-excursion’), respectively.  For aspects of the state that
 can not be preserved with such macros, cleanup should be performed with
 ‘unwind-protect’, to ensure that the cleanup occurs even if the test
 fails.
 
    An exception to this are messages that the code under test prints
 with ‘message’ and similar logging; tests should not bother restoring
 the ‘*Message*’ buffer to its original state.
 
    The above guidelines imply that tests should avoid calling highly
 customizable commands such as ‘find-file’, except, of course, if such
 commands are what they want to test.  The exact behavior of ‘find-file’
 depends on many settings such as ‘find-file-wildcards’,
 ‘enable-local-variables’, and ‘auto-mode-alist’.  It is difficult to
 write a meaningful test if its behavior can be affected by so many
 external factors.  Also, ‘find-file’ has side effects that are hard to
 predict and thus hard to undo: It may create a new buffer or reuse an
 existing buffer if one is already visiting the requested file; and it
 runs ‘find-file-hook’, which can have arbitrary side effects.
 
    Instead, it is better to use lower-level mechanisms with simple and
 predictable semantics like ‘with-temp-buffer’, ‘insert’ or
 ‘insert-file-contents-literally’, and to activate any desired mode by
 calling the corresponding function directly, after binding the hook
 variables to ‘nil’.  This avoids the above problems.
 

Info Catalog ert: Expected Failures ert: How to Write Tests ert: Useful Techniques