Tuesday, November 3, 2009

JUnit, tearDown, and uncaught exceptions

I know this, I've known it for a while, and still I stumble over it.

I tend to generally declare my JUnit test cases as


public void testXXX()
throws Exception
{
}


Within my test case code, I then typically don't catch any exceptions, unless I'm specifically testing that a particular expected exception is thrown. Instead, I simply allow any unexpected exceptions to be thrown out of the test case, and caught by the JUnit infrastructure, which decides that a test case which terminated with an uncaught exception is an error, and JUnit reports the uncaught exception in the output.

However, there is a gotcha, and it involves the tearDown method. If a particular test suite uses the setUp/tearDown paradigm to perform common initialization and termination functions, then it is crucial that these methods themselves must be cautious with respect to exceptions.

Because if:

  • your test case terminates with an uncaught exception,
  • and then your tearDown method terminates with an uncaught exception,

then it is the tearDown exception which "wins"; i.e., it is the exception which is shown in the JUnit output.

This can completely fool you into a wild goose chase of looking at the wrong problem.

In a way, I think this is a mistake in JUnit; I wish it reported the uncaught exception from the test case in preference to the uncaught exception from tearDown, or even better I wish it reported both exceptions.

But, for now, the best thing to do is to ensure that your tearDown methods are extremely careful, and never terminate with uncaught exceptions.

2 comments:

  1. To be fair to junit, I think java finally {} does something similar, though I'm not sure.

    In my ant functional test task I went to a lot of effort to try and not lose exceptions -look at the code in execute() in the task:
    https://smartfrog.svn.sourceforge.net/svnroot/smartfrog/trunk/core/extras/ant/src/org/smartfrog/tools/ant/FunctionalTestTask.java

    but even then I have to make decisions. Does a test failure come ahead or behind a failure in the application? Especially when the app failing could cause the test to fail.

    things would be easiest if test runners assumed tests could return multiple exceptions qualified by strings (setup, test, teardown, etc)

    ReplyDelete
  2. I see this as shortcoming of Junit as it is used in Ant or on the commandline.

    Eclipse for instance does it better, it displays both exceptions.

    ReplyDelete