The last month, I've come across this issue several time, to my surprise (because it hasn't come up in the years before).
If you are wrapping tests around legacy code and the legacy code has a C-style assert in the production code, then what do you do? Do you write a test for it? If so, how? Do you delete it? Why?
C-Style assert
Lets first have a look what C-style assertions are and why they are there. Assert is used for expressing a state that is always true. An assert cannot fail, if it fails then there is a bug in the code. So, C-style assert should and cannot be used for error handling. If the error is possible, then proper return value or exception error handling should be used, but if the error is impossible then this can be ensured and documented by using a C-style assert.
An extremely silly example is:
int a = 5;
assert (a > 0);
If this would fail, it would be a bug in the compiler.
C-Style asserts often go together with design by contract. In design by contract, we design the pre-condition, post-condition and class invariant to be per definition true. If it isn't true, then there is a bug in the caller in the case of breaking a pre-condition, a bug in the supplier in the case of breaking a post-condition or in the class in the case of breaking a class invariant. C-Style assert can be used to document this by, in the case of pre-condition, putting asserts at the beginning of a function. For example:
void doBlah(int x)
{
assert (x != 0);
...
So, this code reads that the doBlah can never be called with 0, if it does, then there is a bug in the function that calls doBlah.
Assert and unit tests
Oki, so now we know why the asserts are there, but how do you deal with them in unit tests?
Well.... you don't.
If we take the doBlah function above, we don't need to write a unit test for the doBlah to assert when x == 0. This is an impossible situation and it is not needed to test that. As said, assert is not error handling, it is error in programming :) You design (the contract) states that this is impossible and thus there is no need to test it.
Though, it ain't that easy. You do need to make sure the assert is really an assert. I often read code where the developer uses assert as a method for error handling (and he will regret that when the assert goes off in a production environment, though usually assert is commented out for production code). If assert is not assert, then you can delete it and add proper error handling.
Assert and test-driven code
Many years ago, I used to write a lot of asserts in my code. It was known as good style to make your contracts explicit. Today, I nearly never write an assert and tend to delete them from production code when I see them. Why?
First, I can't add the assert when I test-drive as it still is a line of code for which I don't have a test :) But that wouldn't be fair, as I *could* in theory test it (stub out the assert in the C-library). To better understand why I don't write them, we'll need to look at the purpose. An assert makes sure something can't happen and documents that. When test-driving code, we document how to use a piece of code in the tests. So, therefore the documentation aspect of tests causes the assert to be less useful. Also, the impossible situation often won't happen because I tested it, so I don't need to put the assert there.
So, what to do with C-Style asserts in existing code? I usually do either of these two things:
- Leave them and use the information as documentation
- Delete them, they clutter the code
I use asserts just like in Java:
- check preconditions in private (static) functions with asserts and
- check preconditions in public (interface) functions with an exception/error-code.
You can't test static/private functions directly and therefore it make sense to check and document this pre-conditions with an assert.
What do you think?
Hi Simon,
Thanks for your comments. Good to know people still do use asserts in production code (not me!).
Related to private/static methods. You can test them, but they suggest deeper problems with the design. Lasse wrote an interesting blog post on it at: http://lassekoskela.com/thoughts/24/test-everything-but-not-private-methods/
Bas
Hi. I just stumbled on this page via a Google search related to assertions. "I ... tend to delete [assertions] from production code when I see them." That sounds like very strange behavior to me. Do you also delete regular comments that serve to document what the code does?
Hi Weston,
Oh yes. Of course I delete comments and try to fix the code so that the code itself explains what it does rather than adding a comment to it.
Most comments just clutter the code and its better to write proper code.
Bas
So, as a concrete example:
double quadraticRealRoot(double a, double b, double c) { assert(a != 0.0); double discriminant = quadraticDiscriminant(a, b, c); assert(0 <= discriminant || !"precondition 0<=quadraticDiscriminant(a, b, c) violated"); double direction = b < 0 ? 1.0 : -1.0; // The following is just the usual quadratic formula, where we // have selected "direction" such that there will not be a // cancellation between -b and sqrt(discriminant): return (-b + direction * sqrt(discriminant)) / a / 2; }What would you consider to be "proper code" that explains what the function does, without using comments or assertions? Imagine that we later port this code to some platform where, due to differences in floating-point rounding modes (or some such thing) the calling code is no longer able to guarantee 0<=quadraticDiscriminant(a,b,c). Without the assertion, won't it take us much longer to figure out what's going on when we get incorrect answers?
Hi Mark,
Well, one of the points I was making is that a lot of the documentation of how it would work, is in the tests. So, if you give a piece of code, but do not deliver the unit tests with it, and ask how to write the code without the tests, then that is a little bit missing the point.
Anyways, related to the not being able to guarantee that the 0
Related to the comments. I'm not sure the comments you added help at all :) At least, I was more confused about the comments than the code :P
Hello
Assert documents the interface, and acts as a sanity check. I don't understand how unit tests replace these useful functions of assert.
Taking your "doBlah" example, how would a unit test look that documents the fact that passing x=0 results in undefined behavior? Also, I don't think users of a module will look at the accompanying tests as carefully as they will look at the source code, so even if there is a test that documents that the function's behavior is undefined if x=0, users may never notice it.
Secondly, an assert can serve to restore a (embedded) system to sanity by causing a reset. For example, if memory is corrupted by cosmic rays (or, more prosaically, by a buggy interrupt handler), an assert could return the system to sanity, at least temporarily. If the corruption caused a variable to assume an "impossible" value, the assert would catch this. This is also an argument against disabling asserts in production code, at least for systems that can be restored to full functionality by a reset.
Unit tests find bugs in the module under test, and assert finds them in the client of the module -- we need both.
Hi Ferdi,
Thanks for your comment. A couple of comments back. Unit tests definitively act as documentation for how you want to use a module or a piece of code. Its documentation value is very important. I regularly check out the unit tests *first* to understand how I should use something.
Unit tests show how you legally would use a module whereas assertions document the illegal cases if you wish. They are just the inverse of each other.
Related to using asserts for putting an embedded system back. I haven't ever seen them used that way, I'm afraid. Usually asserts just crashes the processes.. which in an embedded system might cause a restart, but there is rarely any "real" recovery happening. It could, just haven't seen it.
Unit tests (especially with TDD) are actually not so much for finding bugs...