Quixo3D - Part 2: Code Coverage
Last time we met, I was talking about Code Analysis for the Quixo3D framework and engine assemblies. Today I'm going to talk about Code Coverage.
As I type, my current coverage numbers are:
- Quixo3D.Framework - 94.91%
- Quixo3D.Engine - 95.83%
I'm really trying to get as close as I can to 100% as I possibly can, just to see what I can learn from the experience. So far, I've learned a couple of things.
First, reviewing the results has helped tremendously in eliminating "unused" code. There were sections in my code base that I wasn't touching in my tests, and I quickly realized I would never touch it. Basically, it was unnecessary code (e.g. private no-argument constructors that I foolishly added), so I yanked that stuff out. Second, it helped me write useful utilities for testing. I have a couple of custom exception classes, and originally I had no tests around them. To ensure that they would work under serialization/AppDomain boundaries, I wrote a base class that would test any exception and ensure it followed the basic exception design principles: (i.e. it's serializable, it has a number of "standard" constructors, etc.). Next, it's showing me where I'm still missing coverage. I have a custom formatter to serialize and deserialize a board, but I don't have coverage over some of the property getters and setters, so I have to fix that. Finally, I have a class called
ValidMove that handles
IEquatable<T>, the "==" and "!=" operators, etc. but because its constructor is
internal, I can't create the scenario where I'll get two
ValidMoves back from the board. In fact I should never get that back. So I think I can write some tests that execute all of the "equals" and "not equals" cases, but I haven't done that just yet.
All that said, there's some areas of concern. One is enumerators. C# generates some classes when you do the
GetEnumerator() duck typing pattern, and even though all of the code shows up being covered, the reports shows "you're close but not 100%". I'm not really sure if this is a bug or a feature. Another issue is when I create an internal class that implements
IEnumerable<T>. I have to implement the non-generic
GetEnumerator() method, but I'll never call it in my code. So...it's going to sit unused. I could go the
PrivateObject approach or other Reflection-based techniques to "cover" that code, but I think that's useless. I try to keep it black-box in my unit tests; it would feel downright silly to write a test just to have this method called.
My general conclusion is that you should strive for at least 90% coverage in your code. After that, the ROI of achieving a higher number has to be kept in mind. You may be able to get around 95% with little effort, but I think that there may be cases where 100% just won't be achievable, or, to hit that number requires techniques that I think are suspect for unit testing. I think that's OK. The more code coverage you have, the better off you'll be in the long run; just be realistic with what you can (and should) do in your unit tests.
Next up...desiging a smart engine...
* Posted at 11.09.2007 09:27:01 AM CST | Link *