Unit tests work best when you have close to 100 percent coverage of the methods in your project. The problem, of course, is that it is not always easy to know if you have tested every method in your application. Problems also occur when tests do not thoroughly cover all the paths through a particular method. For instance, there may be logical branches in a method that are never explored. The solution to problems of this type is a tool called NCover, which is based on the JCover tool and similar utilities.
NCover is a free tool that creates reports based on the runs of your unit tests. The report the tool creates shows every method that your tests called, every method that was not called, and every method that was partially called. For instance, you can see that 100 percent of the routines in the class called FtpData were called, while only 35 percent of the code in the class called Files were called. You can also see that the method called FileSize reports having 80 percent of its code called. In particular, you can see that the test checks what happens if the method is called with a valid file name, but it does not confirm what happens if the method is called with an invalid file name.
The NCoverBrowser is a free utility which you can use to explore the XML files returned when you run NCover. Here it reports the run of a unit test that tests two classes called Files and FtpData.
This article discusses NCover and the theory behind its use. My goal is simply to let you know that such tools exist, and to give you a quick tutorial on the simple task of using NCover.
Overview of Code Coverage Theory
Unit testing gives programmers a sense of security. Test infected programmers are addicted to this sense of security, and hence enjoy writing tests. The more cautious, and the more driven, you are, then the more inclined you will be to adopt unit testing as a way of life.
Unit testing also gives you the confidence you need to refactor your code. We can all think of ways to improve our code, but frequently we don’t dare make those changes because we can’t guess all the consequences involved. If, however, we know that 100 percent of our code is covered with unit tests, then we can dare change almost any part of our code because our unit tests should discover all the possible errors that might arise from the refactoring.
The fear, of course, is that somewhere there is a method that we have not thoroughly tested, or perhaps failed to test at all. The idea behind a code coverage tool like NCover is to reveal any uncalled methods or code paths in your test suites.
One argument against NCover is that it can be used as a blunt tool to bludgeon reluctant programmers into fully unit testing their code. Almost anyone, even a manager, can read an NCover report and discover whether or not you are being a good girl or boy. To avoid such difficulties, it would be best to not discuss this kind of tool with non-programmers.
There is a copy of NCover on SourceForge that comes with source, but right now I am using a second free version, that does not ship with source. You can download the latest version of NCover from http://www.ncover.org/Downloads.html. The downloaded zip file contains a simple Windows install file called NCoverSetup.msi.You can run this file by double clicking on it, or by going to the command line and moving into the directory where the file lives and typing start NCoverSetup.msi. Since this is a command line utility, you might try to avoid the temptation to install the program in a directory that has spaces in its path.
You should next get a copy of the NCoverBrowser from http://www.sliver.com/dotnet/NCoverBrowser/. You can unzip the NCoverBrowser download into the same directory where you installed NCover itself. Then make sure that directory is on your path, go the command prompt, and type NCover:
NCover v1.3.2 - Code Coverage Analysis for .NET - http://ncover.org NOTE: This profile driver application is deprecated. Use NCover.Console or NCover.GUI instead. Usage: ncover /c <command line> [/a <assembly list>] /c Command line to launch profiled application. /a List of assemblies to profile. i.e. "MyAssembly1;MyAssembly2" /v Enable verbose logging (show instrumented code)
If you see output similar to what I show here, then you have probably installed NCover correctly.
NCover comes in the form of an executable with an unusual name: NCover.Console.exe. It can be used to run any .NET program:
In this case, however, we want to see the coverage for an unit test. To run a unit test, just execute nunit-console and pass in the name of the DLL that contains the tests that you want to examine. A simplified form of the unit test part of the command line would therefore look like this:
If you throw in NCover, you need to add the /c switch to specify the input. The end result looks like this:
NCover.console.exe /c "nunit-console" "NUnitDataTests.dll"
Throw in the paths to the various files involved, and you end up with the more complex command line:
NCover.Console /c "d:\bin\Compilers\NUnit 2.2\bin\nunit-console" \ "d:\src\csharp\NUnitDataTests.dll"
Please note that this would be typed all on one line, without the forward slash.
You can also add another parameter using the /o switch to specify the output file for your program run:
NCover.Console /c "d:\bin\Compilers\NUnit 2.2\bin\nunit-console" \ "d:\src\csharp\NUnitDataTests.dll" \ /o D:\src\csharp\webapps\CodeCoverage\Coverage1.xml
This is a lot to type all at one time. Therefore you may find the simplest way to run NCover is to build a batch file that will run one of your unit tests:
set COVERAGE_FILE="D:\src\csharp\webapps\CodeCoverage\Coverage1.xml" delete %COVERAGE_FILE% NCover.Console /c "d:\bin\Compilers\NUnit 2.2\bin\nunit-console" \ "d:\src\csharp\NUnitDataTests.dll" /o %COVERAGE_FILE% NCoverBrowser %COVERAGE_FILE% set COVERAGE_FILE = "";
Note that this batch file puts the output from a run of the NCover program in a file called Coverage1.xml. It then launches Coverage1.xml in the NCoverBrowser utility so that it is easy to read:
Reading the NCover Output
As you have seen, the simplest way to read the output from NCover is to use the NCoverBrowser. However, you do not have to use the browser.
The raw output from NCover is a an XML file. Let’s take a look at an abbreviated version of one section of that file, which corresponds to the FileSize method highlighted:
<method name="FileSize" class="Falafel.Utils.Files">
<seqpnt visitcount="13" line="100" column="4" document="Files.cs"/>
<seqpnt visitcount="13" line="102" column="5" document="Files.cs"/>
<seqpnt visitcount="13" line="103" column="5" document="Files.cs"/>
<seqpnt visitcount="0" line="107" column="5" document="Files.cs"/>
<seqpnt visitcount="13" line="109" column="3" document="Files.cs"/>
I’ve ended up cutting quite a bit of the XML to make the output from NCover more readable. For instance, the actual code shows the complete path to Files.cs, and the endpoints are also specified for each line: endline="107" endcolumn="15". However, what you see here should give you an idea of what the code actually looks like that NCover produces.
The most important point’s to notice here are that there are five active lines in the FileSize method, and that four of them were visited 13 times. One line, however, was never visited. If you look back, you can see a visual illustration of this fact.
NOTE: In this particular case, I have not written 13 different tests of the simple FileSize method. Rather, this method gets called frequently by other parts of my code that are exercised by my unit tests.
Some methods in this program were visited exactly once:
<method name="FileExists" class="Falafel.Utils.Files">
<seqpnt visitcount="1" line="90" column="4" document="Files.cs"/>
<seqpnt visitcount="1" line="91" column="3" document="Files.cs"/>
Other methods weren’t visited at all:
<method name="RenameFile" class="Falafel.Utils.Files">
<seqpnt visitcount="0" line="193" column="4" document="Files.cs"/>
<seqpnt visitcount="0" line="194" column="3" document="Files.cs"/>
You could parse this XML file to make up reports of methods that need to be covered, or lines of code in methods that are not covered. In practice, however, I usually just use the NCoverBrowser to explore my code and let it highlight the methods that need attention.
NCover is an easy to use utility that provides a simple mechanism for discovering what percentage of your code is covered by unit tests, and specifically what methods need your attention.
In the ideal Test Driven Development model, programmers would always write their tests first, and then write methods that would fulfill the promise inherent in the tests. If you write your code that way, then you probably will have little need for a tool like NCover. However, if you are taking over someone else’s code that is not properly covered with unit tests, or if you do not follow the standard TDD methodologies, then you will probably find the easy to use NCover utility very useful.