Bugfree.dk – Ronnie Holm's blog

Update, Nov 25, 2010: Rather than mimicking the directory structure of the assembly under test in a separate test assembly, I’m now more inclined to keep code under test and test code in the same assembly.

Update, Aug 5, 2009: This post grew out of a couple of presentations that I did on the subject back in May and June. Here are my slides from The Unit testing and mocking presentation. Among other things, they contain a few C# samples elaborating on some of the points below.

This post is born of the need to formalize a set of guidelines on how to write and organize tests. In the past I couldn’t help feeling that with the lack of guidelines I had to start over explaining my views on every new project. With no guidelines the tests written quickly grew unmaintainable, giving unit testing a bad name.

Why unit test

Unit testing done right helps build confidence in the code base and drives forward development. Both in terms of forcing one to reflect on the requirements by authoring tests, but also by providing a foundation for developing more modular and testable code.

Given the cyclomatic complexity of even simple methods, however, not every path through a method is worth exercising with a test. Instead, focus on writing representative unit tests for good and bad scenarios.

Unit test != integration test

For object-oriented code a unit test is one that exercises a class in isolation, without the code under test relying on other classes to carry out its operation. Similarly, a unit test shouldn’t rely on the presence of a database or a key/value pair in a configuration file for it to run. If it did, it would be an integration test. Not that there’s anything wrong with integration tests. They just take on more dependencies and hence tend to be more brittle. And so they cause more false positives because some dependent part isn’t properly configured. The key point, though, is not to confuse unit testing with integration testing.

Typically, an object delegates part of its operation to other objects. Writing modular and testable code therefore involves the application of design patterns, such as Inversion of Control (IoC). With IoC a test can inject (fake) components into the object under test. The fake components share their interfaces with the real ones, but the test controls how they interact with the object under test and hence how the object under test behaves.

Keep up quality

As tests are written it’s vital for the understandability and maintainability of the test suite to keep them small and focused. As a rule of thumb a test should amount to no more than 10-15 lines of code. Longer tests are indicative of too much functionality being tested at once or that code common to multiple tests should be refactored into helper methods.

In terms of quality, the code comprising the tests should be of production code quality, i.e., the code must be kept clean and refactored as the need arises. Otherwise, tests will start to emit the classic code smells. In the longer run code smells lead to tests that are not maintainable and for the time that went into writing them to be wasted.

Organizing test code

Assuming the use of Visual Studio (VS), for each VS project with code that one wants to test, create matching projects that host the various kinds of tests, i.e., for the Acme.Intranet.Search project, create the following test related projects:

    Acme.Intranet.Search.Common
    Acme.Intranet.Search.UnitTest
    Acme.Intranet.Search.IntegrationTest

The Acme.Intranet.Common project is optional and may include code that is shared between test projects, such as custom assertions. As for the Acme.Intranet.Search.UnitTest project it should be fairly self-contained. One should be able to move the common assembly, the test assembly, and the business code assembly to another machine and have the tests execute there without further setup. Should a test rely on, say, a data file with test data, then include the file as an embedded resource within the test assembly.

Finally, within each test project, a class should be created for each class under test. In addition, the directory structure should match the namespace structure of the class under test, e.g., suppose the fully qualified name of a class is Acme.Intranet.Search.Business.Crawler, then create the following directory structure within the test assembly:

    Acme.Intranet.Search.UnitTest
        Acme
            Intranet
                Search
                    Buesiness
                        CrawlerTest.cs

While it’s important to write tests, it’s even more important to know where to put and find tests for a given functional area.

Naming tests

As far as naming and structure goes, a test should look something along these lines:

    [TestClass]
    public class SomeClassTest {
        [TestMethod]
        public void SomeMethod_should_set_error_message_when_no_                    connection_string_is_configured() {
            // arrange
            // act
            // assert
        }
    }

The test should generally start with the name of the method or property being tested, followed by the word "should" followed by the successful outcome in a descriptive form. Because names of tests tend to be longer than those of regular methods, underscores are used for ease of readability. In addition, the body of most tests should be composed of three parts: (1) the arrange part that sets up the object under test and possibly injects fake dependencies into it. (2) The act part then exercises the method under test, and finally (3) the assert part that verifies that expected state and/or behavioral changes did indeed take place.

Gathering metrics

Whenever a build is kicked of (on a build server) it should exercise all tests. Should a test fail, it should cause the entire build to fail, stressing the importance of keeping tests green at all times. Furthermore, a build report should include basic metrics such as code coverage, number of tests run, time spend running the tests, and so forth.

Keep in mind, though, that a high degree of code coverage isn’t a goal in itself. Instead, focus on writing solid, focused, and representative tests that eventually drive up code coverage.

Tooling

As far as .Net and tooling goes, MSTest or NUnit should be used in concert with TypeMock or Rhino Mocks. The use of TypeMock may be preferred over Rhino Mocks because of its unique approach to mocking. TypeMock doesn’t create fake objects by emitting MSIL and dynamically loading a runtime-generated assembly into the test runner. Instead, TypeMock hooks into the CLR APIs and intercepts calls as the unit test executes. From a coding point of view the TypeMock approach may not change much, but from a functionality point of view it enables the testing of legacy code or new code not written with IoC in mind.

This post details my first use of TypeMock, a commercial mocking framework, for testing a web part that displays a dropdown bound to a LINQ query. Based on the item selected, the web part then displays a result bound to another LINQ query. For the purpose of becoming familiar with TypeMock, though, the focus of this post is on testing the code that initializes the LINQ data context. It does so by reading a connection string from a configuration file and passing it to the data context.

In order to test code whose branching behavior depends on a value read from a configuration file, the test should provide various inputs and assert that the code under test acts accordingly. In addition, the test should run in milliseconds to encourage frequent runs and it should be self-contained, meaning no setup should be required before running the test. These requirements are best met with the web part running in a controlled environment. Yet, as far as the code under test goes, it should behave as if part of an Asp.Net page.

For the sake of brevity, here’s my web part with the code to initialize the data context:

    public class MyWebPart : WebPart {
        MyDataContext _db;    /* LINQ database context */
        string _error;        /* Early stage error message */    	

        public MyWebPart() {
            Initialize();
        }

        private void Initialize() {
            string c = GetMyConnectionString();
            if (c == null)
                 _error = "Unable to do something";
            else
                _db = new MyDataContext(c);
        }

        private string GetMyConnectionString() {
            System.Configuration.ConnectionStringSettings c =
                System.Configuration.ConfigurationManager.
                    ConnectionStrings["MyConnection"];
            return c == null ? null : c.ConnectionString;
        }

        protected override void Render(HtmlTextWriter w) {
            // display error message if set
        }
    }

Notice how the constructor calls out to an Initialize method. Otherwise code within the constructor would fire before I’d have a chance to setup a controlled environment around the web part and I wouldn’t be able to test it. Turning the attention to Initialize itself, it’s made up of only a simple if statement. The idea is that if I can’t mock something as simple as reading a value from a configuration file, most likely I can’t mock something more involved either. On the other hand, for a developer already familiar with TypeMock and test first development, the code above may be born of the need to satisfy a failing test.

I want to test Initialize by controlling how GetMyConnectionString gets to return the connection string. While running in an Asp.Net context, GetMyConnectionString would attempt to read the string from web.config. Within my controlled environment, however, I want to inject into the web part an alternate implementation that returns a value of my choosing. That way I can control the path through Initialize and inspecting the state of the _error or _db fields, I’m able to determine if the code behaved as intended.

Using the method chaining DSL of TypeMock, the test takes on this form (of course, having more tests, I’d move shared code into helper methods decreasing the test to code ratio):

    [TestClass, ClearMocks]
    public class MyWebPartTest {
        [TestMethod, Isolated, VerifyMocks]
        public void Should_set_error_message_when_no_
                    connection_string_is_configured() {
            // arrange
            MyWebPart fake = Isolate.Fake.Instance<MyWebPart>();
            Isolate.NonPublic.WhenCalled(fake,
                "Initialize").CallOriginal();
            Isolate.NonPublic.WhenCalled(fake,
                "GetMyConnectionString").WillReturn(null);
            PrivateObject handle = new PrivateObject(fake);

            // act
            handle.Invoke("Initialize", null); 

            // assert
            Isolate.Verify.NonPublic.WasCalled(fake,
                "GetMyConnectionString");
            string error = (string)handle.GetField("_error");
            MyDataContext ctx = (MyDataContext)handle.GetField("_db");
            Assert.AreEqual("Unable to do something", error);
            Assert.AreEqual(null, ctx);
        }
    }

There’re three parts to tests using TypeMock: (1) the arrange part sets up expectations of what’s going to happen when a method, a property, or some other part of the code under test is invoked; (2) the act part carries out the actual interaction with the object under test, turning to PrivateObject to invoke the Initialize method; and finally (3) the assert part verifies that what was supposed to happen did happen. Worth noting is that this test asserts both state and behavior, i.e., as part of executing the code under test, _error or _db should’ve been set to an appropriate state and GetMyConnectionString should’ve been called once and only once.

This concludes my example, which I believe illustrates the basics of testing and mocking using MS Test and TypeMock. Comparing and contrasting TypeMock to other popular mocking frameworks, TypeMock has the distinct feature that it can mock almost anything, with or without using interfaces. Some argue that not enforcing the use of interfaces is also the disadvantage of TypeMock, because for code to be truly testable, it should be modularized using the Inversion of Control principle. However, with legacy code, such as the .Net framework or the SharePoint API, to me TypeMock appears to bridge the two worlds nicely, leaving the developer in charge.

To learn more about the basic concepts of mocking, Inversion of Control, and Dependency Injection (containers), I’d recommend listening to a couple of Alt.Net podcasts: Dependency Injection and Inversion of Control and More Dependency Injection and Inversion of Control.