This post is my first of 3 posts on unit testing, as introduced in https://philippetruche.wordpress.com/2011/03/11/unit-testing-a-curriculum/. Unit Testing 101 is targeted at audiences that have little to no familiarity with unit testing. This might include junior developers and even managers in software development
Table Of Contents
- What? Who? Why?
- Deciding what to test…
- …and more importantly, when to write the tests.
- Popular unit testing frameworks available to .NET
- Unit test prototype
- Writing your first unit tests
What Is Unit Testing? Who Writes The Tests? Why Do It?
The consensus is that Kent Beck introduced unit testing. There are a number of definitions of unit testing – personally I like the definition from the book Pragmatic Unit Testing: “unit tests are performed to prove that a piece of code does what the developer thinks it should do.” The key word is “developer.” The developers get to write the unit test code.
This leads me to my next point. Unit Testing isn’t a testing activity; in fact, RUP clearly identifies unit testing as belonging to the implementation discipline (see http://en.wikipedia.org/wiki/RUP).
So if this is an activity that is performed as the source code is being developed, then what are some of the benefits that developers get from writing unit tests?
- They get living documentation of how the code works.
- They get the ability to refactor code quickly and safely through automated regression tests.
- It improves the low-level design of their classes.
- It reduces risk. Writing tests helps drive the code quality up, thus reducing the number of bugs found much later in the software development lifecycle (SDLC).
In fact, unit tests are such a part of development that Michael Feathers defined legacy code as any code that does not have tests (see http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf)
Deciding What To Test…
“What should I test?” This is an interesting question I get often enough from developers and managers who are new to unit testing. The question in itself is disconcerting because it reveals just how little is understood about unit testing and often stands as a euphemism for “What is the minimal amount of testing that I can get away with?”
I don’t have a real easy answer to this question. There are no hard and fast rules that I can provide someone with. Rather, I prefer to remind developers that the more they write unit tests, the more they improve their design abilities and the better the software they write.
Would I test a class without behaviors? Probably not. But what if I have an entity on which I add validation attributes (using the EntLib Validation Block or Data Annotations)? I would definitely test the validations to ensure that the validators attached to properties are behaving as expected.
So my general rule is to write unit test that focus on behaviors and where there is value in performing this activity.
…And More Importantly, When To Write The Tests.
This is definitely the more important question.
As a developer, you get the most value from creating the unit tests when creating the code. In fact, rarely does it make sense to go back and write unit tests after the code has been written. Instead, you should write your tests as you write the source code. Do I sometime s write my test before the code? Yes – when I have a good idea what I’d like the API to look like. Sometimes I write the test after creating the initial source code. Most often, I find that writing source code and unit tests is a bit of the chicken and the egg problem – which one came first?
When dealing with bugs identified by testing teams or customers, you should definitely write tests before fixing the bugs. The reason for this is twofold:
- It helps reproduce the issue
- It creates a test for a missing scenario
Popular Unit Testing Frameworks On The .NET Platform
I often get the question: “Which unit testing framework do you use?” There are a number of frameworks specifically for .NET – most are open source and one is Microsoft’s very own. They are all fairly similar, and I think which one you use comes down to personal preference.
The more important point is not to confuse the tests runners and the testing framework. For example, I can run MbUnit tests (part of Gallio) from within Microsoft Visual Studio, and vice versa, I can run Microsoft unit tests from the Gallio Icarius runner.
This being said, here is a list of unit testing frameworks. It includes, but is not limited to:
- csUnit. http://www.csunit.org/
- MbUnit (V3 is part of Gallio). http://www.gallio.org/
- Nunit. http://www.nunit.org/
- Visual Studio Unit Testing Framework (Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll).
- xUnit.net. http://xunit.codeplex.com/
Qualities of Unit Tests
In preparing this course, I researched a few sites that would help establish a guideline as to what a good unit test should look like. I ended up settling on a set of quality attributes that unit tests should have. These quality attributes were presented by Peli De Halleux in an advanced unit testing session in Spain (see http://channel9.msdn.com/blogs/channel9spain/microsoft-pexmoles–advanced-unit-testing-aspects-13). Of course, my own experience also shaped this list.
Unit tests should have the following quality attributes:
- No dependencies between tests. If test[i] must be run after test[j], you have an issue. With some unit test frameworks (e.g. MbUnit), you can specify the order in which you want to run tests. Still, it is something you should avoid.
- Tests can be run in any order.
- Tests can be run repeatedly. I can’t tell you how many times I’ve seen tests that only work the second time and on subsequent runs. That usually points to some initialization issues. Fix them!
- Tests can be run concurrently. If you have thousands of tests and you are running a continuous integration process, this is rather important. In general, when a test fails unless it is run on its own, there is some sort of shared state issue. Fix it!
- Tests should run every time on any machine. “It works on my machine” is a team development killer. The tests should run on any machine. I have seen tests where a local resource from the user’s folder is used!!! If doing a Get Latest on the solution does not get me all required dependencies, this is not going to work. If there is one or more failing tests at any point in time, how can the tests be trusted? What if you make some changes to the source code and there were 50 breaking tests before you started? You make the change, run the tests, and there are still 50 breaking changes (which tests failed? are they still the same ones?). The fact is, you don’t know. But when all tests pass and you make a change that leads to one or more failing tests, you can evaluate the impact of your changes and either fix the source or fix the tests.
- Beware “integration” tests and separate them into their own projects with the word “integration” in them. Unit tests have to run fast – tests that have dependencies on databases or are otherwise not self-contained should be separated out. My personal preference is to put them into their own test projects. I expect these tests will take longer. At the same time, I expect that unit tests will run very fast. This is really critical for continuous integration to be efficient.
- Test code is code also. The same design principles we apply to source code should be applied to test code. This is important because tests change over time to adjust for changing source code, so it is important that the tests be maintainable too.
- Avoid repeating code. You can use fixture setup and test setup methods to refactor otherwise repeating code. You can also create helper classes to set up test data.
- This is really important. I should be able to open any tests and understand them. Otherwise, it is difficult to maintain the tests.
- There are many ways to name the test methods. I really like the suggestion I’d heard on a Podcast and explained here: http://osherove.com/blog/2005/4/3/naming-standards-for-unit-tests.html. It follows this pattern: <Method_Being_Tested>_<Scenario>_<Expected Behavior>. It works pretty well for me; I encourage you to try it.
A Unit Test Prototype
I have been testing for many years, and I found a pattern repeated over and over. I did not have a name for it, but once I started working with RhinoMocks (a mocking framework – I will introduce this in the 201 or 301 course), I learned a name that I thought described well what I’d seen in my tests and other people’s tests.
The pattern is named the Arrange-Act-Assert (AAA) pattern.
- Arrange. Populate any objects and/or create any necessary Mocks or Stubs required by your test.
- Act. Execute the code being tested.
- Assert. Verify that expectations were met or report failure.
Asserting is the critical step in any unit tests. If you are not making an assertion, then you are not really unit testing. How many assertions should there be you might ask? Preferably one and only one. On occasion I may have a couple of assertions is they are strongly related and it does not make sense to slit the test into two tests. The problem with having too many assertions in a test is that when a test fails, it can be difficult to understand right away what the problem is (remember the single responsibility principle?).
Writing Your First Unit Tests
While I was teaching the class, I showed the students samples I’d put together in Visual Studio. For the readers of this blog post, I encourage you to try the tutorials provided by the unit testing frameworks. In particular, I find NUnit to be a good place to start because the tutorial is well written and easy to follow. But that does not mean you can’t work through other tutorials.
Coming next, Unit Testing 201 and 301.
Until then, happy unit testing!