Philippe Truche’s Blog

22 March 2011

Unit Testing 101 – Fundamentals of Unit Testing

Filed under: .NET, Testing — Tags: , , — Philippe Truche @ 8:40

This post is my first of 3 posts on unit testing, as introduced in https://philippetruche.wordpress.com/2011/03/11/unit-testing-a-curriculum/.  Unit Testing 101 is targeted at audiences that have little to no familiarity with unit testing.  This might include junior developers and even managers in software development

Table Of Contents

  • What? Who? Why?
  • Deciding what to test…
  • …and more importantly, when to write the tests.
  • Popular unit testing frameworks available to .NET
  • Unit test prototype
  • Writing your first unit tests

What Is Unit Testing? Who Writes The Tests? Why Do It?

The consensus is that Kent Beck introduced unit testing.  There are a number of definitions of unit testing – personally I like the definition from the book Pragmatic Unit Testing: “unit tests are performed to prove that a piece of code does what the developer thinks it should do.”  The key word is “developer.”  The developers get to write the unit test code.

This leads me to my next point.  Unit Testing isn’t a testing activity; in fact, RUP clearly identifies unit testing as belonging to the implementation discipline (see http://en.wikipedia.org/wiki/RUP).

So if this is an activity that is performed as the source code is being developed, then what are some of the benefits that developers get from writing unit tests?

  • They get living documentation of how the code works.
  • They get the ability to refactor code quickly and safely through automated regression tests.
  • It improves the low-level design of their classes.
  • It reduces risk. Writing tests helps drive the code quality up, thus reducing the number of bugs found much later in the software development lifecycle (SDLC).

In fact, unit tests are such a part of development that Michael Feathers defined legacy code as any code that does not have tests (see http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf)

Deciding What To Test…

“What should I test?”  This is an interesting question I get often enough from developers and managers who are new to unit testing.  The question in itself is disconcerting because it reveals just how little is understood about unit testing and often stands as a euphemism for “What is the minimal amount of testing that I can get away with?”

I don’t have a real easy answer to this question.  There are no hard and fast rules that I can provide someone with.  Rather, I prefer to remind developers that the more they write unit tests, the more they improve their design abilities and the better the software they write.

Would I test a class without behaviors?  Probably not.  But what if I have an entity on which I add validation attributes (using the EntLib Validation Block or Data Annotations)?  I would definitely test the validations to ensure that the validators attached to properties are behaving as expected.

So my general rule is to write unit test that focus on behaviors and where there is value in performing this activity.

…And More Importantly, When To Write The Tests.

This is definitely the more important question.

As a developer, you get the most value from creating the unit tests when creating the code.  In fact, rarely does it make sense to go back and write unit tests after the code has been written.  Instead, you should write your tests as you write the source code.  Do I sometime s write my test before the code?  Yes – when I have a good idea what I’d like the API to look like.  Sometimes I write the test after creating the initial source code.  Most often, I find that writing source code and unit tests is a bit of the chicken and the egg problem – which one came first?

When dealing with bugs identified by testing teams or customers, you should definitely write tests before fixing the bugs.  The reason for this is twofold:

  • It helps reproduce the issue
  • It creates a test for a missing scenario

Popular Unit Testing Frameworks On The .NET Platform

I often get the question: “Which unit testing framework do you use?”  There are a number of frameworks specifically for .NET – most are open source and one is Microsoft’s very own.  They are all fairly similar, and I think which one you use comes down to personal preference.

The more important point is not to confuse the tests runners and the testing framework.  For example, I can run MbUnit tests (part of Gallio) from within Microsoft Visual Studio, and vice versa, I can run Microsoft unit tests from the Gallio Icarius runner.

This being said, here is a list of unit testing frameworks.  It includes, but is not limited to:

Qualities of Unit Tests

In preparing this course, I researched a few sites that would help establish a guideline as to what a good unit test should look like.  I ended up settling on a set of quality attributes that unit tests should have.  These quality attributes were presented by Peli De Halleux in an advanced unit testing session in Spain (see http://channel9.msdn.com/blogs/channel9spain/microsoft-pexmoles–advanced-unit-testing-aspects-13).  Of course, my own experience also shaped this list.

Unit tests should have the following quality attributes:

  • Atomic.
    • No dependencies between tests.  If test[i] must be run after test[j], you have an issue.  With some unit test frameworks (e.g. MbUnit), you can specify the order in which you want to run tests.  Still, it is something you should avoid.
    • Tests can be run in any order.
    • Tests can be run repeatedly.  I can’t tell you how many times I’ve seen tests that only work the second time and on subsequent runs.  That usually points to some initialization issues.  Fix them!
    • Tests can be run concurrently.  If you have thousands of tests and you are running a continuous integration process, this is rather important.  In general, when a test fails unless it is run on its own, there is some sort of shared state issue.  Fix it!
  • Trustworthy.
    • Tests should run every time on any machine.  “It works on my machine” is a team development killer.  The tests should run on any machine.  I have seen tests where a local resource from the user’s folder is used!!!  If doing a Get Latest on the solution does not get me all required dependencies, this is not going to work.   If there is one or more failing tests at any point in time, how can the tests be trusted?  What if you make some changes to the source code and there were 50 breaking tests before you started?  You make the change, run the tests, and there are still 50 breaking changes (which tests failed?  are they still the same ones?).  The fact is, you don’t know.  But when all tests pass and you make a change that leads to one or more failing tests, you can evaluate the impact of your changes and either fix the source or fix the tests.
    • Beware “integration” tests and separate them into their own projects with the word “integration” in them.  Unit tests have to run fast – tests that have dependencies on databases or are otherwise not self-contained should be separated out.  My personal preference is to put them into their own test projects.  I expect these tests will take longer.  At the same time, I expect that unit tests will run very fast.  This is really critical for continuous integration to be efficient.
  • Maintainable.
    • Test code is code also.  The same design principles we apply to source code should be applied to test code.  This is important because tests change over time to adjust for changing source code, so it is important that the tests be maintainable too.
    • Avoid repeating code.  You can use fixture setup and test setup methods to refactor otherwise repeating code.  You can also create helper classes to set up test data.
  • Readable.
    • This is really important.  I should be able to open any tests and understand them.  Otherwise, it is difficult to maintain the tests.
    • There are many ways to name the test methods.  I really like the suggestion I’d heard on a Podcast and explained here: http://osherove.com/blog/2005/4/3/naming-standards-for-unit-tests.html.  It follows this pattern: <Method_Being_Tested>_<Scenario>_<Expected Behavior>.  It works pretty well for me; I encourage you to try it.

A Unit Test Prototype

I have been testing for many years, and I found a pattern repeated over and over.  I did not have a name for it, but once I started working with RhinoMocks (a mocking framework – I will introduce this in the 201 or 301 course), I learned a name that I thought described well what I’d seen in my tests and other people’s tests.

The pattern is named the Arrange-Act-Assert (AAA) pattern.

  • Arrange.  Populate any objects and/or create any necessary Mocks or Stubs required by your test.
  • Act. Execute the code being tested.
  • Assert.  Verify that expectations were met or report failure.

Asserting is the critical step in any unit tests.  If you are not making an assertion, then you are not really unit testing.  How many assertions should there be you might ask?  Preferably one and only one.  On occasion I may have a couple of assertions is they are strongly related and it does not make sense to slit the test into two tests.  The problem with having too many assertions in a test is that when a test fails, it can be difficult to understand right away what the problem is (remember the single responsibility principle?).

Writing Your First Unit Tests

While I was teaching the class, I showed the students samples I’d put together in Visual Studio.  For the readers of this blog post, I encourage you to try the tutorials provided by the unit testing frameworks.  In particular, I find NUnit to be a good place to start because the tutorial is well written and easy to follow.  But that does not mean you can’t work through other tutorials.

Coming next, Unit Testing 201 and 301.

Until then, happy unit testing!

Advertisements

11 March 2011

Unit Testing – A Curriculum

Filed under: .NET, Testing — Tags: , , , , , — Philippe Truche @ 3:36

The subject of unit testing is one that comes over and over in my professional experience. I have been asked many times to teach developers how to unit test. It seems like the words “unit testing” somehow got some level of fame and everyone wants to claim that they are doing it. But to do it well and to do it consistently seems elusive. Worse yet, there tends to be a dichotomy between development managers and the developers about what unit testing really is and why we unit test in the first place.

When asked to put a webinar together, I decided to break it down into 3 separate classes. My first one is called Unit Testing 101 and covers the fundamentals of unit testing. Though it focuses on concepts primarily, it also shows a basic unit test so as to get a flavor. I then created the second course: Unit Testing 201 – Intermediate Unit Testing. In this course, I look at techniques for handling dependencies and introduce basic concepts of test doubles and mocking frameworks. Finally, I reserved the most advanced techniques for Unit Testing 301 – Advanced Unit Testing: I cover topics on compiler techniques in Visual Studio to access members that aren’t public, how to address dependencies without necessarily resorting to using an IOC container, techniques for faking out the HttpContext, employing last-resort frameworks like Microsoft’s Moles to stub out System.DateTime for example, how to manipulate the configuration file, how to host WCF in process (I know, this is bordering on integration tests clip_image001), and a basic overview of Microsoft Pex’s framework.

Yes, it sounds like there is a lot to know to write good unit tests. In fact, just like anything else – practice makes perfect.

In my next post, I will go through the contents of Unit Testing 101.

25 April 2007

Using Visual Studio Team System to load test web services

Filed under: .NET, Testing, VSTS, Web Services — Philippe Truche @ 1:55

I recently had to load test web services for the smart client application we have developed.  I wanted to be able to go through quick cycles (test, evaluate, adjust) without having to wait for a few days for a load test slot in our LoadRunner lab, so I offered to use Visual Studio team system to perform the tests.  I already had experience with Microsoft’s previous load test tool – Application Center Test.  I was pleased with VSTS’ version of a load test tool.  Not quite as powerful as LoadRunner, but much more accessible in ease of use.

Anyway, I used VSTS successfully to load test web services and satisfy the following requirements:

  • Randomly pick data from a set for a given test.
  • Invoke the web services using HTTP POSTs (i.e. without having to create a client)
  • Use a configuration file to store the SOAP envelopes needed to make the web service calls.

Picking data randomly 

When using VSTS tests, the test methods are decorated with the [TestMethod] attribute.    The [DataSource] attribute can also be used on the test method to indicate that the test is data-driven.  Caution: the test is executed as many times as there are rows in the data source.  That may be OK when performing unit tests, but it is not OK for a load test.  So how do you get around this?  Well, I got my data source to return only one row picked randomly.  Using SQL Server 2005 Express, I defined views that would select one row of data using the following construct: “SELECT TOP(1) columnName1 AS alias1, columnName2 AS alias2, …. , columnNameN as aliasN FROM tableName ORDER BY NEWID();”

You might wonder why I am using an aliases on the column names.  The reason why I did this was to decouple the column names in the tables from the element names in the SOAP envelopes.  When the data source provides the row of data, I have a utility running through the column names of the DataRow object and substituting the data in the SOAP envelope based on the name of the columns found in the DataRow provided through the [DataSource] attribute.  This simple convention makes it very easy to parameterize the SOAP sent to the service.  Because the load test client should be as efficient as possible, I use a simple string parsing algorithm to perform the substitutions.

Invoke the web services using HTTP POSTs

I created a static class that uses an HttpWebRequest object to create the POST.   The result is returned into an HttpWebResponse object by simply doing this:

using ( HttpWebResponse response = (HttpWebResponse)request.GetResponse())

{

return response.StatusCode.ToString();

}

If the web server responds with error response codes (e.g. HTTP 401, HTTP 500), VSTS detects that and reports on the errors.  This is much better than Application Center Test where I was left to parse through the responses to make sure I did not get an error back.

Use a configuration file to store the SOAP envelopes needed to make the web service calls.

For this, my team member created an XML file that is loaded into an XmlDocument from which XmlNodes are selected and stored into an object that holds the SOAPAction to perform, the SOAP envelope to send the service, and the Url to use.  Yes, I know, this is very ASMX-centric, and will likely need to change with WCF, but we were not developing a framework either.  We needed a point solution to get going quickly.

Leave a comment if you have any questions or comments.

Blog at WordPress.com.