Testing in .Net Core

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

Testing is something we need to do to increase the confidence in what we are building. After all, we want to ship working software. We do know that bugs happen so we need to be disciplined and preferably add a test for each bug so we at least know we fixed that one. It's a game of whack a mole and we need to keep up. There are different ways of doing tests though, unit tests, integration tests, E2E tests. In this article, we will focus on unit tests and ensure we adopt some good practices.

TLDR; This is a primer on testing in .Net Core. If you are completely new to testing or testing in .Net Core, then this is for you. I will follow up with a more advanced article on testing discussing object mothers, mocking and other things.

In this article, we will cover

• Creating test projects and running your tests
• Different types of testing, here we will mention different types of testing if you are completely new to testing and need a primer on the different phases and levels.
• Authoring tests, here we will go through some good practices for naming and arranging your tests.

.Net Core supports testing using MsTest, xUnit as well as nUnit so you have quite a lot of choices. For this article, we have selected MsTest. Check the reference section for links to the other frameworks.

References

• xUnit testing
  This page describes how to use xUnit with .Net Core

• nUnit testing
  This page describes how to use nUnit with .Net Core.

• dotnet test, terminal command description
  This page describes the terminal command dotnet test and all the different arguments you can call it with.

• dotnet selective test
  This page describes how to do selective testing and how to set up filters and query using filters.

• Get started with C# and VS Code

• .Net Core Series on NuGet, Serverless and much more

WHY

We mentioned at the beginning of the article that testing is important especially in the context of bugs. I would argue and say that testing is the most important tool in your toolbox as a developer. Learning to check your work will not only build a great reputation with your clients but also with your colleagues. The better you are at testing your code and find various ways to do so, the better for everybody.

Different types of testing

There are different types of testing. They are used at different stages of the software's life cycle. You also write tests at different levels to test details or large scale behavior. We usually say that unit testing is meant to test implementation details where integration testing is used to test if two or more, larger pieces work together. We should test at all levels and life cycles.

Test last

We are at a point where we've written the whole software and the so-called happy path seems to work. With happy path we mean the scenario we think the client will use to carry out a task or whatever else is running our software, it could be other software :). In this scenario, we realize that we might get in trouble if something unexpected happens so we start adding tests at this point and logging to ensure that our software covers some scenarios that we think might happen. Adding tests at this point often feels like a chore and frankly not fun. But we must add them to call ourselves professionals.

Regression testing

Well, I would say that testing is something that should always be with you, a mentality of how can this break. Is it the type of input, is memory consumption, is there a dependency that answers with something we didn't expect?

You can't think of everything of course, even though you try and this deliberate analysis before, during and after you've written the code will make the code better for sure. Because you can't think of everything, you will have bugs. A good practice here is trying to write a test that proves that the bug exists in the first place. Once established you can go on to fix the bug and see your test pass. At least you've cured that symptom/bug.

The above isn't really regression testing but rather a good practice when you find a bug. Regression testing is more when you do a change, to add a feature or fix a bug and you rerun some tests and ensure everything is still working. It's tightly connected and a term you should know.

Refactoring

Of course, there's another case refactoring. Most codebases I've seen turns into an untangled mess over time as more and more features are added to it or the features themselves change. What was once an easy to understand piece of code becomes a ball of spaghetti. At this point, you feel like starting from scratch, most likely. Having tests is a great way to feel confident that you can change code and it still does whats it's supposed to once you've stopped improving it through refactoring.

Test driven development, test first

This is a methodology in which you think out tests and write them before you actually start writing any production code. The idea is that you start with a failing test and then write production code and the test passes. This is called red-green testing. Red for failing test and Green for passing test. Some people like this way of working and others feel constrained by it. On the positive side, you don't write code that you won't need as all the code you write are meant to make a certain test scenario pass.

Integration testing

This is usually some high-level testing where we test if different components, large parts of the system, work together. It's usually a certain layer talking to another layer, e.g an API layer talking to a data layer for example or even a separate component talking to another component that together makes our a large complex system.

WHAT - creating a demo

In this article, we aim to test at a lower level, the unit testing level. We will show how you can easily create a test project, author tests and run them. Hopefully, we will also give some great guidance on authoring and how to improve the tests. We will carry out the following:

1. Scaffold a test project
2. Author a test
3. Run the test
4. Improving our test

Scaffold a test project

Essentially we want to create a solution with application code as one project and test code in a separate project. So the overall solution will be this:

solution
  app // project containing our implementation
  app-test // project containing our tests

Creating the solution

Let's start by creating a directory. This hold our solution. The directory can be called anything you like but we call ours TestExample so we do:

mkdir TestExample

Next we will create a solution by calling:

cd TestExample
dotnet new sln

Create the library project

Thereafter we will create a library project that will hold our production code:

dotnet new classlib -o app

Next we want to add the project reference to the solution, like so:

dotnet sln add app/app.csproj

Creating the test project

Thereafter we want to create a test project like so:

dotnet new mstest -o app-test

It should produce an output looking something like so:

and add a reference to our solution like so:

dotnet sln add app-test/app-test.csproj

Lastly, we want to add a reference to the app project in the app-test project so we can refer to code in the app that we want to test. To do so type the following commands:

cd app-test
dotnet add reference ../app/app.csproj
cd ..

Writing a test

Let's create a file CalculatorTest.cs and give it the following content:

// CalculatorTest.cs

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace app_test 
{
  [TestClass]
  public class CalculatorTest 
  {
    [TestMethod]
    public void Add() 
    {
      Assert.AreEqual(1,1);
    }
  }
}

The decorator TestClass means our class CalculatorTest should be treated as a test class. TestMethod is a decorator we add to our Add() method to signal that this is a test method.
Inside the method bofy of Add() we add an assertion statement Assert.AreEqual(1,1). The prototype of AreEqual() is AreEqual(expected, actual). This is all we need to create a test.

Running a test

Now, let's run it. We do that by either:

1. A terminal command

This command dotnet test comes with a lot of arguments so make sure to check out the documentation page

1. Click Run Test in VS Code

As you can see above we can easily debug a test as well and if we look at the class level there will be a link to debug/run all tests in the class.

We have a thing called filter that we can use to run specific tests that we target. Let's add another test like so:

[TestMethod]
[Priority(1)]
public void Subtract() 
{
  Assert.AreEqual(1,2);
}

Above we have not only added test method Subtract() but we have also added a decorator Priority(1). We can filter so we only run tests matching this name by typing the following:

dotnet test --filter Priority=1

As you can see above it's only running 1 test Subtract(), it's skipping over our Add() test.

There's another decorator we could be adding, TestCategory. Let's add the following two tests:

[TestMethod]
[TestCategory("DivideAndMultiply")]
public void Divide() 
{
  Assert.AreEqual(1,1);
}

[TestMethod]
[TestCategory("DivideAndMultiply")]
public void Multiply()
{
  Assert.AreEqual(1, 1);
}

The two tests have the same tags DivideAndMultiply. We can filter on that by typing dotnet test --filter TestCategory=DivideAndMultiply. The TestCategory allows us to create more descriptive tags than the Priority one.

A more real scenario

Ok, we have a test class CalculatorTest but let's face it how many times in production are we coding a calculator? Yea I didn't think so. Most likely we are doing something like an e-commerce company. So let's dream up what that could look like. Let's talk about orders and how to evaluate them.

That's easy I got an order and a number of items and then just loop through them and I got a total and then I add some shipping cost and BOOM I'm done :)

I wish it was that easy. When you are starting out maybe, but then someone mentions the word discount and sure having one discount on 20% is easy to calculate but suddenly it grows into a monster. Before you know it you got a discount on different product types, 3 for 2 discounts, holiday discounts, discounts on all items that are tagged in a certain way and so on and so forth. Trust me on this Discounts soon becomes it's own out of control spaghetti monster that takes a team to manage and you can imagine what the code looks like trying to keep up with that.

Starting out

Let's start small though. Imagine you've just started a WebShop and you need to write some code to support calculating the sum of an order. We start by creating the file Order.cs in our app project and give it the following content:

// Order.cs

using System;
using System.Collections.Generic;

namespace OrderSystem {
  public enum ProductType 
  {
    CD,
    DVD,
    Book,
    Clothes,
    Game
  }

  public class Product
  {
    public string Name { get; set; }
    public string Description { get; set; }
    public ProductType Type { get; set; }
  }

  public class OrderItem
  {
    public int Quantity { get; set; }
    public double Price { get; set; }
    public Product Product { get; set; }
  }

  public class Order 
  {
    public List<OrderItem> Items { get; set; }
    public DateTime Created { get; set; }
  }
}

Then we create a file OrderHelper.cs that will help us calculate the sum of an order and related Order logic. For now, we give it the content:

namespace OrderSystem
{
  public class OrderHelper
  {
      public OrderHelper() 
      {}

      public static double Cost(Order order)  
      {
        return 0;
      }
  }
}

Ok, that doesn't do much, that's the point, cause now we will write a test.

Let's create a file OrderHelperTest.cs in our app-test project and give it the following content:

using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OrderSystem;

namespace app_test
{
  [TestClass]
  public class OrderHelperTest
  {
      [TestMethod]
      public void ShouldSumCostOfOrder()
      {
          // Arrange

          // Act

          // Assert
      }
  }
}

Good practice when authoring

Now let's talk about some good practices to use when authoring a test. Tests should be easy to read. The test name should say something about what we are doing and what we hope of the outcome. Giving it the name ShouldSumCostOfOrder() tells us what's going on. You could be even more specific and say something about the outcome e.g ShouldSumOrderTo30(). Ultimately it's what makes the most sense to you and what you are testing.

Next thing to look at is the three As, Arrange, Act and Assert. In arrange you should set up everything you need. In Act you should call your production code. In the final Assert step, you should verify you got the outcome you expected.

As you can see we have a method ShouldSumCost().

Writing the test

Ok, we understand a bit more on naming and what steps to have in a test so let's flesh out the test a bit. The first thing we need to focus on is setting up our test, the so-called Arrange phase, with the following code:

// Arrange
var productCD = new Product()
{
    Type = ProductType.CD,
    Name = "Nirvana",
    Description = "Album"
};
var productMovie = new Product()
{
    Type = ProductType.DVD,
    Name = "Gladiator",
    Description = "Movie"
};

var order = new Order() 
{
    Items = new List<OrderItem>() { 
        new OrderItem(){ 
            Quantity = 1, 
            Price = 10,
            Product = productCD
        },
        new OrderItem(){
            Quantity = 2,
            Price = 10,
            Product = productMovie
        } 
    }
};

we could easily move these to another class to make the test easier to read. For now though let's keep going and move to the next phase Act.

// Act
var actual = OrderHelper.Cost(order);

Note the naming of the variable actual. It's good practice to use names like actual and expected when writing tests as these are terms other developers know and it will make it easier to read your code.

Finally we have the Assert phase where we check whether the code does what we think it does:

// Assert
Assert.AreEqual(30, actual);

The full file OrderHelperTest.cs should look like this:

// OrderHelperTest.cs

using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OrderSystem;

namespace app_test
{
  [TestClass]
  public class OrderHelperTest
  {
    [TestMethod]
    public void ShouldSumCostOfOrder()
    {
        // Arrange
        var productCD = new Product()
        {
            Type = ProductType.CD,
            Name = "Nirvana",
            Description = "Album"
        };
        var productMovie = new Product()
        {
            Type = ProductType.DVD,
            Name = "Gladiator",
            Description = "Movie"
        };

        var order = new Order() 
        {
            Items = new List<OrderItem>() { 
                new OrderItem(){ 
                    Quantity = 1, 
                    Price = 10,
                    Product = productCD
                },
                new OrderItem(){
                    Quantity = 2,
                    Price = 10,
                    Product = productMovie
                } 
            }
        };

        // Act
        var actual = OrderHelper.Cost(order);

        // Assert
        Assert.AreEqual(30, actual);
    }
  }
}

At this point our test will fail

Adding code

Ok, so we need to fix the test by adding a real implementation to OrderHelper.cs. Change the code to the following:

using System.Linq;

namespace OrderSystem
{
  public class OrderHelper
  {
      public static double Cost(Order order)  
      {
          return order.Items.Sum(i => i.Price * i.Quantity);
      }
  }
}

Rerunning the test we get:

Useful functionality

Ok, we've taken you through so-called Red-Green testing where you start out with writing a test, ensures it fails, it turns Red. Then we write the implementation. Finally, we rerun the test and it passes, it turns Green.

Many times we end up writing tests that tests the same flow through code, we just need to vary the input to make sure our production code really really works.

Let's take our CalculatorTest.cs file as an example. First, let's give it a real implementation Calculator.cs in the app project. Now give it the following content:

// Calculator.cs

namespace app 
{
  public class Calculator 
  {
    public static int Add(int lhs, int rhs)
    {
      return 0;
    }
  }
}

Update our CalculatorTest.cs to now say:

// CalculatorTest.cs

using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace app_test 
{
  [TestClass]
  public class CalculatorTest 
  {
    [TestMethod]
    public void Add() 
    {
      var actual = Calculator.Add(0,0);
      Assert.AreEqual(actual,0);
    }
  }
}

Running the test at this point means it passes. Now, can we trust our code at this point to work? I mean the test passes right? Because the implementation is so small we can tell that it WON'T work if we change the input. With larger implementations it might be harder to tell.

Let's try to find an approach!

Change your code to the following:

// CalculatorTest.cs

using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace app_test 
{
  [TestClass]
  public class CalculatorTest 
  {
    [DataTestMethod]
    [DataRow(0)]
    public void Add(int value) 
    {
      var actual = Calculator.Add(value, value);
      var expected = value + value
      Assert.AreEqual(actual, expected);
    }
  }
}

Note how we remove TestMethod and replace it with DataTestMethod. Also, note how we add DataRow which takes an argument. Now, this argument becomes the input parameter value. Running the test right now means it calculates 0 + 0 and thereby our test should still pass. Let's add some more scenarios though so the code looks like this:

using System;
using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace app_test 
{
  [TestClass]
  public class CalculatorTest 
  {
    [DataTestMethod]
    [DataRow(0)]
    [DataRow(1)]
    [DataRow(2)]
    public void Add(int value) 
    {
      Console.WriteLine("Testing {0} + {0}", value);
      var actual = Calculator.Add(value, value);
      var expected = value + value;
      Assert.AreEqual(actual,expected);
    } 
  }
}

Running the test now we get:

From the above we can see that 0 + 0 passes whereas 1 + 1 and 2 + 2 fails. Our code don't work so we need to fix it. Let's change Calculator.cs to the following:

namespace app 
{
  public class Calculator 
  {
    public static int Add(int lhs, int rhs)
    {
      return lhs + rhs;
    }
  }
}

Now if we run our tests we get:

What we just did is called data-driven testing. Let's stop here, the article is long enough already

Summary

We've taken you through creating a test project, authoring some tests and learn to run our test. Additionally, we have discussed some good practices in naming and how to improve your tests even further with data-driven testing.

I hope this was useful and that you are looking forward to the next article that aims to go more in-depth.