3

Writing deterministic tests in a variable world

 2 years ago
source link: https://headspring.com/2021/12/11/writing-deterministic-tests-in-a-variable-world/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Writing deterministic tests in a variable world

by Dane Schilling | Dec 11, 2021 | tech tutorial

This post was written for the C# Advent Calendar. Follow them to receive the gift of coding insights, twice per day in December!

We want our code to be easy to use and test. When dealing with things that change, like the current time or randomized numbers, static helper methods are a simple way to get these variable values. However, testing these helpers and their many consumers can be a challenge, as you are trying to write tests for a moving target.

Let’s look at an example I created based on a dice game:

var roll = Dice.Roll();

While we can test this method; it can be difficult to write deterministic tests where it is used. Deterministic means we can rely on the result being the same every time it runs.

We expect Dice.Roll()to return a random distribution of 1, 2, 3, 4, 5, and 6 but how do we verify this functionality with tests. First, we look at the code behind Dice.Roll():

public class Dice
{
    private static readonly Random _random = new();
   
    public static int Roll()
    {
        return _random.Next(1, 7);
    }
}

We can see that the code is simple, but it isn’t easy to test. We would have to run a large number of calls to the “Roll” method and verify that we only get expected values with an even distribution. But even then, we wouldn’t know for sure that it would pass if we ran it just one more time. As it is important for the method to conform to these rules, we can use the following test:

public void ForANumberOfRollsDistributionShouldBeEven()
{
    const int totalRollCount = 100000;
    var expectedDiceRollCount = new Dictionary<int, double>
    {
        {1,0},
        {2,0},
        {3,0},
        {4,0},
        {5,0},
        {6,0}
    };
    var expectedAverage = 1.0 / expectedDiceRollCount.Count();
    const double withinRangeBuffer = .02;

    for(var i = 0; i < totalRollCount; ++i)
    {
        expectedDiceRollCount[Dice.Roll()]++;
    }

    foreach(var count in expectedDiceRollCount)
    {
        var average = count.Value / totalRollCount;
        average.ShouldBeGreaterThan(expectedAverage - withinRangeBuffer);
        average.ShouldBeLessThan(expectedAverage + withinRangeBuffer);
    }
}

There are a few things not to love about this test, but it would do the job of helping us know if something were seriously wrong. For example, if within 100,000 tries it rolled anything other than the expected values it would throw an exception. It would also fail if the underlying algorithm favored a particular side of the die. While helpful and capable of giving us a warm and fuzzy feeling, this test doesn’t give us any absolute assurances.

While this test does the job of putting some safeguards around our roll method, it doesn’t give us a way to test things that use this method. It is one thing to create this complex test for one method, but we wouldn’t want to write similar tests for all its dependencies. For example, we want to roll the dice inside of the Player’s Action method to determine some outcome.

public class Player
{
    public static string Action()
    {
        var roll = Dice.Roll();

        switch(roll)
        {
            case 1:
            case 2:
            case 3:
            case 4:
                return "Jump";
            case 5:
                return "High Five";
            default:
                return "Fall Down";
        }
    }
}

We could write some crazy tests to check the distribution of the returned value, but there should be a better way.

So what do we do? Do we migrate to an interface that we could mock out? Where we have two implementations of IDice, the real implementation, and a test-only implementation that we could use to pre-ordain the numbers being returned. That felt heavy-handed for this usage. Instead, we can reach for delegates.

public class Dice
{
    private static readonly Random _random = new();
    private static Func<int> _defaultRoll = () => _random.Next(1, 7);
    private static Func<int> _roll = _defaultRoll;
   
    public static int Roll()
    {
        return _roll();
    }
   
    public static void StubRoll(Func<int> newRoll) => _roll = newRoll;
}

By making the change above we can now write a deterministic test.

public void ShouldAlwaysRollAOne()
{
    Dice.StubRoll(() => 1);
    var roll = Dice.Roll();
    roll.ShouldBe(1);
}

This test shows us how to use it, but doesn’t provide a lot of value. Where this ability to stub the roll is helpful is when we want to test something consuming Dice.Roll() like our Player Action method.

public void PlayerShouldHighFiveWhenAFiveIsRolled()
{
    Dice.StubRoll(() => 5);
   
    var action = Player.Action();
    action.ShouldBe("High Five");
}

With this technique, we can easily make our tests deterministic. This strategy is also very useful for people who prefer static current date and time providers versus injecting a provider. This solution does assume single-threaded test runners, so if you require parallel test runs you will want to reach for an interface-based solution and give up your dreams of a static helper.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK