Unit Tests vs Integration Tests - Battle continues - Knoldus Blogs

Unit Tests vs Integration Tests

Reading Time: 4 minutes

One would expect that after being in the industry for almost 20 years, a trivial comparison like this would be years behind me but I probably less correct. It seems that the right amount of Unit tests is still a burning issue which needs some attention. At Knoldus, we have the policy of maintaining a very high code coverage so that we can objectively validate the quality of the code through our now publicly available tool CodeSquad. 

We take pride in the fact that we are able to reduce the software lifecycle cost by 57% because of our focus on quality when the software is being developed. Almost, 80% of the cost of the software is when it is in the maintenance and enhancement phase post delivery.

Given the logic above, we place a very high premium on writing very very effective unit tests. However, a recent discussion with one of our esteemed clients made us rethink the strategy. Let us be clear though that we do not believe that we should not be writing unit tests but the distribution between Unit and Integration tests, the scenarios in which we can skip unit tests does need a review. Following is an attempt to do that,

Let us start with the advantages of the Unit tests, as we have known them through ages

TDD and Unit tests

1. Allow us to do a better design. (This is the one in which I believe the most) If the software is easily testable, it is modulated correctly, has the right structure and follows SRP to allow for separation of concerns
2. Allow us to make big changes in the code easily since we know that we have a test harness to fall back on
3. Provide lightning visual feedback and are the fastest feedback cycles
4. Help with providing documentation and allow for easier code re-use
5. Help with coding constipation. When faced with a large and daunting piece of work ahead writing the tests will get you moving quickly.
6. Boost developer productivity and confidence 

Now, let us look at scenarios in which Unit tests might not be “as” useful

Unit test smells

1. Scenarios wherein the code is obvious in the first glance. There isn’t a cyclomatic complexity to talk about. Example if the code is picking up the current RAM usage and passing it to the logging service, then there is less benefit of testing it
2. Scenarios wherein all the work which a piece of code is doing is to add coordinating logic. In these scenarios, there are a lot of units involved and the testing involves quite a few mocked objects which might not add a lot of value. All that the piece of code is doing is delegating the flow or coordinating with various units without any algorithmic logic.
3. Tests which are written just to improve the code coverage but your gut would tell that they do not matter. These would be tests which would check whether a particular log statement is published or not. These are very low-level tests and would be very very brittle.

Now, let us look at scenarios in which Unit tests are absolutely mandatory

1. Scenarios which involve algorithmic logic – Typically this means self-contained algorithms for business rules or for things like sorting or parsing data. This cost-benefit argument goes strongly in favour of unit testing this code, because it’s cheap to do and highly beneficial.
2. Complex code with many dependencies – This code is very expensive to write with unit tests, but too risky to write without.  – The best option is to refactor this code so that it falls in the first category.

Let us look at the quadrants of unit testing with this post

As you would notice, it is less useful to write unit tests for code which is either trivial or just does coordination.

So how does this compare to Integration tests and End to end tests?

In his blog Martin Fowler described the following pyramid

As you would notice, it is easy cost wise to write Unit tests which run faster as well as compared to End to End tests which are costly and run much slower. One of the parameters which is not brought about in the post explicitly is

The Level of confidence

With unit tests though we can ensure that the units are working fine, however, the level of confidence is lower as compared to an E2E test where we are assured that the system is working fine.

The sweet spot, however, is somewhere in between.

Integration tests strike a great balance on the trade-offs between confidence and speed/expense. This is why it’s advisable to some of your effort here as well.

Integration tests would run across the units, provide more level of confidence, reduce the need of mocking at all places and would run easily with your build cycles without causing undue stress on cost or confidence.

Conclusion

There are major advantages with Unit tests and the most important in my perception being the clarity of design. A well designed software can be tested easily. If you are finding it hard to test a piece of software then it needs a design re-look in most cases.

That said, going overboard with unit tests to improve code coverage is a Unit test smell. When you find yourself making endless mock objects and the logic falls under the category of Unit test smells as described above then we are getting diminishing returns.

A common myth is also that Unit tests slow down the development. Done right, unit tests speed up the development cycles.

Hence, do write unit tests for all the mandatory cases described above, for other scenarios make sure that the pieces for which you might want to skip the unit tests are covered by integration tests. The code coverage in CodeSquad should still be >85% when you run it with your CI environment