Tuesday, 28 July 2015

You Still Need Manual Tests

This blog has numerous examples of why unit, integration and contract testing is essential. However you still need manual tests. It is foolish to believe that all testing can be covered by automated tests despite the bias in this area.

Why?

  • Manual tests can catch anything you may have missed at an automation level.
  • Manual tests can be unique. Use exploratory testing to try different scenarios.
  • Manual tests that fail become automated tests, so they can never regress.
  • Manual tests exercise the full stack. Many areas such as DI conventions, logging, and other framework related configuration are better suited to manual verification.
  • UI changes require visual verification - automation is near impossible here.

More Than Just Functionality

Over the years, manual testing has caught numerous bugs, issues or things I've just plain missed. When you are deep in a problem it can be hard to see the wood for the tress. A second party manually testing provides an unbiased check of your code for a second time.

The key with manual tests is to ensure any issue is converted into an automated test. This offsets the fact that manual testing is expensive both in terms of time and cost. By doing so any regressions will be prevented.

QA includes more than functional testing. Security, performance and usability to name a few are equally important. Do not avoid the manual test step. Automated tests are only as good as the tests themselves. Embrace manual and automated testing for the best of both worlds.

Tuesday, 21 July 2015

The Benefits of Contract Testing

I previously claimed that you need some integrated tests but as few as possible. There are huge benefits to this approach, but there is a problem. How do you stop your test doubles falling out of line with the real implementations? The answer is to use Contract Tests.

Steps

  1. Create a base test fixture, this is where your tests live. All assertions belong here.
  2. Subclass this base class with each implementation.
  3. Override each setup step to provide the implementation that is to be tested.
  4. All tests should pass for each instance.

Example

In this example there is a SQL repository and an in memory repository. It is not possible to change either in any manner that causes them to behave differently. We can safely use the in memory repository for tests, with confidence that the test double matches the contract of the real implementation.

The test double implementations can be executed on every test run. While real implementations can be relegated to execution prior to commit or during continuous integration. This trade off allows for fast feedback cycles while ensuring all tests are run against production like implementations.

References

Tuesday, 14 July 2015

Integration Tests

It is well documented you need a balance between different categories of automated tests. The split is usually in the form.

  • 70% unit
  • 20% integration
  • 10% acceptance

While unit tests make up the majority of tests, there is a limit to their effectiveness. As soon as you leave the system boundary you need integration tests. Examples of when integration tests are required is code that interacts with databases, web services or the file system.

These integration tests should not test logic, this is a mistake. They will become brittle and slow to execute otherwise. Instead of checking domain logic, test at a lower level. Go as low as you can without leaking implementation details of the subject under test. By going as low as possible you will radically reduce the number of integration tests required. Less tests means easier maintenance. Less tests also means faster tests.

Example

Assuming a SQL database, invoke the repository and test as lightly as possible. Do not indirectly test this repository by invoking the code higher levels in the stack. Avoid concerning yourself with what is happening behind the scenes. Simply test that you can insert a record, and retrieve the newly inserted record. Any other code that is involved at higher levels can suffice at a unit level.

Assertions should be loose enough to verify that the code is working, but not asserting basic correctness. In other words prefer assertions that check for the presence of results, rather than what those results look like. If the value is of concern, convert into a fast, isolated unit test.

Integration Tests are a Scam

The term Integrated Tests is my preference given that integration tests are a scam. This slight change in terminology helps keep these tests focused. Rather than spiraling out of control, they are small in number and simply verify that "something is working". This is done by pushing all tests of logic to the unit level.

The key point here is that integration tests are required. Strongly resist the urge to write all tests at the integrated level. Likewise do not fall into the trap that thinking all tests must be done at the unit level. The key here is balance.

There is a fatal flaw with integration tests however. They can be wrong. Given tests at a unit level will stub out anything that is out of process, how do you stop such tests falling out of sync with the real implementation? This is where Contract Tests come into play.

Tuesday, 7 July 2015

Static Code

Static code is considered a bad thing by developers. This is especially true when working with legacy code. The use of static code is often seen as a smell and should not be used.

This is not as black and white as it first seems. Static code can be problematic when global state is involved. Not only is it hard to change, static code is very hard to test in an automated fashion. Bad examples of static code include persistence, third party services, and out of process calls. These examples should avoid static code where possible.

One guideline that served me very well in my early days of TDD was treating static code as a death to testability. Unfortunately some developers don't move on from this guideline and treat any use of static code as bad.

In fact static code can have a benefit. If a method within a class can be promoted to a public static method (PSM) it shows that the code is stateless. This allows the "extract class" refactoring to be performed. Without a PSM such refactoring is much more difficult. IDEs can automate this step and if in a dynamic language you can simply lean on the runtime to catch issues.

The steps to perform this refactor are easy. If at any stage this is not possible the method contains state.

  1. Make the method public.
  2. Make the method static.
  3. Move the public static method to the new class.
  4. Update usage of the previous calls.
  5. Optionally remove the static modifier and update previous call sites.

If the code cannot be promoted to a PSM then state exists. Increasingly the code I write leads itself to a functional paradigm despite not be written in a strictly functional language. Small, focused classes that tend to be immutable. The use of PSM makes transition to this style of code easy. There is no reason to avoid the use of static code as an intermediate step to get to this position.

Tuesday, 30 June 2015

DRY vs Coupling in Production Code

Duplication in tests can be a good thing. The same can be said for production code as well in some cases. No. I'm not going mad. I probably wouldn't believe this if you showed me this several years ago either. More experience has shown me that loose coupling is often more desirable than removing duplication. In other words, the more duplication you remove, the more coupling you introduce inadvertently.

Duplication of logic is bad and will always be the case. I am not debating this. You should only have one logical place for any domain logic. As always follow the DRY principle. However just because two pieces of code look the same, does not mean there is duplication.

Example

A system from my past had two places where an address was required for display and serialization. Billing and Delivery addresses.

My gut reaction was to introduce a common address model that could be used for serialization and display. After all this screams of duplication. However a billing address and delivery address are two conceptually different things despite appearing identical.

Given time the needs of the billing functionality may very well differ from the needs of the delivery domain. Duplication of models/contracts is weak duplication. There is no logic here.

In DDD each bounded context will have different needs. As it turned out the Billing Address began to have specific billing related functionality added such as "IsDefaultAddress" and "IsSameAsDelivery". At this point the two models are very different. This was a problem.

Sharing via a common library would have removed the total lines of code but increase the number of dependencies. The Address is now coupled to a single form meaning updates and new requirements are harder. Versioning and packaging are now a concern. Any updates would need to be coordinated across teams. Udi Dahan has warned about this previously in what is summarized as "Beware the Share".

Inheritance?

This example makes inheritance look like a good fit. While the use of inheritance when applied correctly is not a bad thing, this scenario is not appropriate. Inheritance is one of the strongest forms of coupling. Applying inheritance across a type that we don't own is risky for the reasons detailed previously. Now change is not only harder, it would potentially be a breaking change. How would we model a delivery address with multiple addresses? Why should both the billing and delivery domain use the same terminology for its fields? If we accept that both addresses are conceptually different despite looking identical at present, we can side step these issues.

What to Share?
  • Domain types should be shared. Using the previous example a PostalCode would make a good type to share. The functionality here is identical regardless of the type of address. PostalCode would likely have logic associated with the type which would not make sense to duplicate or implement in each sub system.
  • Shared functionality that must be consistent makes a good candidate also. Examples such as UI widgets including headers and footers.
  • Crossing cutting concerns such as logging, security and configuration can be shared when appropriate. A downside to this is you now force your consumers to take specific dependency versions which may or may not be acceptable.
Shared Kernel

DDD has the concept of a Shared Kernel. The dictionary definition of a kernel is "the central or most important part of something". Shared Kernel's make sense to share the common functionality previously. The name "common" is poorly thought out however. Most codebases will have a common or utility library but by there very nature these will grow into large components.

The reason for this growth is everything is common across applications. All applications need some sort of data access, so stick it in the common library. All applications need some sort of serialization mechanisms, so stick it in the common library. All applications need some sort of web technology, so stick it in the common library. You should be able to see where this is going.

Conclusion

As always when dealing with duplication apply the Rule of Three where appropriate. If you really must create a shared component, a small, concise library is better than a library that handles multiple concerns. This will allow consumers to adopt a "plug 'n play" approach with which components they require. Even then, try to fight removing duplication unless you can be really sure there is a good reason to increase coupling. That reuse you are striving for might not even come to fruition.