Wednesday, 24 August 2016

Test Your Live System using Live Service Tests

Traditionally there are three categories of functional tests.

  • Acceptance
  • Integration
  • Unit

This is often refereed to as the testing pyramid. Unit tests form the bulk of your suite, followed by a smaller subset of integration tests. Acceptance tests that cover features should be the tip of your testing strategy, few in number. These are great but there is a missing suite of tests - live service tests.

  • Live Service Tests
  • Acceptance
  • Integration
  • Unit

Live Service Tests.

The role of live service tests (LST) is to test the live system against the production environment and configuration. LST would be fewer in number than acceptance tests. Unlike acceptance tests, these should run constantly. Once a run has completed, kick of a new test run. This will require a dedicated machine or piece of infrastructure, but the value provided is well worth it.

LST should focus on journeys instead of functionality or features. In contrast to acceptance tests a user journey would be the core purpose of the system. For example, a LST suite to cover this blog would ensure the home page loads, an individual post can be loaded, and the archive is accessible. The rest of the site such as comments or social media interactions could be broken, but the core purpose of the system is working. Readers can read and browse the blog. If at any time the tests detect a failure in the core journey there is a big problem.

Why

LST offer the fastest feedback possible due to the fact they are constantly running. It is far more desirable to detect a problem before your users do. Naturally LST offer great protection after deploys. Deployment of new code is one of the times you are more likely to encounter issues, so a suite of tests triggered after a deployment is a natural fit. LST also protect against unplanned events. In my experience, exceeding disk space, DNS failure, third party issues and more have all be detected.

How To

Adding another suite of tests may sound like increased effort but the cost associated with LST is rather low. Sometimes acceptance tests can be run as LST, meaning no extra effort. Care must be taken here if the tests perform anything destructive or anything that interacts with third parties.

Alternatively writing LST is simpler than standard acceptance tests. The same tooling can be used such as Selenium, NUnit and so forth. As the tests themselves focus on journeys rather than functionality, the tests are often less complex to write.

The only difficulty LST introduce is the fact they are executing against the live system. Consider interactions with a third party. Using a real account on the real system may be problematic. One way to get around this issue is to embed test functionality within the system itself. For example you could set up a test account which the tests use. Instead of executing against the third party system, the dummy account is used. Likewise most third parties have test accounts which can be setup and used instead.

LST are a nice compliant to a diagnostic dashboard. If your dash is reporting no issues, and your tests are green, you can be confident the system is operating in a good enough state.

Lessons

  • Functional tests are not enough.
  • Use live service tests to test the real production system.
  • Run live service tests constantly for the earliest feedback possible.
  • Alter production code to introduce test functionality.
  • Make use of test accounts and anything else that third parties may offer.

Thursday, 4 August 2016

Why You Should Do Code Katas

Code katas are simple exercises that are meant to be repeated. They are great for learning a new language or tool. The goal is to learn something, not to complete them. In fact, if you don't finish a kata that is perfectly normal as long as you take something away. The beauty of code katas is they are small enough to be repeated and possibly completed within small time slots.

Takeaways

Beginner or expert there will be something you can learn or take away. Beginners should take away language skills, tooling tricks and more. This is furthered when pair programming.

Experts should be inclined to try new solutions rather than rinse repeat. Instead of solving the problem in the manner you do 9-5, try something completely new. Solving a kata ten times the same way is a waste. Instead solve it once, ten different ways. In short, experiment in code katas, not your production code.

Going Further

Sadly a small minority seem such simple exercises as below them. No one is above doing a kata. Constant improvement or training is essential regardless of level. Katas alone are not enough to support personal development. Experience, reading and practical development are all still required. However when code katas are combined they are an excellent aid.

Sunday, 31 July 2016

I Need to Stop Misusing Divs

I a certainly not a skilled or expert front end developer. While I'm more than capable of creating pages I lack any design magic to make them look half decent. Despite this one area where improvement can be made is in my markup itself.

Over the past few months I've spent most of my time getting to grips with recent additions and changes in the HTML5 and CSS3 space. During this one area stood out, my misuse of the division element or <div>.

Before the addition of the newer elements pages nested with div after div was normal. However this is no longer the case. From this point onwards I will be ensuring that every time I introduce a div element I question whether a more appropriate element should be used.

The HTML div element (or HTML Document Division Element) is the generic container for flow content, which does not inherently represent anything. It can be used to group elements for styling purposes (using the class or id attributes), or because they share attribute values, such as lang. It should be used only when no other semantic element (such as article or nav) is appropriate.

Semantic Meaning

Two huge side effects that are often overlooked when ignoring semantic markup is device compatibility with screen readers or other input methods and future proofing content.

Many people wrongly assume that all users are either keyboard/mouse or mobile (touch) users. By using semantic elements, users of other input methods get a much smoother experience. It is possible to jump to navigation or content without having to page through dozens of unrelated sections added only for stylistic purposes. Having used such devices first hand, the joy such simple changes make are outstanding.

Using semantic elements also helps future proof content. Screen scraping and other technologies can be simplified massively if content is correctly marked up. The thought that pages of content written now will still be used and accessible decades from now is incredible.

Lessons

The lesson here is an easy one. Every time you write a generic division element, stop and ask yourself is there an element with more semantic meaning that will do the same job?

Tuesday, 19 July 2016

Anaemic Domain Models and Code Smells

An anaemic domain model (ADM) is considered a code smell in many cases. An ADM is present when you have a entity representing your domain, but void of any behaviour. Any logic is separate and operated upon in isolation. Such domain models can be thought of as simple property bags, plain old language objects or DTO's.

Code Smells

With an ADM your behaviour ends up split across many domain services instead of being grouped with the data it operates upon.

As the domain and your understanding evolves, the problem an ADM introduces can get worse as more and more domain services are added.

A complex domain or one that evolves will end up paying a price. Converting to and from what looks like a domain model, only to perform domain logic separately is quite redundant. Why not ditch the domain model altogether? If you have a simple problem, a simple solution such as a transaction script may very well do the job.

Simple Problems - Simple Solutions

Sometimes you don't have a complex domain. Input, basic logic, then some form of CRUD is incredibly common. Due to this it is easy to see why anaemic models exist.

Rather than the cost associated with attempting to model the domain, choose easier solutions such as transaction scripts, table gateways or similar.

The big argument for anaemic domain models is following the SRP. Adding behaviour to domain models does not violate SRP. There is no reason why such additions cannot be formed from composition or delegation. Likewise the internal representation can be private. On the flip side domain services operating under the disguise of SRP lack cohesion, despite doing one thing.

The good news is that the ADM is very easy to extend and refactor at a later point. Moving to a richer domain model is not difficult, though the process may take time.

Refactoring from an ADM

Simply push behaviour onto entities, one method at a time. As you do this, services will begin to dissolve. All of this can be done when supported by a good suite of tests.

An equally simple step is to being introducing value types. Over time these will act as code magnets pulling any related behaviour towards them.

Lessons

  • In most cases an ADM is a code smell.
  • There may be easier solutions than a anaemic model that mimics your domain.
  • The ADM is not a good example of SRP.
  • Refactoring towards a rich domain model is easy and achievable at any stage.

Sunday, 17 July 2016

UI Composition Techniques for Services

When using services be it SOA, microservices or some other hybrid approach, at some point you will need to display an aggregation of data onto a UI. This simple task can actually involve some complexity and hidden pitfalls.

As an example, this blog could be powered by three independent services. A comment service, a post service and a archive service. Displaying this content on the page could involve a few different approaches both with pros and cons. Each vary in terms of benefits and complexity.

  • Service Composition
  • Server Side Composition
  • Backends For Frontends
  • Frontend UI Composition

Service Composition

Composition within independent services should be avoided at all costs. In these cases service A invokes service B which invokes service C, which has a dependency on A and so on. The problems such composition introduce defeats any benefits that a service based approach brings. In short composing data in this manner will lead to problems.


Server Side Composition

Invoke each service behind a single request and perform composition on the server. Return the results of the aggregation which can then be processed by the UI. In most cases a request will be a request over HTTP.

Pros
  • Single request to fetch all data.
  • Single server side place to change if UI requirements change.
  • Forms an anti corruption layer in front of the independent services. Client specific changes do not leak down into the service.
Cons
  • Coupling is moved to the server side. Harder/slower to change compared to HTML/JS/UI layer.
  • Handling failures must be considered more so than client side composition.
  • Timeouts or lack of responses must be considered using asynchronous techniques.
Where Does The UI Live?

Due to the nature of having to serve up data for the UI layer to consume, it makes sense to purely store the UI components within the host application. In this case the whole host application will use the same UI techniques.


Backends For Frontends (BFFs)

An alternative which builds upon server side composition is server side composition but performed individually based on the UI required. In other words, a set of server side applications for each host are created.

As UI clients can differ drastically a single server side composition technique may not be sufficient. Mobile devices may require a slimmed down version of data, while desktop dashboards may prefer large quantities. Additionally it is quite common to find certain clients asking for additional fields or requirements specific to their client implementation. In these cases BFFs make a great deal of sense.

Pros
  • Same pros as server side composition.
  • BFFs allow full control of server side composition tailored to the clients.
  • Specialized BFFs reduce change and prevent independent services handling UI specific edge cases.
Cons
  • Same cons as server side composition.
  • More moving parts, though BFFs should be owned by the client themselves for true autonomy.
Where Does The UI Live?

The same guidance as server side composition stands. Where the actual BFF lives depends on how it is used. If the mobile client is expected to have multiple implementations then a standalone service would be required. Alternatively if only a single mobile platform is targeted, then the service could live within the host application itself.


Frontend Composition

Invoke each service and display its data independently via the client. The host application will include the front end of each service by conforming to a common standard such as Javascript or other UI components. The use of IDs and client side identifiers will be required to ensure all services are linked in some manner.

Pros
  • Failure tolerant by default, if a single request fails the others carry on processing.
  • JS/UI layers make asynchronous calls easy, composition is a natural fit.
  • Weakest form of coupling is in the UI layer - easier and cheaper to change.
  • Flexible as you can create mash ups that would otherwise violate service boundaries.
Cons
  • Multiple requests to fetch content (four HTTP requests using the blog example above).
  • Aggregation may be complex. You may need to use more than just plain Javascript or face complex, coupled JS.
  • Depending on where the UI layer is stored, you may be coupled at the UI level due to the same framework or approaches needed across each service.
Where Does The UI Live?

Storing the UI within each service is ideal on paper, but in practice has some limitations. Each service can vary and iterate at its own pace which is fantastic as long as the integration of the service remains unchanged. Unfortunately the downside is that each service is actually independent in terms of the UI. This means that versioning the front end component becomes an issue. Likewise there is nothing stopping different services using different libraries or frameworks. If the UI component requires any server side additions this becomes even harder. For example a host application written in one language will be incompatible with other services if they differ. The final issue relates to storing UI components outside of host application frameworks. Many frameworks simply make this either impossible or very difficult to achieve.

Using the host application to store the UI components side steps the disadvantages and issues above. While you are at the mercy of the host application to integrate each component, this is not a show stopper. Chances are most applications have multiple views, so a single UI component would never be reusable across applications. Additionally the use of thin vertical slices should mean that even though the UI component is physically separate from the service, there is no reason why the two cannot be worked on in conjunction.

A final factor to consider is that there is no reason why a hybrid approach cannot be taken. Each service should store its own UI component, but also allow host applications the ability to integrate. This UI component can act as a form of dog fooding as well as providing an excellent development and test bed. It is far easier to work and test a small widget with an automated test than it is to exercise this within the context of a full blown application.


Lessons

  • Avoid the use of internal service composition - remote calls to third parties being the obvious exception to this rule.
  • There is no best approach overall, the chosen solution will vary based on application.
  • Server side composition has benefits, but client side UI composition opens up new possibilities.
  • Client side composition seems more complicated but in reality it is merely different, though does require some up front planning.
  • Default to using the services directly, only introducing a BFF if client requirements differ or client requirements are being forced upon the independent services.