Tuesday, 19 July 2016

Anaemic Domain Models and Code Smells

An anaemic domain model (ADM) is considered a code smell in many cases. An ADM is present when you have a entity representing your domain, but void of any behaviour. Any logic is separate and operated upon in isolation. Such domain models can be thought of as simple property bags, plain old language objects or DTO's.

Code Smells

With an ADM your behaviour ends up split across many domain services instead of being grouped with the data it operates upon.

As the domain and your understanding evolves, the problem an ADM introduces can get worse as more and more domain services are added.

A complex domain or one that evolves will end up paying a price. Converting to and from what looks like a domain model, only to perform domain logic separately is quite redundant. Why not ditch the domain model altogether? If you have a simple problem, a simple solution such as a transaction script may very well do the job.

Simple Problems - Simple Solutions

Sometimes you don't have a complex domain. Input, basic logic, then some form of CRUD is incredibly common. Due to this it is easy to see why anaemic models exist.

Rather than the cost associated with attempting to model the domain, choose easier solutions such as transaction scripts, table gateways or similar.

The big argument for anaemic domain models is following the SRP. Adding behaviour to domain models does not violate SRP. There is no reason why such additions cannot be formed from composition or delegation. Likewise the internal representation can be private. On the flip side domain services operating under the disguise of SRP lack cohesion, despite doing one thing.

The good news is that the ADM is very easy to extend and refactor at a later point. Moving to a richer domain model is not difficult, though the process may take time.

Refactoring from an ADM

Simply push behaviour onto entities, one method at a time. As you do this, services will begin to dissolve. All of this can be done when supported by a good suite of tests.

An equally simple step is to being introducing value types. Over time these will act as code magnets pulling any related behaviour towards them.

Lessons

  • In most cases an ADM is a code smell.
  • There may be easier solutions than a anaemic model that mimics your domain.
  • The ADM is not a good example of SRP.
  • Refactoring towards a rich domain model is easy and achievable at any stage.

Sunday, 17 July 2016

UI Composition Techniques for Services

When using services be it SOA, microservices or some other hybrid approach, at some point you will need to display an aggregation of data onto a UI. This simple task can actually involve some complexity and hidden pitfalls.

As an example, this blog could be powered by three independent services. A comment service, a post service and a archive service. Displaying this content on the page could involve a few different approaches both with pros and cons. Each vary in terms of benefits and complexity.

  • Service Composition
  • Server Side Composition
  • Backends For Frontends
  • Frontend UI Composition

Service Composition

Composition within independent services should be avoided at all costs. In these cases service A invokes service B which invokes service C, which has a dependency on A and so on. The problems such composition introduce defeats any benefits that a service based approach brings. In short composing data in this manner will lead to problems.


Server Side Composition

Invoke each service behind a single request and perform composition on the server. Return the results of the aggregation which can then be processed by the UI. In most cases a request will be a request over HTTP.

Pros
  • Single request to fetch all data.
  • Single server side place to change if UI requirements change.
  • Forms an anti corruption layer in front of the independent services. Client specific changes do not leak down into the service.
Cons
  • Coupling is moved to the server side. Harder/slower to change compared to HTML/JS/UI layer.
  • Handling failures must be considered more so than client side composition.
  • Timeouts or lack of responses must be considered using asynchronous techniques.
Where Does The UI Live?

Due to the nature of having to serve up data for the UI layer to consume, it makes sense to purely store the UI components within the host application. In this case the whole host application will use the same UI techniques.


Backends For Frontends (BFFs)

An alternative which builds upon server side composition is server side composition but performed individually based on the UI required. In other words, a set of server side applications for each host are created.

As UI clients can differ drastically a single server side composition technique may not be sufficient. Mobile devices may require a slimmed down version of data, while desktop dashboards may prefer large quantities. Additionally it is quite common to find certain clients asking for additional fields or requirements specific to their client implementation. In these cases BFFs make a great deal of sense.

Pros
  • Same pros as server side composition.
  • BFFs allow full control of server side composition tailored to the clients.
  • Specialized BFFs reduce change and prevent independent services handling UI specific edge cases.
Cons
  • Same cons as server side composition.
  • More moving parts, though BFFs should be owned by the client themselves for true autonomy.
Where Does The UI Live?

The same guidance as server side composition stands. Where the actual BFF lives depends on how it is used. If the mobile client is expected to have multiple implementations then a standalone service would be required. Alternatively if only a single mobile platform is targeted, then the service could live within the host application itself.


Frontend Composition

Invoke each service and display its data independently via the client. The host application will include the front end of each service by conforming to a common standard such as Javascript or other UI components. The use of IDs and client side identifiers will be required to ensure all services are linked in some manner.

Pros
  • Failure tolerant by default, if a single request fails the others carry on processing.
  • JS/UI layers make asynchronous calls easy, composition is a natural fit.
  • Weakest form of coupling is in the UI layer - easier and cheaper to change.
  • Flexible as you can create mash ups that would otherwise violate service boundaries.
Cons
  • Multiple requests to fetch content (four HTTP requests using the blog example above).
  • Aggregation may be complex. You may need to use more than just plain Javascript or face complex, coupled JS.
  • Depending on where the UI layer is stored, you may be coupled at the UI level due to the same framework or approaches needed across each service.
Where Does The UI Live?

Storing the UI within each service is ideal on paper, but in practice has some limitations. Each service can vary and iterate at its own pace which is fantastic as long as the integration of the service remains unchanged. Unfortunately the downside is that each service is actually independent in terms of the UI. This means that versioning the front end component becomes an issue. Likewise there is nothing stopping different services using different libraries or frameworks. If the UI component requires any server side additions this becomes even harder. For example a host application written in one language will be incompatible with other services if they differ. The final issue relates to storing UI components outside of host application frameworks. Many frameworks simply make this either impossible or very difficult to achieve.

Using the host application to store the UI components side steps the disadvantages and issues above. While you are at the mercy of the host application to integrate each component, this is not a show stopper. Chances are most applications have multiple views, so a single UI component would never be reusable across applications. Additionally the use of thin vertical slices should mean that even though the UI component is physically separate from the service, there is no reason why the two cannot be worked on in conjunction.

A final factor to consider is that there is no reason why a hybrid approach cannot be taken. Each service should store its own UI component, but also allow host applications the ability to integrate. This UI component can act as a form of dog fooding as well as providing an excellent development and test bed. It is far easier to work and test a small widget with an automated test than it is to exercise this within the context of a full blown application.


Lessons

  • Avoid the use of internal service composition - remote calls to third parties being the obvious exception to this rule.
  • There is no best approach overall, the chosen solution will vary based on application.
  • Server side composition has benefits, but client side UI composition opens up new possibilities.
  • Client side composition seems more complicated but in reality it is merely different, though does require some up front planning.
  • Default to using the services directly, only introducing a BFF if client requirements differ or client requirements are being forced upon the independent services.

Sunday, 10 July 2016

Notes on Building and Deploying Software

Builds and Deploys

Ideally a build and deploy should be a single step, included within the check out of the repository. Additionally the build should include and install pre-requisites if missing. You can safely assume the target OS is at least configured, but any missing packages should be installed as needed.

The core steps regardless of platform or technology follow a common pattern. A number of these steps can be performed asynchronously. For example, it is possible to run a suite of tests in parallel, rather than individually. This can save massive amounts of time. As a first approach perform all steps synchronously, only adjusting once stable.

  • Compile/Lint - Sync
  • Run Unit Tests - Async
  • Deploy - Async
    • Warm up (cache hits, web server)
    • Data (run migrations)
    • Configuration
    • Host Environment
  • Run Integration/Acceptance Tests - Async

This whole process should be executed regardless of environment. A local developer machine only differs to production in terms of topology. If you cannot execute the exact same process that you are performing upon commit, you can never be sure the commit will work. CI tooling should act as nothing more than an aggregation of stats and artefacts that are generated by invoking the build/deploy script.

Components

A working software application can be decomposed into four components.

  • Executable code - your application.
  • Configuration - connection strings or credentials.
  • Host environment - target deployment machine.
  • Data - required data for use, or persistent stores such as databases.

If any of these change, trigger the feedback process as they can all change the behaviour of the application.

Automation

  • Aim to automate everything, but do it gradually over time. Aim for the bottlenecks to begin with if the task of automating the deployment pipeline is daunting.
  • Keep everything in source control. Never allow your CI tool to control your pipeline.
  • Don't check passwords or credentials into source control.

Speed is Key

  • Too long and people wont commit often.
  • Under thirty seconds is ideal. Faster is better.
  • Fail the build if the time exceeds a set threshold. You can use ratcheting to reduce this over time.
  • Monitor tests regularly and try to speed the slowest ones up.

Build and Deploy Etiquette

  • Don't check in on a broken build, unless you are fixing it.
  • Perform an update prior to commit and run the whole build/deploy process.
  • Never go home on a broken build, and ideally don't check in just before leaving.
  • Never deploy on Friday's or whenever there will be few developers around on hand, such as holiday periods.
  • Always be ready to revert to the previous version. If the fix takes longer than just simply reverting the changes, don't try and fix the build. Just revert.
  • Fail the build for architectural breaches such as a module referencing a module it shouldn't.
  • Fail the build for warnings or code style breaches. Warnings can hide serious problems.

Infrastructure

  • Lock down production environments.
  • Treat test environments equally, these should also be locked down from manual changes.
  • If you cannot create your infrastructure from scratch via an automated process to begin with, implement access control so that any changes must be requested and logged.
  • Any changes should be back filled and tested. Use some form of audit or access control to manage these steps.

Data

  • Use migrations to manage data. Have a version table that stores the schema version. Use migrations to roll forwards or roll back incrementally.
  • Each migration should have an "up" and a "down" step. For example, up adds a table, down removes it. Run these both before committing to make sure they work.
  • Migrations allow you to decouple the deployment of your application from the deployment of your database.
  • The use of views or stored procedures can add a layer of abstraction between applications and databases. This allows changes to the underlying DB to have a smaller, limited effect.

Saturday, 25 June 2016

Ten Lessons from Rewriting Software

  1. It Will Take A Lot Longer Than Estimated

    • Its navie to actually think this but if a system has been in production for say five years, expecting to reproduce it in five weeks is not possible. You may be able to get 80% of the core functionality done, but the remaining 20% that was added to, iterated and stabilized over the remaining five years is what will destroy any form of schedule.
    • If your estimate exceeds three months, you need to reasses what you are doing by breaking down the work, or changing plan. The bigger the estimate, the bigger the risk.
  2. Deploy Incrementally Via CI

    • If you aren't deploying to a live environment as soon as possible, any future releases are destined to be failures, troublesome or just plain difficult.
    • Soft releases and feature toggles should be used to aid constant releases.
  3. Morale Will Drop The Longer It Goes On

    • Probably the biggest and most surprising realization is the drop in personal and team morale.
    • If you miss a "deadline" or keep failing to ship, then morale will tank.
    • While software is never complete, a rewrite has a definitive target. If this target continues to move, team morale will move too.
  4. Users Will Probably Hate It Anyway

    • Predominantly the UI, but your users will complain about change.
    • Big sweeping changes often receive the most hate. A website I frequent had a major change both in visuals and the underlying technology used. While there was warning, you were left to your own to figure out where features were. This caused a great deal of frustration and negative feedback.
    • Small, incremental changes allow your users to keep pace.
    • Alternatively some tutorial or hint system can help reduce user pain.
  5. Do What The Legacy System Does

    • As many of the original developers will likely have moved on, no one is really sure what the legacy system does.
    • Even with the source code available, it is likely going to be hard to figure out the intent, afterall that's one of the reasons for the rewrite.
    • If you are not careful you will end up simply reimplementing the same legacy in a new language or framework. Always weigh up preserving existing behaviour versus introducing technical debt.
  6. Be Cheap And Quick - Use Stubs

    • When implementing the new system, don't build a thing. At least at first.
    • Use stubs to build the simplest, dumbest thing you can to get feedback.
    • Without fully integrating the system in an end to end manner you'll end up throwing away a great deal of code.
  7. Feedback, Feedback, Feedback

    • Early and fast feedback is essential.
    • With a working end to end system gather as much as you can from any stakeholders.
    • Chances are as you begin you'll naturally incur some additions, removals or modifications.
    • Waiting months or longer for feedback is a guaranteed path to failure.
  8. Thin Vertical Slices Over Fat Technology Splits

    • Avoid the temptation to have a UI team, a backend team and a data team and so on.
    • Splitting at technology boundaries leads to systems that do not integrate well, or worse fail to handle the required use cases.
    • Your first iteration should consist of all parts of the technology stack, in the thinnest manner possible. Combine this with early feedback and the fast development speed of stubs.
  9. Strangle Existing Legacy Code

    • When rewriting in increments or by logical sections the technique of strangulation is useful.
    • Instead of releasing the new code as a standalone piece, integrate the new code into the existing legacy code base.
    • This may be tricky at first however over time the legacy system will form nothing but an empty shell that integrates with the new system.
    • The beauty of this approach is early feedback, and a guarantee that the new system behaves as intended.
    • The final step would be to replace the legacy shell with the new modern interface or frontend.
  10. Refactor Where Possible

    • Deciding to refactor or rewrite is never easy. Refactoring should be the default approach in many cases.
    • Old languages or unsupported frameworks are good reasons to adopt a rewrite, but this varies case by case.
    • If business agility is suffering such rewrites can be beneficial when using some of the techniques above.

Tuesday, 14 June 2016

DDD - Bounded Contexts

A single domain can grow large when applying Domain Driven Design. It can become very hard to contain a single model when using ubiquitous language to model the domain. Classic examples prevalent in many domains would be Customer or User models. A bounded context allows you to break down a large domain into smaller, independent contexts.

In different contexts a customer may be something completely different, depending on who you ask and how you use the model. For example, take three bounded contexts within a typical domain that allows customer administration, customer notifications and general reporting.

Example

Notification Context

A customer is their account id, social media accounts, email and any marketing preferences. Anything that would be required to uniquely identify a customer, and send a notification.

    + Id
    + Email
    + Marketing Preferences
    + Social
Reporting Context

When reporting customers are nothing more than statistics. A unique customer ID is more than enough just for aggregation and statistic collection.

    + Id
Account Context

Allowing the customer to administer their account would require anything personally related to the customer to be modelled.

    + Id
    + First Name
    + Last Name
    + Address
    + Email

Despite the common elements such as Id and email, the other elements are specific to the context in which the customer is used. One of the biggest mistakes I've made by ignoring a bounded context is to see a common model and try to apply this everywhere. This leads to less code, but increases coupling. A single small change in one context can cause a rippling effect. In fact the best solution is to have a customer model per context.

The result of this approach is you will end up with at least three models using the example above. While structural duplication increases, coupling decreases. Each context can change and evolve at its own pace. This is a good thing. No business logic here is being duplicated, only the model. As each context operates in its own speciality, there should never be a case where this is problematic.

Lessons

  • Structural duplication outside of bounded context is not a bad thing.
  • Resist the urge to use a base class for common attributes. This is especially true if you use an ORM or anything that will couple you further when these models are used.
  • Ending up with multiple models per bounded context is likely going to happen, embrace it.