Other articles


  1. The QA Test Matrix

    Published: Mon 03 April 2017

    tags: testing

    Historically teams I've worked with have taken a few varying approaches when designing tests against acceptance criteria. One is to have the business define the feature, while the team help define the acceptance criteria. Ultimately the business gets the final say if they agree, and further acceptance criteria is either …

  2. Convention Based Tests

    Most projects have some form of convention. Examples would include:

    • Attributes/Properties for REST API's
    • Inheritance for third party base types
    • Assemblies/Packages for third party code that is loaded dynamically
    • Folder or namespace conventions
    • And many other forms of conventions

    In a few of these examples static analysis can …

  3. Test Your Live System using Live Service Tests

    Published: Mon 01 August 2016

    tags: testing

    Traditionally there are three categories of functional tests.

    • Acceptance
    • Integration
    • Unit

    This is often refereed to as the testing pyramid. Unit tests form the bulk of your suite, followed by a smaller subset of integration tests. Acceptance tests that cover features should be the tip of your testing strategy, few …

  4. Given When Then Scenarios vs Test Fixtures

    There are two common ways of writing automated tests which apply from unit to acceptance tests. These are typically known as test fixtures and Given-When-Then scenarios.

    Test Fixture

    • Traditional method of writing tests.
    • The common JUnit/NUnit approach. Other languages have very similar concepts.
    • Single test fixture with multiple tests …
  5. Pulling the Plug on Date Time Parsing

    Date/time logic is hard. Throw in time zones along with daylight saving and it's even harder. Recently a suite of tests that had happily been running for months started failing. There were no code changes and all the tests were somehow related to date/time ranges.

    Despite this the …

  6. Write Assertions First

    Writing a test as part of the TDD process is simple.

    1. Arrange
    2. Act
    3. Assert

    Many individuals recommend the process be reversed. Write assertions first. Then write the steps to perform the action. Followed by the required setup to complete the action.

    1. Arrange
    2. Act
    3. Assert

    Simplicity

    You will write just enough …

  7. Types of Test Doubles

    Mock is an overloaded term in software development. Sadly this leads to developers answering with "mock it" when a mock object may not be the right solution. Test Doubles are a more general term. I should try to use this naming more than I do at present - a goal I …

  8. Integration Tests

    It is well documented you need a balance between different categories of automated tests. The split is usually in the form.

    • 70% unit
    • 20% integration
    • 10% acceptance

    While unit tests make up the majority of tests, there is a limit to their effectiveness. As soon as you leave the system …

  9. Testing Private Code

    A common problem many people ask is - should you test private code? In short, you shouldn't. You should always test the public api of your code where possible. This is not always easy. Based on the context of the code in question there are a few options available.

    Don't Test …
  10. Randomly Generated Values in Tests

    The use of randomly generated test data seems like a good thing at first glance. Having worked with several teams that have used this concept I generally discourage the practice. Consider a simple method that joins together two strings. A test using random values may look like this.

    Problems

    Harder …
  11. Acceptance Testing need not use the Full Stack

    • Joined a team with thousands of unit tests (\~10k)
    • But bugs still got through our QA process
    • How could this be?
    • Team had a small number of full end to end live service tests
    • So my answer was to just increase the number of these
    • Surely this would solve our …
  12. Ratcheting

    Some tasks in software development are mundane such as formatting and code conventions. Where possible tooling should take away some of this pain, however sometimes you need a developer to take on a task that requires a great deal of time and/or effort to complete. Tooling will only get …

  13. Characterization Tests

    Having worked with some truly awful codebases a common problem tends to arise every now and then. You need to make a change within some legacy component that most likely has limited or no automated tests around. This can be a scary process.

    There are a few techniques you can …

  14. Flexible Selenium Tests via Page Objects

    Published: Thu 01 May 2014

    tags: testing

    A fast, automated suite of unit and integration tests are not enough. At some point you'll need to test your presentation logic. Ideally your domain/business/game logic is stubbed so all you'll need to do at this point is check that the presentation is complete. For example, does view …

  15. The Problem with Auto Updating Browsers

    Published: Fri 01 June 2012

    tags: testing

    At the time of writing the latest version of Firefox (version 13) has just been released. Bear in mind that a week ago I updated our Selenium bindings so that we could use Firefox 9+ for running our browser tests.

    The latest release is another great release for the Firefox …

  16. Recursively Building a Web Service using the same Web Service

    Published: Thu 01 March 2012

    tags: testing

    Back during the later part of 2011 there was a common theme occurring in our retrospectives each week. How can we replicate our live environment as close as possible?

    We took steps to achieve this goal by creating a single machine image to ensure all our machines were configured correctly …

  17. Unit Testing C# attributes

    Published: Sun 01 May 2011

    tags: testing

    For a recent coding session I needed to handle an exception being thrown when some Json was incorrectly bound to a view model. With the framework we were using (ASP.NET MVC2) I was unable to handle the exception at the controller level, nor could I handle it at the …

  18. MBUnit to NUnit

    Over the last few weeks we've ported our tests from MBUnit to NUnit. This was done as after a quick spike it was seen that NUnit tests run almost fifty percent quicker. For example our common projects' test time went from around 40s to around 20s on average.

    This whole …

  19. Mock Roles not Types

    "if it feels wrong, it probably is" - numerous Codeweavers' developers

    The framework we use at Codeweavers is the excellent Moq, therefore when something is difficult to mock we are forced by the framework to write an adapter. We use an interface for testing, then create a concrete type which simply …