Automated Testing

  • Nick
  • 19 March 2022
Automated Testing
Automated Testing
Best Practices

Automated testing is the practice of writing code to test your code, and then run those tests in an automated fashion. So, with automated testing, your source code consist of application code, which is called production code and test code. Here is an example:

        public float CalculateTax(int input)
            if(x) return ...;
            if(y) return ...;
            return ...;

It's a basic calculated function that takes an input and depending on some conditions it returns different values. If you want to test this functions manually, you have to:

  1. Launch your application in the browser.
  2. Perhaps you have to Login.
  3. Navigate to get to the page where this function is used.
  4. Fill out a form.
  5. Submit it.
  6. Verify the result of this function on the screen.
  7. Repeat all the steps each time using different values in your form.

As you can see this is very time consuming. This workflow to test this function may take several minutes every time. To make matters worse, this is not the only function in you application. In a real application you have tens or hundreds of functions like this. As your application grows, in size and complexity, the time required to manually test all the different bits and pieces increases exponentially. So, that's why we use automated testing.

With automated testing, you write code and directly call this function with different inputs and verify that this function returns the right output.

        var result = CalculateTax(1);
        Verify(result == 1.5f)

Now, you can re-run these tests every time you change your code, every time you commit your code to a repository and before deploying your application. With this approach, you can test all the execution paths in this function in less than a second! you can write several hundred or thousands of automated tests for various parts of your application, and run them all in just a few seconds.

Benefits of Automated Testing

There are several benefits of automated testing:

  1. Test your code on a frequent basis in less time.
  2. Catch the bugs before deploying. This is extremely important because it allows you to deploy your application with more confidence.
  3. Refactor with confidence. Refactoring means changing the structure of your code, without changing its behavior. When you do not have automated tests you have to manually test every part of the application each time your refactor your code. This is very painfull because first of all, it's time consuming, and second as your application grows, you may forget about the parts that need to be tested! With automated tests, every time you refactor your code, you run your tests and make sure you didn't break anything that used to previously work.
  4. Focus more on the quality of the methods your are writing. You make sure that every method works with different inputs under varying circumstances.
  5. Tests serve as documentation

Types of Tests

In the book, Succeeding With Agile, Mike Cohn describes a concept he referred to as the test automation pyramid:

The test automation pyramid identifies four types of tests:

  1. Unit Tests. Tests a unit of an application without its external dependencies such as files, message queues, web services and so on. This unit tests exercise your code without any external dependencies, they are cheap to write and they execute fast. So, you can run hundreds of them in just a few seconds, and this way you can verify that each building block in your application is working as expected. However since you're not testing these classes or components with their external dependencies, you can't get a lot of confidence in the reliability of your application. So, that's when integration tests come to rescue.
  2. Integration Tests: Tests the application with its external dependencies. So it tests the integration of your application code with these concrete dependencies like files, databases and so on. These tests, take longer to execute because they often involve reading or writing to a database, but they give us more confidence the health of our application.
  3. UI Tests: Tests the functionality of the full application from the user interface down to the database. These tests give you the greatest confidence but they have 2 big problems. First is that they are ver slow, because they require launching the application and testing it through the UI. Second problem is that they are very brittle, because a small enhancement to the application or a small change in the user-interface can easily break these tests.
  4. Manual Tests: Tests performed by a human that verify the functionality of the full application.

Now that we know the types of tests the question is what tests should we write in the application? Well, all of them. The test pyramid argues that most of your tests should be in the category of unit tests, because these are easy to write, and they execute quickly. But since they don't give you much confidence about the health of your application, you should have a bunch of integration tests that test the integration of your application code with its external dependencies. These tests provide many advantages of UI tests, but without the complexities of dealing with the user interface. And finally you should write very few end-end tests for the key functions of the application, but you should not test the edge cases with these UI tests. You only test the happy path, and leave the edge cases to unit tests.