Test Automation. What. How. Why. When.

Automation – replacing the human element with technology – is a part of modern life. As a function of efficiency, automating tasks impacts our everyday lives, usually in ways we barely notice. Your email client has sorted the annoying spam emails into your junk folder and Alexa will remind you to feed the cat after work. And why wouldn’t you automate these tedious tasks? The advantages are clear. More time to work on things you consider important.

Test automation applies a similar ethos to web development. Most websites will see many iterations with updates that will have the potential to impact on other parts of the website. Before flipping the switch and going live with a new iteration, you want a level of confidence that only testing can provide. Conventionally, manual testing is the main option available, however it can be a time-consuming use of valuable human resources, especially on larger projects.

Automated testing can help you to:

• Execute more tests in less time
• Run multiple complex data driven tests that you simply wouldn’t be able to do manually
• Carry out high level checks of a whole system
• In depth testing of specific features
• Or potentially full tests of a whole application. 

What is Test Automation?

In software testing, automated testing is the use of software to execute test cases or other test activities, comparing actual results against expected outcomes. An automation test framework can be customised to your specific needs and can be built to integrate with various management and issue tracking systems such as Jira, Azure DevOps or even Slack. Reporting can feed directly into these systems and provide you with rapid feedback on how development updates have affected your system as a whole. 

The Test Automation Process

The automation process typically begins when you have a product, like a website or mobile application, that has known end-to-end user journeys – for example, “the user can create a blog post” or “the user can purchase an item”. The product should be accessible for testing and any areas targeted for automation should have been manually tested prior to creating automated test scripts. Automation should not be carried out against an unstable or untested system. 

Once this has been completed the process follows 4 key steps:

Review & Baselining

The website or application is reviewed by an automation specialist to see what parts of your product would benefit from automation and what would not, gaining a baseline knowledge of the product and its current state. 

Scope

The report is fed back to the client with a suggested list of tests identified for automation. 

Client Review

A feedback and review session is held collaboratively with the client, using a combination of test expertise and client business needs to identify a final set of test cases.

Framework Set-Up & Scripting

The automation framework and test cases are created with the features agreed.

This process can be iterative. As you add and remove features during your product lifecycle, the automation can scale to fit your needs and updates.

Many automation systems are often used as a rolling “health check”,  running many times a week or a day to report any changes that the system has undergone or any faults that may develop as a result of extended use. It’s common for automation to be integrated into a continuous integration (CI) development process, where the automated test suites are triggered as new code is deployed. 

What are the Benefits?

Increased Test Efficiency

Although the initial setup can take longer, automated testing typically takes significantly less time than manual testing. Often, an automated test suite is available to you and your team 24/7 and can be run multiple times a day, week or month, with little excess processing requirements. 

Faster Feedback & Bug Detection

Automation can provide faster feedback and potentially identify bugs earlier in the development phase. The earlier the bugs are identified, the easier and more cost effective they are to fix. 

Quicker Product Release

Test automation can help to produce quality products, consistently, and in less time. With regular execution of test cases, products can be launched quicker, supporting rapid release cycles on agile projects. 

What are the Disadvantages?

Test automation must be handled and viewed as accurately as possible. It can provide a sense of security beyond its means and therefore understanding the limitations of automation is also important.

Automation is Rarely Exhaustive

Automated testing cannot test every possible scenario or error in a system. Coding for each possible negative path would not be viable in any non-trivial scenario. Automated testing is often utilised to focus on “happy paths” through a system to ensure that the functionality works as intended, but may not target edge cases or negative scenarios.

Changes to Functionality

Changes or new functionality added to your system will likely require further development of the automation. The automation cannot adapt to the changes in your system’s behaviour without additional baselining exercises of the new functionality and integration of new code into the automation suites. This will require additional resources to complete the necessary updates.

It Does What it’s Told

It is a common misconception that automation may provide a greater insight into the system under test than what is actually being provided. A passing test result should be viewed as a test that has passed for only the situation in which the test has been run, the result cannot be used to predict or forecast future results.

When to automate testing?

There are a few key points that are important to consider when looking into the creation of an automation suite. 

A checklist that would help guide your decision as to whether automation is right for your product might include:

• I have monotonous tasks with known,  expected outcomes that are slowing my workflow
• The health of the system needs to be checked more often
• I need to know of a fault in my system as soon as possible
• Manual testing currently takes us too much time to complete.

When not to automate testing?

There are instances where automation may not benefit your product. These vary on a case by case basis but from a high level there are some common factors that often combine to make automation less effective. 

Rapidly Changing Requirement

Automation is best at testing a known set of values against a known set of requirements. If these values and requirements are often in flux, the additional development time and cost for producing matching automation suites can be prohibitive.

Unstable Platforms

Automation relies on a platform that is “Known Good”, i.e. it has been manually tested and is considered stable and functionality sound. Results from automation testing may not be reliable for an unstable system, and the amount of maintenance and updates required may outweigh the benefits of having an automated test pack.   

Automation, Manual or a Blend of Both?

There is a time and place for both types of testing. Automated testing will help you to save time and resource over a longer period, and can help you achieve more regular test runs with less demand on human resource. Manual testing is still required for all software products and will ensure you cover usability, exploratory testing and edge cases or negative scenarios that automation simply won’t do. When considering which method to use, the tools, their cost and the benefits they provide all need to be considered. 

Test automation is a very powerful tool, that (when deployed correctly) can increase efficiency, rapidly provide feedback and cut overall operating costs.  If you would like to find out more about test automation, check out our Automation Service page or get in touch with one of our Client Partners or Technical Analysts by heading over to our Contact page.

Share this post