Cutting regression testing time by 77% with test automation

Cyclr is an embedded integration platform that enables software companies to connect their products with other applications and services.

Its platform allows product teams to build powerful automated workflows using a growing library of connectors, making integrations an important part of both the product and the customer experience.

Keeping testing at the pace of development

Cyclr was developing rapidly using an Agile/Scrum methodology, with continuous integration, automated builds and deployment supporting a fast-moving release cycle. Its testing approach needed to keep pace.

Changes and enhancements needed to be assessed quickly so developers could get fast feedback on user stories without slowing down the wider development process.

The growing number of connectors added another challenge. Each allowed Cyclr to integrate with different applications, creating a large number of combinations that would have been difficult and time-consuming to regression test manually.

Cyclr wanted to understand where automation could add the most value and build an approach that would integrate with its existing CI process while continuing to scale as the platform grew.

Building automation around Cyclr's platform

Our automation specialists started with on-site consultancy sessions to understand Cyclr's product, development process and existing testing challenges.

We carried out an automation feasibility assessment, looking at where automation could provide value and the potential return on investment. From this, we developed a high-level automation test design covering the proposed approach, framework, tools, scope and test suites.

Once the approach was agreed, we built a scalable test automation framework using Ruby and Watir-WebDriver. The framework used a data-driven approach, with JSON defining scenarios, test data and expected outcomes. Because connectors sit at the heart of Cyclr's platform and are continually added and updated, we designed the framework so new connectors and methods could be added with minimal additional coding.

Working closely with Cyclr's development team, we then integrated the automated regression tests into its existing CI process.

Tests could run daily, with failures automatically reported through the team's communication tools. This gave developers regular visibility of the current status of key areas of the platform and faster feedback when changes introduced a problem.

Building automation around Cyclr's platform

The impact on Cyclr's regression testing was significant.

Testing the connector functionality manually took an average of 6.5 hours. With the automated regression testing and follow-up analysis, the same testing could be completed in around 1.5 hours - a 77% reduction in regression testing time.

Daily, repeatable execution also meant Cyclr no longer had to wait for a manual regression cycle to understand the impact of changes. Its development team had much faster feedback on core functionality as part of the existing CI process.

Our relationship continued beyond the initial framework build, with our teams working together to maintain and extend the automation as Cyclr's platform evolved.