It’s that time of year again! Apple and Google are launching the latest versions of their mobile operating systems (OS). Android 10 officially launched for Google Pixel devices on the 3rd of September 2019, with support for more devices to follow in the coming months. iOS13 is expected for formal release around mid-September. New hardware will likely coincide with the operating system release.
So what does this mean for your mobile testing strategy, and what are the key concerns?
Don’t get caught out! Test early!
A major UK high street bank has already been caught out by not being prepared for Android 10 support. Having identified their mobile banking app didn’t work on the latest Android version post-launch, they issued an apology and advisory to customers that a fix is being worked on. For financial institutions, technology issues can seriously undermine user confidence, especially in a space that is now seeing competition from new fintech companies that have the latest technology at the heart of their service. However, it’s not just financial institutions that are at risk, any business with a digital product available on mobile devices runs the risk of being caught out!
Before formal public release, both Apple and Google offer Beta versions of their new operating systems. This allows testing departments to facilitate early compatibility checks.
At Zoonou, we have an extensive test lab with over 300 devices, that is continuously growing as new devices are released. Once Beta versions become available for a new major release, we install this on a range of compatible handsets and begin carrying out early assessments for client projects that are already live. For example, an initial assessment may include the execution of key user journey tests to ensure that the primary app functionality is maintained when the application or website is viewed on the new operating system and browser.
This process proactively identifies issues. By testing before the operating system is available to consumers, it avoids the potential for negative reviews together with the embarrassment and disruption that can come with not being fully prepared for a new operating system release.
When should you update your test matrix?
For all digital products, it is important to maintain a suitable test matrix that is based on browsers, operating systems, and devices that are representative of the most significant proportion of the product’s user base.
Businesses can often get caught out by focusing only on what their users have been using in previous months, rather than looking ahead to what they will be running in the future.
iOS testing considerations
The evidence from previous years indicates that iOS users typically upgrade to the latest OS as soon as it becomes available. Apple Developer Stats around the 2018 release showed that over 50% of users had migrated to the new OS within 4-6 weeks after the launch date.
The reason for the fast uptake for Apple users is because all currently supported Apple mobile devices will be compatible with the new operating system and eligible to receive the update.
Each year some legacy Apple devices may be removed from the supported list and these devices then begin to languish, stuck on an older version of the operating system that can no longer be updated beyond a certain point. When looking at analytics data this can sometimes explain why there may appear to be a stubborn group of users who are running an earlier operating system. They simply cannot upgrade because support for their device has ceased.
For iOS users, it is important to accommodate testing of a wider selection of devices ahead of, and post-launch of a new major operating system release because the uptake will usually be swift.
Android testing considerations
It is a different story for Android. Not all Android devices will be eligible for the latest update, which can typically be reserved for the latest and greatest flagship handsets. For Android 10 this includes Google Pixel devices on immediate release, and other manufacturers to follow over the coming weeks and months.
Since the release to specific devices is handset manufacturer dependent, the rollout to the latest Android release is typically slower compared with iOS updates.
There is significant fragmentation across Android versions due to older devices being locked to only receive updates to a certain version of the operating system – this is largely to control maintenance overhead for legacy devices for handset manufacturers.
Android 10 support should certainly be a focus for applications and websites that anticipate a user base that chases the latest tech. We suggest regularly monitoring analytics data to keep on top of changes in OS, device and browser version popularity as this information should feed into updates to your test matrix.
What if you work with an external agency or development partner?
When commissioning a new project, or as a consideration for business as usual (BAU) product maintenance, it is important to be able to have a regular dialogue with your partner on the browser, operating system, and devices that will be supported. This may require a change request, or update to your statement of work (SoW).
Here are our top 5 takeaways on focusing your test efforts around new mobile operating system releases:
#1: Be proactive! Test early with the beta versions.
#2: Keep the lab up to date! Ensure you have the necessary devices available to test.
#3: Be mindful of your users! Think about what devices, operating systems, and browsers they are using now, and plan for what they may be using in the months to come.
#4: Monitor industry trends! Keep an eye on the Apple and Android developer stats which offer some very useful insights into OS usage.
#5: Work with flexible partners! Your developer and test partners should be willing and able to support the ongoing changes in technology used by your customers and advise you accordingly.
Related blog posts
Test Automation. What. How. Why. When.
Software Testing Glossary. A-Z Guide.
Android Pie Gains New Accessibility Features.
Share this post
Published by Garry Hardy
on the 9th September 2019
Commercial Director at Zoonou