AMP & PWA – A New Test Approach

Manual functional, compatibility & technical checks for AMP & PWA will be closer to how a user interacts with content than can be achieved with most tools.

What are they and how are they different?

Accelerated Mobile Pages or AMP are pages that load almost instantly when viewed on a mobile web browser, even if the service is limited. AMP is intended to solve the problem of ‘first time’ page load, giving the user quicker or instant engagement with the content.  Progressive Web Apps or PWA load the ‘second time’ that the page appears and will load with enhanced features that make it appear and function more like a native mobile app.

From a technical point of view, AMP & PWA use stripped down HTML and a dedicated JavaScript library to reliably load pages very quickly.  When AMP is loaded it leaves certain page elements in cache that are then used by PWA to create a more enhanced experience for subsequent use, even when offline.

From a business and publishing point of view, AMP & PWA address the issues of user disengagement and the fall off rate from slow responding pages.  AMP pages also appear in a carousel as cards at the top of general web searches which means that for now the content is placed above other search results.

AMP & PWA are open-source initiatives from Google in consultation with publishers and other technology companies.

Who uses them?

At the moment, the main creators of AMP & PWA are publishers and those that are in competition with others to release new content.  The trend however is for broader usage of AMP & PWA especially PWA where native app features are enjoyed as part of a normal website and content offering with none of the separate maintenance that native apps require.

Testing

For developers of PWA there is an open-source tool called Lighthouse that will run an automated suite of 50 audit tests including performance and accessibility checks.  While these baseline checks are important, they don’t replace manual functional, compatibility and technical checks that can be conducted by an independent test team. From a business owner’s point of view, this type of testing will be much closer to how a user will interact with AMP and PWA. Terence Young, a project lead at Zoonou, says:

“Testers look at things as a user would, with an understanding gained from the thousands of previously tested sites and apps we have experience of.”

Testing on real devices, running a variety of OS versions, is the only way to really observe how AMP & PWA perform on mobile. Lighthouse will cut down the browser window to simulate a mobile screen size but it will not perform as a real device would.  Our early tests have picked up device specific issues especially around the rendering of pages and page elements overlapping in some cases.  Additionally, not all devices allow the transition from AMP to PWA in the way they should.  This requires a multi-environment test programme in order to find issues that are device specific.

Another challenge for testers is that there are often no obvious differences between an AMP or PWA version of a page – they may look identical.  In this instance developers can add a temporary icon or something visual to each version so that it’s possible to distinguish between them.

In terms of a test approach, either a scripted or exploratory method can be used.  At Zoonou we are developing specific scripts for AMP & PWA as well as a growing suite of non-functional tests.

Advertising challenges

Advertising presents another issue for testing AMP & PWA. AMP for Ads or A4A is an initiative designed to produce ads that work efficiently on AMP and PWA and do not slow down page loads.  However, AMP does still support older mobile ad versions meaning that clients need to know which ad formats they want AMP and PWA tested against.

AMP and PWA are still in early stages of adoption, but indications are that the uptake will be significant over the coming year.  An important part of the development of AMP and PWA is to get the quality of the output right,

“Part of what we do is to cover the things that clients and developers don’t think about”

said Terence, leaving them to consider the new technical and business challenges they face with these emerging technologies.

Test Essentials

  • Project Lead: Terence Young

For more information on our Device lab and the large range of devices we have available to test on, please see our Device Lab page. For more details on our compatibility testing service please contact us at info@zoonou.com and one of our team will get back to you. If you’d like to get in touch about anything else, please head over to our Contact page.

Keep up to date with the latest news from Zoonou

Get up-to-date information on our business and services by email. You can unsubscribe at anytime, and we will not share your data with any 3rd party marketing organisations. Read our Privacy Notice.

Blog Update Sign Up Form

  • This field is for validation purposes and should be left unchanged.