+44 (0) 203 811 8344
In this article, Mike O’Hara of The Realization Group asks Ash Gawthorp of The Test People, KPMG’s Daryl Elfield and smartTrade’s David Vincent to assess the testing challenges facing firms looking to deliver banking and financial services via mobile apps.
Your smartphone has more computing power than the combined super-computers used by NASA to send a rocket to the moon 50 years ago. Mobile devices such as smartphones and tablets are rendering PCs irrelevant for many. They provide a single gateway – via built-in functions and downloaded apps – to the services, tools and information we need to run our personal – and increasingly our professional – lives.
With so many of us increasingly relying on mobile devices to organise our lives, it is no surprise that banks and other financial services firms should be looking to mobiles as a channel to market with the potential to reconnect with a retail and wholesale customer base whose trust has been severely dented over the past decade. If banks can deliver services that are as quick, reliable, innovative and user-friendly as app-based retail and entertainment service providers, there is an opportunity in the retail space to re-establish themselves as trusted partners. In the wholesale markets, there is perhaps less ground to make up, but there is significant scope for competitive differentiation and market share growth from delivering greater convenience and richer functionality combined with appropriate controls and security.
There are good reasons why banks are not bleeding-edge adopters of new technologies, not least their responsibilities to customers and their highly-regulated status. Know-your-customer and anti-money laundering requirements have intensified over the past 15 years or so, for example, while data protection and related security laws are under constant review as the nature of criminal activity takes new forms in cyberspace.
These rules and responsibilities impose certain restraints for banks and their technology partners when it comes to delivering services via technology, whether in the office, at the branch or on the move. But to exploit new channels successfully, they also need to bear in mind another set of factors to ensure their mobile apps enhance the customer experience, not distract from it further. After all, they will not just be compared with other banks, but every other organisation that has an app-based presence on a customer’s phone.
“Mobile is all about the customer journey. It’s all about making the user experience as satisfying as possible compared to offering a standard set of banking services. The fact that you’ve got a good desktop browser-based service is irrelevant,” says Daryl Elfield, Director of Testing Services at KPMG, who asserts that the demands of the mobile app market are “completely different” in terms of user needs and expectations. “The innovation is in making friendly, accessible, speedy customer journeys without sacrificing security.”
To achieve this, says Elfield, banks require a change of mindset. The creativity and competitiveness of the wider mobile app market has created expectations that latecomers must match. Banks should take no more than three months to get a new app to market, then must have the ability to capture and analyse all the various forms of feedback quickly enough to update the app on a regular basis.
“If you spend four years developing your mobile app, you’ve missed the boat completely,” says Elfield, who suggests that the agility required to compete effectively in the mobile app market is far from second nature in the financial services sector. “Some challenger banks and mid-tier insurance companies already have the necessary development experience. But among some of the larger players, agility is not particularly widespread.”
“If you want to test mobile apps, you need to be able to develop agile testing practices, which will be challenging for a lot of banks.”
Daryl Elfield, Director of Testing Services, KPMG
In addition to the pace of development, a further characteristic of the mobile app market that further cements the need for agility is the diverse range of environments in which new apps must operate. With user needs, network capabilities and device functionality constantly evolving, firms looking to drive business through mobile channels must be set up for continuous change. Critical to this is access to suitable testing facilities and processes. “If you want to test mobile apps, you need to be able to develop agile testing practices, which will be challenging for a lot of banks,” says Elfield.
Some banks make the mistake of assuming that mobile testing is similar to that required for browser-based software. But the multiple variables that must be accommodated when preparing an app for the mobile market add up to a level of cost and complexity that is compelling many to outsource testing to specialist third parties. In Elfield’s opinion, outsourcing mobile app testing is likely to be the most cost-effective option, with the caveat that there is currently a wide range of options on the market. “There are a lot of pitfalls when choosing the correct lab, the correct setup, the correct testing partner,” he warns.
While many outsourced testing service providers offer mobile testing facilities, not all have the necessary range of mobile-specific services, nor do they all have finance sector expertise. Capabilities flagged by Elfield include emulation of mobile devices (in addition to physical), automation testing, performance testing, user-experience testing specific to mobile apps, and social media testing. “The only way to differentiate yourself in the mobile testing market is to both demonstrate your ability to handle the specific demands of mobile testing and credible sector-specific knowledge,” he says.
How complex can mobile app testing be? A mobile testing ‘lab’ consisting of a group of human testers sitting in a room manually testing an app on a variety of the latest phones is expensive, inefficient and time-consuming. Testing today requires automated testing tools that allow for consistent testing of mobile apps in a variety of conditions, environments and devices. According to Ash Gawthorp, CTO of testing solutions and consulting firm The Test People, it is an ongoing service that must be closely tailored to the needs of individual firms and anticipates the future challenges that changes in technology and user preferences may bring.
In terms of automated testing tools, for example, Gawthorp regards the different levels of functionality required like layers of an onion, peeled back successively to reveal another level of complexity. The core requirement of an automated testing tool is the ability to ‘drive the screen’, i.e. allowing either human testers or automated tools to see what happens in response to use of hard and soft buttons on the device and to ensure the app’s displays work as expected. The second layer of testing focuses on how the functionality of a mobile app performs on a particular device, which can vary considerably given the range of devices and large differences in capability for a given platform, particularly the Android operating system.
The third level of variants that the automated mobile app testing tool must handle relates to the impact on application performance of differing wireless and mobile networks, specifically bandwidth, packet loss and latency, as well as the effect of switching between networks and recovery following loss of connection.
Closely related is the fourth layer: power, a finite resource in the mobile world. Automated tools must be able to track how an app uses power and how a reduction in power affects its performance. “If an app is chewing through the battery, users’ perception of an application is going to be quite negative,” observes Gawthorp. On the outer layer, the automated testing tool must be able to assess how the mobile app interacts with the device features that make it truly mobile. The business case for a mobile app might be driven by how it interacts with the microphone, GPS facility or camera, for example. “If you have an app on a device which is using location services, you need to be able to trigger that GPS component under controlled conditions to simulate what happens if you’re in a certain place at a certain time,” Gawthorp explains.
Moreover, to be truly effective, an automated testing tool must be able to interact with a wide variety of mobile apps and devices in both ‘real life’ and simulated conditions. In terms of apps, the automated tool must be able to test both web apps that can run on a mobile platform and native apps, as well as hybrid apps. The testing tool must also be able to run on physical and simulated devices – use of emulators and simulators can increase scalability significantly – and to configure the device in various ways, for example to allow installation of various operating systems, and return it to a known state. Finally, the tool must be able to test the app without requiring a significant change to the build process – this isn’t so bad in an organisation writing its own apps, but in a large multivendor environment with commercial agreements in place with suppliers maintaining a separate build purely for test automation this can add significant management overhead.
Performance testing sits at the very heart of the mobile app testing process as it focuses on how an app is able to handle various real-life impediments to its ability to deliver the user experience as intended. It’s also one of the most complex areas of testing because client side performance depends on the interplay between the app, the device and the network. Almost by definition, a mobile app is in use on the move as we switch between networks, whether for business trips or just in the course of our daily working lives. The coding and data usage of an app can make a big difference to how it performs on a device as it moves from a 4G to a 3G or even 2G network. Testing can identify both reasons for under performance and potential solutions. “It might be that there are a lot of data being retrieved but not required at that point in time or indeed at all, alternatively performance could be improved if requests are batched up to reduce the number round trips,” explains Gawthorp.
“Financial markets clients have users located in multiple geographies who expect those prices to come back in a timely fashion.”
Ash Gawthorp, CTO, The Test People
In terms of client-side performance testing, there is increasingly an overlap between performance testing and automation. An automated testing tool that has the functionality to monitor timings as required for detailed performance testing can add considerable value. “By making adjustments to a network characteristics tool, you can add the latency characteristics that would apply if you were accessing the solution from halfway around the world. Financial markets clients have users located in multiple geographies who expect those prices to come back in a timely fashion,” says Gawthorp. Failover and resiliency are also critical parts of the performance testing to identify how the app responds to a temporary loss of network connectivity or power.
Technology vendors specialising in the financial services space have had to respond to the evolving needs of clients to deliver services via new channels such as mobile apps. In the wholesale market, there is increasing competition between banks to enable critical decisions to be made securely and efficiently regardless of location, either by staff or by clients.
From a software development efficiency perspective, a key underpinning of specialist multi-asset trading system provider SmartTrade’s approach is use of HTML5 as the basis for all applications, which allows them to be rendered as required for use on mobile or desktop. The firm uses HTML5 to code the content then CSS (cascading style sheets) as the markup language to adapt the app to the specific device. Once the app is written, then comes the true challenge of mobile app delivery. “For me the challenge of writing a mobile application is first, the testing, second, the control.” says David Vincent, CEO of smartTrade, and previously the firm’s CTO for 13 years.
“The challenge of writing a mobile application is first, the testing, second, the control.”
David Vincent, CEO, smartTrade
smartTrade has taken a largely in-house approach to the challenge of ensuring that their mobile apps can work as effectively as possible in the event of network-related limitations, including performance testing. To minimise the impact of variations between networks on the performance of its mobile apps, the firm has developed a back-end engine that can identify – and respond to – any on-screen problems caused by a device running on a low-bandwidth network. To prevent the mobile app from slowing down, the back-end engine, or presentation server, adapts to the bandwidth limitations, for example by altering the flow of data between app and server. This presentation server also responds to a dropped network connection by ensuring that content is redistributed to the app to enable the user to continue to use it seamlessly, which is particularly important for trading apps. “When people are placing trades, you need to have a way to reconcile when you log in again,” explains Vincent.
When it comes to testing, smartTrade’s approach has been to develop device emulators rather than testing its mobile apps on multiple physical devices. As the apps run on the emulators, smartTrade deploys tools that control and monitor bandwidth levels, essentially ensuring that the streaming engine adapts appropriately. “We measure round trips and other elements, so that we know exactly the way the application is going to react, but do so within a very secure and controlled environment,” says Vincent.
As well as coping with unexpected network events, such as a lost connection if a user’s train goes into a tunnel for example, trading apps must also contend with unpredictable market events. A sudden price fall or other market shock could result in a surge of usage as multiple traders log-on to check and very possibly trade out of the existing positions. Developing apps in HTML5, as opposed to building native apps, and being able to simulate a sudden rush of connections in lab conditions – independent of device – is critical to ensuring smartTrade’s apps can adapt to load changes. “In case of a significant market event,” Vincent explains, “it might be necessary for us to only send the most recent data to the end-user device, which would otherwise be flooded by data.”
As a firm specialising in delivering a variety of solutions to financial markets clients, it is second nature for smartTrade to test the ability of its mobile apps to handle market shocks, but that same combination of industry vertical expertise and mobile-specific app testing capabilities is fundamental to the credibility of any third-party testing service.
For a firm such as The Test People, which provides testing-related services across a range of industry verticals, the challenge is to draw on expertise and capabilities to best serve the needs of the individual client. An increasing range of firms may be looking for a cohesive, end-to-end automation and performance offering for mobile apps, but this may look very different depending on objectives and budget. “It’s important to understand the client well in order to work out how to focus your combined efforts,” says Gawthorp. “They may want to test performance across a wide variety of devices or prefer more in-depth testing on a narrower variety of devices, for example. An outsourced testing services provider needs to find out what ‘acceptable’ is, and then advise the client on where to focus their effort in terms of performance improvement or addressing functionality or usability issues.”
“An outsourced testing services provider needs to advise the client on where to focus their effort in terms of performance improvement.”
Ash Gawthorp, CTO, The Test People
Inevitably, best practice in testing mobile apps must move as quickly as the app market itself, remaining open to changes from any number of sources. Mobile banking is seeing notable growth. A report published in June by the British Bankers’ Association – ‘The Way We Bank Today’ – found that UK banking customers had downloaded banking apps on 22.9 million occasions by the end of March 2015, an increase of 8.2 million in 12 months. It also predicted customers would use mobile devices to check their accounts 895 million times in 2015, compared with 427 million branch transactions.
The BBA report may focus on retail banking, but there are many signs of the potential opportunity represented by the nascent, but fast-growing market in financial services mobile apps. And like all opportunities, the ability to grasp it successfully will be determined by the underlying investment strategy. “Firms must recognise that mobile is different and their traditional testing teams will need support. Stumbling into mobile testing can be an expensive mistake if you don’t get some help along the way,” KPMG’s Elfield warns.
