The mobile application space is competitive and crowded. If you want to succeed, you need to be sure that your application works well at launch and after each new release. And you need to stay ahead of the competition with new features and support for new operating systems and platforms. No matter how skilled and careful your developers are, bugs are inevitable. How do you continue to improve your app without introducing new bugs? 

Complete and effective testing is your primary line of defense against a buggy new release. You need to test every new feature while checking for regressions at the same time. So when you send an update to your users, you know your app still performs as expected. 

You need a mobile app testing checklist to ensure your app is ready every time you release. These ten mobile app tests should be performed every time you prepare a new release. 

Mobile Application Testing Checklist

Mobile apps need to work on thousands of different phone models. There are two major operating systems with their own application stores and several active versions at any given time. And, the mobile phone market is truly global, with a wide variety of languages and localizations. How can you get started with testing and make it a part of every release? 

One way to make sure your latest version is ready for users is with a checklist that you can refer to at the start and end of every test cycle. 

1. Test push notifications
2. Run automated tests on real devices
3. Test performance
4. Check memory consumption
5. Test for security issues
6. Test against different network conditions
7. Ensure that localizations and languages work
8. Verify user customizations
9. Check power usage
10.  Verify accessibility

1. Test Push Notifications

Push notifications are critical. Your users rely on notifications for alerts and reminders, and you need them to keep your users engaged. So, they should be the first thing you test.

Be sure to test that your notifications:

• Still arrive after a user closes your application.
• Are delivered when another application is active.
• Don't conflict or cause issues with Do Not Disturb mode.
• Are visible when your application is active.
• Work correctly with different time zones.
• Are clickable and display the correct content or take the user to the correct screen in your application.

Learn more about testing notifications.‍

2. Run Automated Tests on Real Devices

Your tests aren't complete until you’ve verified your app on devices that represent your user community. This means you've run tests on several actual devices instead of simulators, and that you made sure it looks and acts correctly on all of them. 

This is a heavy lift. You need to test on devices that represent your user community, and you need the time to execute the tests.

First, use your application analytics to identify the devices that the majority of your users own. If you don’t have that, use the most common device data you can find.

Then, use automated testing to get your tests done in a reasonable amount of time. Automation reduces errors and saves time. Automated testing with real devices is a game changer for verifying new features and checking for regressions. 

When you set up your test, you need to check for: 

• Different screen resolutions
• How new and old OS features affect your application
• Varied device capabilities, such as memory, storage, and CPU speed

3. Test Performance

Users notice when a new release introduces performance problems. But, application performance is a complex subject that involves a few overlapping factors. Testing it isn’t easy. 

First, if the new release involves server or API changes, they need to be tested for performance separately.

On the device, run these tests:

• Startup time - new feature sets often require loading new data on startup. Make sure that this doesn't have too much of an impact on initialization.
• Different performance profiles on different devices. Newer hardware versions will perform better than older ones, but your application still must perform well on all common hardware.
• Background performance - when a user backgrounds your app and then brings it back to the foreground, it should stay in the same state and be responsive.

4. Check Memory Consumption

High memory usage can have a serious impact on performance, and it deserves its own checklist entry. It can lead to crashes or simply make the application unusable on older hardware that has less RAM. 

This is another issue that can crop up after a seemingly innocuous code change. It can happen because of a memory leak or an unexpected usage pattern. 

The best way to test memory consumption is by simulating an end-user. Exercise the application in a variety of ways for extended periods of time, while recording memory usage. It should remain predictable, depending on what the application is doing. 

How can you do this? Here again, testing on an assortment of different hardware profiles is your best approach. 

‍5. Test For Security Issues

Security is a primary concern for developers and users alike. It needs to be a big part of your application testing plans. 

The obvious first step is ensuring that your application is secure. 

Verify that it: 

• Uses only encrypted network connections
• Strictly enforces all user permissions
• Doesn't store unencrypted passwords
• Uses robust encryption algorithms

These are mostly design issues, but that doesn't mean that new features can't slip in and break these rules. You need to do a complete security check for every release. 

For more details on security, the Open Web Application Security Project (OWASP) has a mobile security testing guide here. Use it as a starting point for your security testing. 

And there's more to security testing: each new mobile operating system release brings its own set of new security features. You need to stay up-to-date on new OS versions and update your tests. 

6. Test Against Different Network Conditions

Mobile devices are subject to different network situations. They have to switch between Wi-Fi and different levels of cellular connectivity. They're often switched in and out of airplane mode to save battery. 

Your applications need to handle these changes, even if it's to display an error message until the user has network access. If it freezes because it can't reach a network service like your API server, it can lead to a frozen phone and a very confused and unhappy user. 

These situations are simple to test. At a minimum, you can toggle Wi-Fi, cellular data, and airplane mode on and off while running the application. If you want to set up more sophisticated scenarios, you can disable network connectivity by disabling the network or placing the phone in a Faraday cage. 

7. Ensure Localizations and Languages Work

Customizations based on location and language are important to the user experience, especially if you're trying to reach a global market. Whether it's as simple as adjusting search results based on GPS location or supporting languages other than English, these features are important and you must test them. 

Verifying these customizations needs to be a part of your automated tests. Your automated test system should support localized testing and geolocation, so running all your tests in supported languages and with a variety of different locations should be easy. 

‍8. Verify User Customizations

Your users expect a customized experience. They want to go beyond localizations and set preferences like dark vs. light mode and custom swiping gestures. Add setting, saving, and restoring these preferences to your test suite to be sure that they're not broken as a part of a new release. 

9. Check Power Usage

Power utilization, like application performance, can ruin a user experience. Mobile users have wrestled with short battery life for years and an application that uses too much battery too rapidly is quickly removed and replaced with a competitor. This is a bug that can easily happen because of a coding error with network access or even just a missed loop variable, so verifying that your app isn't suddenly using extra power must be part of every regression test. 

This testing should be another part of your automated suite. As you put your application through its paces, watch the battery level and ensure that it doesn't drop more quickly than it did in the last version.

10. Accessibility

It's important that your application and content are accessible to all users. Besides being good business, it's also a legal responsibility. Businesses that don't make their content available to people with disabilities open themselves up to legal liability.

W3C has published guidelines for making online and mobile content usable for everyone. These standards can serve as your baseline for testing.

• Web Content Accessibility Guidelines (WCAG) is their guide for websites.
• Mobile Web Best Practices (MWBP) has their mobile web guidelines.
• Mobile Web Application Best Practices (MWABP) is for mobile applications.

Testing Your Mobile Applications

We've covered a mobile application testing checklist composed of ten key test types and scenarios. You need to test every release extensively, and manual testing or simulators aren't up to the task. Use this checklist as the foundation for your testing efforts before you send a release to the app store. 

With Mobot, you can perform your tests with mechanical robots on real devices and run your tests as a real user would. It's the best way to be sure that you don't miss a thing and cover your mobile app testing checklist. 

‍Get in touch with Mobot today to see how you can test better, faster, and smarter.     

‍This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).‍