How can you test push notifications? We've put together a comprehensive list of frequently asked questions with the basics to get you started with testing this critical feature.
Push notifications are a critical feature within mobile apps, with users counting on them for reminders and alerts, and app publishers relying on them to keep users engaged and bring them back into the app. Users rely on these reminders and messages for timely notice of events like news, market trends, and group messages. As a result, it's important to test your push notifications with each code release. If they don't work, your users will miss out on critical information and you risk losing them to competitors.
But, testing push notifications isn't easy. They present several challenges that many other features don't share. They behave differently on Android and iOS, and users have significant control over how they act on their devices. We've assembled a list of frequently asked questions (FAQs) and answers about testing mobile push notifications below.
Let's dive in!
While they seem straightforward, push notifications have a lot of moving parts. For example, users should receive your messages even when your application is closed. They should also work when a user's phone is locked, and if they contain clickable content, the link needs to be valid and point to the right location inside your application.
So, make complete verification of your push notifications a part of your test suite.
To test mobile push notifications, you need to cover the different circumstances under which your users might receive them. This includes different configurations, various application states, diverse user options, and as many different hardware and operating system versions are you can cover.
This isn't an exhaustive list of features, but it should give you an idea of where to start with your test cases for push notifications.
As of XCode 11.4 beta, you can test push notifications for iOS with Apple's iOS Simulator. You can register for them in your application, drag-and-drop APN files to the simulator, and simulate them from the command line. You can also test push notifications in the Android emulator. Like the iOS Simulator, it can register for notifications and display them.
Of course, the best test is with a real device. The emulators make it easier to test with a variety of screen sizes, operating system versions, and configurations. Still, they don't account for subtleties like how applications act when they are running in the background or how they interact with other processes on native hardware. The final test of a new release needs to be on real hardware and has to include push notifications.
In addition to simulators and real hardware, you can perform your end-to-end testing with the notification service you use in production. For example, Firebase has instructions for testing Android and iOS applications here. OneSignal has an FAQ here.
While push notifications seem to be simply a matter of sending an asynchronous message to your application, there's a lot that can go wrong.
As discussed in the previous question, complete testing of push notifications requires testing with real devices. (This is true of any mobile app, not just an app with push notifications.) Assembling a set of mobile devices with the right mix of versions and configurations is hard but not impossible.
The obvious difference between testing iOS and Android push notifications is their GUIs. You'll need to write different tests to verify the contents of your notifications. But the differences don't stop there.
iOS and Android manage notification delivery in very different ways. Here are a few things to take into account when testing them:
You can improve your application's stability and customer satisfaction by avoiding a few common testing errors:
While push notifications require extra care, that doesn't mean you can't include them in your CI/CD pipelines. You can integrate the tests into automated builds and deploys and adjust the nature of the tests depending on the type of build. For example, real device tests are more expensive in terms of both time and money, and adding them to every build would slow down development and may even lead to developers skipping the test altogether.
Here's a possible approach to testing push notifications in a CI/CD pipeline:
Push notifications are an important feature, and supporting them can get complicated. But with proper care and careful testing, you can build and release an application with reliable and useful messages for your users.
This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).