iOS Testing: A Practical How-to Guide for Engineers
Learn why iOS testing is important and the major steps you need to take to get testing in place and ensure that your development team follows through on implementing tests.
Everyone involved in the iOS development process knows that testing is important. If you submit an app that doesn't function properly to Apple's App Review, they'll reject it. This, in turn, leads to another release cycle, unwanted delays, and lost revenue. And even if a buggy app makes it through review, it will simply lead to unhappy customers and uninstalls.
But knowing the importance of testing doesn't mean actually doing it. Creating a robust testing process isn't easy, and many dev leaders either put it off or don't know how to make testing work for them. Let's talk about what you need to do to get iOS testing right.
iOS Testing Planning and Challenges
Before we discuss how to implement iOS testing, let's identify the key challenges you'll need to overcome.
While the iOS ecosystem is more unified than Android's, iPhone and iPad developers still must account for various screen sizes. Apple provides iOS developers with tools like Auto Layout to simplify GUI development for different devices, but that doesn't mean that your apps will "just work." Your tests need to account for multiple screen sizes and profiles.
Identifying Valid Tests
What should you test?
Developers new to testing will often answer this question with "everything!" But if you adopt that approach, you'll never release your application. Or, you'll write tests that are so ponderous that you end up skipping them because they take too long to run.
We've already covered the App Review process. Apple provides basic guidance on its criteria, even if it's not crystal clear. They also provide Human interface guidelines for iOS. These documents and your internal specifications will go a long way toward identifying test criteria.
When I look at testing tools for iOS, the paradox of choice immediately comes to mind. There are many different tools, and each one comes with its own recommendations on how you should perform your testing. Where should you start? We'll discuss this below.
Budgeting Time to Write and Update Tests
Just as you never finish developing an application, you're never done testing it, either. Each time you add a new feature set, you must add a new set of tests. You should accompany every bug fix with at least one test that verifies that the fix works and the bug won't come back.
Writing tests takes time, and sometimes writing a useful test is more difficult than writing the application code or fix. So, you need to account for test development time in your schedules, or your tests will gradually become outdated and misleading.
Implementing iOS Testing
Now that we've covered some of the issues you'll encounter while setting up your iOS testing, it's time to implement them.
What Type of Testing Do You Need?
There are many different testing tools, so there are many different types of testing. Here's a non-exhaustive list of the kinds of testing you should consider for your iOS application.
Unit tests are low-level tests that test specific behaviors (units) in your code. You usually run them as part of builds, and they are very easy to automate.
If you think of unit tests as the tests that are closest to your application code, integration tests are the next level "up." They verify that components work well together. So, if your application connects to a RESTful service, integration tests verify that the app and service work correctly with each other.
While unit and integration tests verify that the code works the way the developer expects, acceptance tests ensure the app works as specified by the requirements. This may seem like an academic difference, but it's not uncommon for a development team to lose sight of a business requirement during a major release or an emergency bug fix. Acceptance tests act as a critical backstop when things are moving fast.
Some teams refer to acceptance tests as functional tests. Or in some organizations, functional and acceptance tests are discrete activities, with functional tests performed inside the development organization and acceptance tests performed with or by the customer or stakeholders. Either way, the tests ensure that the application continues to meet business requirements.
As the name implies, performance tests verify that the application performs acceptably under the expected load. This may include reliability, speed, and responsiveness. For an iOS app, you might verify CPU utilization and power consumption.
Picking Your Tests
So, which tests do you need? Where should you start?
Unit tests are probably a good place to start if you don't have formal tests set up. They're easy to automate, especially if you start with XCTest, since Xcode makes it easy to add your tests to your build, and they'll run easily from a CI/CD pipeline. But nearly every iOS application interacts with the outside world, so you'll want to get some automated integration tests under your belt as soon as possible, too.
Choose Your Test Tools
We briefly touched on the abundance of test tool options for iOS testing above. Going through even an abbreviated list of the most popular options and their advantages and disadvantages would take up an entire blog post.
And which tool is best for you and your team depends on how you work. So, when you evaluate test tools, ask yourself and your team these questions.
- What tools are you already using? Does this tool integrate well with them, or will it require changing how we work?
- Will the new tool work with build and test automation? If you're using CI/CD already, does the tool play nicely with it?
- What type of application are you developing? Is it a pure Swift or a hybrid web application? Are you using cross-platform tools?
- Is the tool open-source, or is there a licensing fee? Many of the most popular test tools for mobile applications are open-source.
Integrating Testing Into Your Development Workflow
You can pick the tests you want to write. You can choose the tools and make them available to your developers. But your testing efforts will only be as effective as your efforts are to integrate them into your daily development workflow.
Code Reviews and Unit Tests
Some development managers might try to implement testing by decree. They may announce that every object must have a set of tests or even burrow into the relative lines of code rabbit hole. A better approach is to institute code reviews if you haven't already and make unit tests part of the code review. This will get your developers talking about tests and will produce code that has tests based on consensus. It will also help build a culture around writing tests and discussing them with fellow engineers.
New Features and Tests
A new feature is an opportunity to include tests as part of the requirements. Features and user stories shouldn't include implementation details, but they can include how a test will verify that they work.
Automate, Automate, Automate
Finally, you need to run your tests as part of every build and deployment.
Your unit tests should run for local builds and when a developer pushes code into your source control systems. This means that the tests need to run quickly enough that developers won't be in the habit of skipping them. You also need to set up continuous integration (CI) and enable unit tests as part of an automated build.
Having a few users test it manually when you create a beta or preview and push it to TestFlight isn't good enough. You need automated testing on physical devices to check for new regressions and verify new behavior.
Get Started with iOS Testing
Now that you know how to get started with iOS testing, get to it! For more information on how to get automated testing right, check out Mobot. Our system uses mechanical robots to run your tests just like a real user. This ensures that your app is ready to launch, every time you deploy a new version.
This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).