Citizen Gets 5-Star Rating & Eliminates 1,000s of Manual Testing Hours

Citizen, dubbed “The most powerful safety app”, is a top-five ranked app that delivers real-time safety alerts and live video of incidents happening near their users. With apps on iOS and Android, Citizen has over 10 million users, 50% of which are active, as of January 2022. Citizen monitors 911 calls and keeps its users notified of crimes, accidents, and other incidents in their neighborhoods.
Technology, Information and Internet
10M+ Users

The Challenge for Citizen

To meet the demands of their rapidly growing app and burgeoning user base, Citizen’s product and engineering teams are constantly developing new features, with a new release to the app stores once per week.

"QA is always the last hire"

Lou joined Citizen as the lone-QA, responsible for ensuring the highest quality experience for Citizen’s millions of users throughout the country. Prior to a release, Lou knew the importance of running thorough regression tests, but the priority was primarily focused around new feature testing. 

The Citizen team would execute tests using emulators with manual tests on 1-2 physical devices, but they needed much greater device coverage to ensure a seamless user experience. For Lou, device procurement was difficult due to the cost, time to manage devices, and amount of devices for full coverage.

  • Rapid growth and limited QA resources led Citizen to frequently release without full device coverage. 
  • Releases would lead to a negative customer experience on several device/OS combinations, ultimately causing negative reviews and user churn. 
  • Testing on emulators wouldn’t catch all issues, resulting in bugs slipping through the cracks
"I've never seen an emulator get something 100% correct. When it comes to mobile it is not the same. Nothing can beat a physical device."

Mobot Solution

Mobot’s Customer Success Team worked closely with Citizen to map out a strategy addressing each of their main pain points, and a test suite specifically designed for their release process and business.

While the Citizen team is rapidly building new features, Mobot is used on a weekly basis to ensure Citizen's 5M users have access to a stable production release.

  • Using a combination of Citizen user data and Mobot’s data regarding device usage, the test plans were designed to include a large number of physical devices with comprehensive coverage.
  • Mobot’s team created test plans addressing critical user flows, monitoring performance of the app including battery usage, multi-device interactions (ex: messaging a friend), location spoofing, and much more.
  • A shared Slack channel was set up where Citizen can monitor the execution of tests, view test results as they’re completed, and communicate priorities for the week.
  • As the app has updated over the 2+ years of the partnership, Mobot’s team has updated each test plan as needed, eliminating hundreds of hours of work for Citizen.
“Mobot eliminates the smoke test. We can test in a day whereas a QA team would normally take multiple days.”

The Results

Some highlights of the 2+ year partnership:

  • App store rating from a 3.9 to a 4.8
  • 5x QA efficiency, measured in hours spent testing prior to release
  • 4x increase in physical device coverage
  • 600 test cases run each week
  • Thousands of bugs caught, including many undetected by internal systems
“Feels like magic to our team”

Robot-powered testing drives results

We've helped leading development teams around the world to improve their testing processes

5 star icon

Mobot is amazing! Great partner to make sure we don’t have any surprises when we release new versions of our SDK! This has been a huge game changer for our mobile team and has saved us and our customers a ton of time and headache.

Lewis Chung
Engineering Manager at Persona
5 star icon

Can’t say enough about how much better I feel with each release now that Mobot is part of the pipeline. Our stability rating for iOS is now at 100%!!!

Swamy R
CTO/COO of Sandboxx
Browse All Success Stories →
5 star icon

"With Mobot in place, Citizen’s app has seen tremendous gains, it feels like magic to our team"

Louis Johnson
QA Tester

More Case Studies You'd Enjoy

Batch partners with Mobot to eliminate manual testing across thousands of mobile test cases

"I think there’s a use case for Mobot for anyone who’s building mobile apps…it is like 100% a no-brainer from my perspective. If you’re dealing with push notifications, SMS, anything that’s mobile device specific, I think that’s where Mobot really excels."

Read Case Study →
View All Case Studies →