The rapid proliferation of smartphones and mobile internet has resulted in increased interest in mobile app development. No business is left untouched by smartphones, as it empowers consumers and opens-up new expansion opportunities. As per reports[1], close to 58.43 percent of all website traffic comes from consumers using mobile devices!
Hence, ignoring the mobile medium can put brakes on the growth of your business. What this also means is you are bound to face significant competition from businesses looking to leverage smartphones for their growth! Progressive Web Apps (PWA) can be used to offer a more native-app kind of experience to the customers. E-commerce major Flipkart managed to triple time spent on site and drive 70 percent[2] more conversions using the PWA strategy!
Not only PWA, businesses should also tap into mobile applications since it offers them a different medium to reach out to their prospects/customers. However, there is intense competition on the Google Play Store and App Store across different categories (e.g. e-commerce, fintech, etc.). Though close to 8,900 apps[3] are published on App Store every week, the overall app uninstall rate after 30 days remains as high as 28 percent!
App Store Trend
Along with functionalities, application developers should also focus on building a highly-performant app that functions well (without any glitches) under varied load conditions. This is where performance testing becomes extremely crucial, as it helps in verifying whether the app meets the expected performance requirements.
In this blog, we look at the nuances of performance testing of mobile applications. We would also cover the role of mobile app testing services companies in accelerating the process of app performance testing.
What is mobile app performance testing?
Mobile application performance testing is the process of measuring and verifying the app’s performance under different load conditions. This particular app strategy is instrumental in identifying app’s bottlenecks when compared with the KPIs (Key Performance Indicators).
Here are some of the key performance metrics that every app (or website) developer should take into account:
| Front-End App Performance Metrics | Back-End App Performance Metrics |
| 1️⃣ Throughput | 1️⃣ Latency |
| 2️⃣ App Response Time (or frames/second) | 2️⃣ Server Response Time |
| 3️⃣ App Startup Time | 3️⃣ Time to first byte |
| 4️⃣ Screen Rendering Time | 4️⃣ Error and Request Rate |
| 5️⃣ Battery, CPU, and GPU consumption | 5️⃣ Network Bandwidth |
| 6️⃣ Retrieval time from background to foreground | 6️⃣ Concurrent Connections |
Most of the above KPIs are applicable to websites as well as mobile applications. Let’s consider the case of an e-commerce application where users can buy electronic items from the website (or app). The response time of any user-request must be no more than two to three seconds[4].
The response time (or latency)[5] can increase if more concurrent users are using the application. Higher the response time, greater is the probability of cart abandonment, app uninstall, etc.
This is where app performance testing can be leveraged, as it helps in identifying issues and resolving bottlenecks encountered under different load conditions. A mobile app testing services company with experience and expertise in performance testing can be useful in accelerating the testing process at scale!
App Performance Testing – Key Performance Indicators
Uptil now it is evident that developing a mobile app that works flawlessly with higher loads is not an easy task! This is why it becomes integral to keep a close watch on all the app performance KPIs. Akin to automated testing, performance tests must also be a part of the CI/CD pipeline.
Irrespective of whether the code is deployed on stage/prod or any other environment, performance testing must be at the center-stage of the QA process. Let’s deep dive into some of the most essential KPIs when it comes to performance testing of mobile applications:
1. Load Time
The load time of a mobile application is the time taken by the mobile application to initialize all the necessary resources for it to startup and load the UI. As per reports, the ideal load time of any mobile app should be not more than 2 seconds[6].
Maximizing the number of concurrent users and/or requests is one of the ideal ways to verify if the load time is impacted by any form of concurrency. It is important to note that load time is not the app startup-time, which is why load time must be measured with users at different stages (or states) in the application.
For an e-commerce application, load time can be tested with a large number of users at account creation stage and equivalent/more users in the product checkout (or payment) stage.
Since load time can get impacted with concurrency (users and/or requests), it is essential to stress test the app with either/both set to maximum capacity.
Also Read: A Deep Dive Into Application Performance Testing
2. Response Time & Throughput
Response time (measured in milli-seconds) is the amount of time taken by a server to address a user request. Higher response time means increased latency. While Response Time is the metric to measure performance of an individual query, Throughput is the metric to measure performance of the entire system (or app).
Response Time and Throughput
In simple words, Throughput is the number of requests (or transactions) that can be seamlessly handled by the application, without putting them in a queued state. TPM (Transactions per Minute) or TPS (Transactions per Second) is the measurement unit of Throughput.
As an app user, you could have come across multiple scenarios where the app (e.g. e-commerce) performance shows deterioration during festivals, sale days, etc. This is because the server infrastructure is under heavy-load with a significantly high amount of concurrency (i.e. users & requests). Since response time is so critical for the app, it is imperative to measure the same by subjecting the app to different load levels.
3. Screen Rendering Time
Every mobile application comprises front-end assets like images, icons, text, buttons, and other graphical components. Caching in mobile applications is normally used to avoid unoptimized downloads, which eventually helps in improving the response-time and throughput of the application.
Page Load Timeline
As seen above, screen rendering (or page ready) time is the time for application to be usable. This includes successful loading & rendering of the front-end components (e.g. HTML, CSS, JavaScript) on the screen. Screen rendering time is one of the most important metrics for app performance since it directly impacts the usability aspects of the app.
Higher the screen rendering time[7], more are the chances of the user abandoning (or closing) the application. Optimization of resources, device side-caching, reduction of number of HTTP requests, and Minification are some of the prominent techniques used by app developers to improve the page-ready time.
Before proceeding with app performance testing, you need to prioritize the app features, app types (i.e. hybrid, web app, etc.) and device combinations on which tests have to be conducted by the team.
Image Source
App performance testing is instrumental in testing the app’s readiness by subjecting the app under varied loads. Apart from the above mentioned KPIs, developers & testers should also look into monitoring app crashes and emulation of device performance to verify the app’s performance at scale.
Popular App Performance Testing Tools
Though there are a number of open-source and proprietary tools, we are listing down the widely-used tools below:
Appium
Appium is one of the most popular open-source test automation frameworks for mobile app testing. This cross-platform framework can be used for testing hybrid, native, and web applications. You can leverage the framework for testing Android and iOS applications.
As stated in the official Appium documentation, there is no need to recompile the app since standard automation APIs are used in the framework. Like Selenium, Appium also uses the W3C protocol for automating mobile and native applications.
Appium Architecture
At the time of writing this blog, the latest version of Appium is 2.0. Appium supports popular WebDriver-compatible programming languages – C#, Python, Java, JavaScript, and Ruby. Drivers like XCUITest and UiAutomator2 have built-in mobile-web and hybrid-app support. Due to this, the same Appium script lets you switch between native app automation and webview automation[8].
You can refer to the Appium 1.0 to 2.0 migration guide in case you are migrating the apps to 2.0. Integration with popular CI/CD frameworks and flexibility to use the framework with cloud-testing platforms (or tools) lets you accelerate app performance testing.
Also Read: [Guide] Best Practices in Performance Testing
Apptim
Apptim is also a relatively popular app performance testing tool that works seamlessly with Android and iOS apps. Like Selenium IDE (used for automated testing), Apptim automatically captures videos, screenshots, logs and measures end-user response times[9].
Apptim Desktop lets developers and testers benchmark app performance by comparing two test sessions. You can also set custom performance benchmarks. Tests can also be triggered using Apptim CLI (Command-Line Interface). It also supports Java TestNG and NodeJS test runners[10].
Apptim tool also sports an intuitive mobile performance dashboard that provides detailed information (on Android and iOS devices) :
- Measurement related to app render times
- Power consumption information
- Device resource usage
- App crash captures
You can also add performance metrics to Appium tests, using which you can leverage the benefits of Appium and Apptim.
LoadRunner
LoadRunner from MicroFocus is one of the widely-used tools for running app performance and load tests. It can be used for testing a range of mobile apps – native, hybrid, and web-app. Integration with popular IDEs makes LoadRunner a popular choice for app performance testing.
It can also be used to run performance tests in a continuous CI/CD pipeline. Hence, LoadRunner tests help in unearthing performance bottlenecks at every stage of app development. LoadRunner is ideal for scalable and reliable mobile app performance tests, as it lets you run tests on devices that are located on the cloud.
Apart from the above mentioned tools, you can also use Android Profiler, Memory and CPU profiler.
Also Read: Jmeter Vs LoadRunner: Which One Is Better for Performance Testing?
Conclusion
Performance of mobile applications is pivotal since a non-optimal app can result in uninstalls and negative reviews on the App Store/Play Store. Hence, app performance tests must be a part of the continuous CI/CD pipeline.
In this blog, we covered some of the popular tools for verifying all aspects of app’s performance. Since performance is critical for the long-term growth of the app, it is recommended to onboard a proven app testing services company so that you can ship a high-performant app at a quicker pace.