The online gaming market is increasing at a faster pace owing to the rapid proliferation of mobile phones and mobile internet. Gamers of all types (i.e. casual gamers, hardcore gamers, pro gamers, or newbie gamers) are now able to play games on different devices like mobile, gamepads, joysticks, etc. The global gaming market is expected to grow to $267.46[1] billion in 2027 at a CAGR of 17.9 percent.
Emerging technologies like AR (Augmented Reality), VR (Virtual Reality), etc. have further fuelled the growth of mobile gaming! These technologies have the capability to provide gamers with an immersive gaming experience. Also, the fundamentals of habit loops[2] and psychology are applied to almost all mobile games.
Different gaming personas, usage of emerging technologies in mobile gaming, and mobile & OS fragmentation are some of the factors that need to be considered when releasing mobile games at scale! Shown below are some of the personas of mobile gamers that indicates the mobile game developers (or gaming companies) must cater to all types of gamers to achieve growth:
Different Gaming Personas
Since every gamer is different, it is essential to perform game testing by stepping into their shoes 🙂To top it all, every gamer will have his/her own preference of device for playing games. Exploratory and manual tests are on the top of the test pyramid which means it helps in identifying issues that might have gone uncaught in earlier phases of testing.
Test Pyramid
In this blog, we look into the essentials of exploratory testing along with deep diving into why gaming companies (or game developers) must run exploratory tests for improving game quality. Let’s kick-start with a quick recap of exploratory testing.
What is Exploratory Testing?
As mentioned earlier, exploratory testing is an approach in testing where testers unearth issues in the product (i.e. in this case mobile game) by running random tests. The tests run by the exploratory testers are not a part of the automation rule-book (or plan), which is why exploratory tests are sitting on top of the test pyramid.
Exploratory testers normally use their domain knowledge to locate potential issues in the product. Let’s take the case of a mobile game that can be played on Android as well as iOS devices. Here the testers have the flexibility to test the game play, game design, game navigation, and more. Think of it as tests that cannot be covered using traditional approaches (e.g. automation testing).
Sample exploratory test scenario for a mobile game:
| 1. Create an account using the preferred option (e.g. SSO, Email, etc.)
2. Login to the mobile game using the credentials created in step (1) 3. Play the game 4. Perform random activities (e.g. scroll, swipes, internet ON/OFF, etc.) in the middle of game play 5. Observe whether there is any breakage in the game play or distortion in the game graphics 6. Continue the steps (1)~(5) for every release. Tweak (or enhance) the steps as and when there is a necessity |
In a nutshell, exploratory testing is more about storytelling where the exploratory tester plays around with the characters (e.g. buttons, progress bars, web pages, etc.) for the product improvement.
In product sense, mobile games are way different than other software products (e.g. ERP, CRM, e-commerce, etc.) as there is scope for exploratory testing to test UI, performance, and more. This is where gaming companies and game developers can leverage the expertise of mobile application testing company so that they can release a high-quality mobile game to their audience.
Also Read: Exploratory Testing: A Detailed Guide
Benefits of Exploratory Testing of Mobile Games
As discussed so far, exploratory testing of mobile games plays a vital role in improving it’s quality since testers are able to focus on uncovering defects that were left undiscovered from the earlier stages of testing. Now that we have got the basics covered, let’s look at some of the major benefits of running exploratory tests on mobile games.
Testing third-party integrations
Like any other software product, the first step to playing any mobile game is sign-up/sign-in. To reduce friction in the user-onboarding process, almost all mobile games support signing-up using SSO (Single Sign-on). However, third-party integrations can cause breakage in the user-onboarding in case there are issues with the said service.
For instance, many apps started witnessing issues with Twitter sign-in[3] after it made changes in their API pricing. Such issues can be unearthed using exploratory testing, as testers do not use the traditional rule-book for testing the application.
Game Play Testing
Game play (i.e. game graphics, game logic, leaderboard logic, etc.) is one of the important aspects in a mobile game. It plays a vital role in improving the key gaming metrics (i.e. DAU and MAU) of the mobile game. Many mobile games are graphically-intensive so that gamers are hooked to the game. Every gamer is different, thereby making exploratory tests an important aspect of testing.
Exploratory testers with different skills and domain expertise can be huge assets in unearthing issues in the game play. For instance, exploratory testers can test the game play by randomly swiping on different characters, testing the game logic on different throttling conditions, and more. Experienced game testers could be huge assets in such cases since they can help in testing the game from aspects like UI, UX, performance, and more.
Also Read: Detailed Mobile Game Testing Checklist
Testing Game Flow
Be it a single-player game or a multi-player game, every mobile game has a game flow and complexity levels that keeps the gamers engaged to the game. Along with game logic and game UX, game flow is one of the most vital elements in the game since it has a monumental impact on metrics like Time Spent in Game, engagement, and user retention.
Exploratory testers who have extensive experience in game testing can be huge assets in testing the game flow. Testing the mobile game from different perspectives (i.e. random swipes, random button clicks, etc.) as an exploratory tester helps in identifying potential gaps in the game flow (or game mechanics).
At the same time, exploratory testers can also test the game mechanics by donning the hat of a normal gamer and surpassing different levels in the game. Using this approach, testers are able to identify issues in the game from different perspectives.
Discover undiscovered bugs via SEBT
Till now you would already know about the potential possessed by exploratory testers 🙂 However, it is important to mention that exploratory testing of a mobile game comes along with a lot of complexities – heavy game graphics, different levels of game play, game installation on a diverse range of devices, and more.
As shown in the image below, there can be close to 30+ categories of bugs (e.g. Art related bugs, UI, Stability, etc.) if only FPS (Frames Per Second) is considered.
Bug Taxonomy Of Gaming
Using bug taxonomy in gaming, software bugs can be categorized into different categories:
- Performance bugs
- Graphics bugs
- Localization bugs
- Audio bugs
- Network bugs, among others
You could make use of SEBT (Session Based Exploratory Testing) taking the limitation of human attention and diversity of bugs into consideration. Exploratory testers can focus on Charter (a specific part of the problem) and maximize the testing time using Session.
The Divide & Conquer approach can be used to test different levels of the mobile game. If there is a possibility of 100+ bugs in a game with 20+ levels, have a team of exploratory testers and focus on a certain number of bugs for a particular session.
Sample Charter Definition
Sample Charter Management
We recommend reading this insightful blog on session-based exploratory testing to gaming where all the aspects related to SEBT in gaming are covered in great detail!
Testing Game Performance
As shown below, mobile gaming is one of the fastest-growing markets as mobile games are capable of offering more immersive experiences. As mobile devices become powerful[4], it is possible to run graphics intensive games on almost all the mid-segment and high-end mobile phones.
Mobile Gaming Market
However, there is a possibility that your end-users (or gamers) might witness lags, crashes, etc. on their devices[5][6] owing to thermal throttling, insufficient memory, network issues, among others. Testing the game on devices with different hardware/software configurations can help in locating the issues listed earlier.
This is where exploratory testing can be used exhaustively, as testers can help in testing the game’s performance on different device and platform combinations. Testers can also test the game under different network, battery, and temperature conditions; thereby helping locate issues that might have gone undetected in the earlier tests.
Also Read: Complete Guide To Game Performance Testing
It’s A Wrap
Exploratory testing of mobile games is extremely beneficial in the short-term as well as long-term. This is because exploratory testers use their domain knowledge and intuition to uncover potential issues (or defects) in the game.
All the other testing approaches that are a part of the test pyramid rely either on automated scripts or pre-defined inputs against which the game functionalities are tested. This is not the case with exploratory testing! In this blog we deep-dived into some of the major reasons on why game developers (or gaming companies) must invest in exploratory testing.
Considering the benefits associated with exploratory testing, it is recommended to onboard game application testing services in order to maximize the ROI from exploratory game testing.
