What Is Automated Mobile App Testing and What Are the Benefits?

Naturally, you want your application to meet user expectations and gain traction. For this, you need a solid team of developers and effective mobile app testing that will ensure a high-quality end product.

Estimates show that testing takes up to 40 per cent of the overall IT spend and sometimes, testing costs can even surpass the development budget. This mainly happens when testing is considered to be the final stage of making an app.

When a mobile application is tested only at the end of the development process, fixing errors takes several times more time and money. Studies even show that developers may not have enough time for proper testing.

With something as changeable as app development, it just doesn’t make sense. Why not do it gradually, testing an application at each iteration? Developing new features along with checking already developed ones will save time and improve the quality. What saves even more time is opting for automated mobile app testing and utilising it repetitively from the beginning.

At MadAppGang we used automation while working on a restaurant ordering app, Tayble, and a project called Creator Connect aimed at helping artists sell their work. Automated testing allowed us to make the products our clients envisioned faster and more efficiently.

During the app-testing process, we check general app performance, how the application performs on different devices and OS versions, and double check that new features haven’t negatively impacted on existing functionality.

The variety of devices and operating systems is one of the biggest challenges in app development. Developers should deliver a product which doesn’t display compromised quality on any of the many devices or OS versions.

Even with native development, for a single platform, an app won’t work equally on all devices. Some problems may occur on older versions of the OS, for example. The end product needs to work smoothly regardless of the version, integrate with third-party services, and perform data transfers well.

There are two ways of testing applications: manual and automated. Manual testing is a series of steps performed by people, while in automation testing, testers write a specific framework to achieve the same goal.

For example, an application has three possible scenarios for login: username and password credentials; via Facebook; and via Twitter. A tester can perform all three actions manually, filling in the form on a smartphone. Or, a tester can write scripts for an automated tool. What automation does here is performs the same actions faster. Isn’t it what every entrepreneur wants, to cut time?

In any case, testing a mobile app works the following way: developers make a build; a quality assurance tester identifies problematic issues in the app by running automated scripts, creating bug reports, and assigning them to developers; the latter fix the bugs and then the tests are rerun.

Manual vs. Automated Testing

The fundamental difference between testing manually and automatically is that with repetitive automated testing, feedback is received much faster. But you can’t apply only one approach to testing, instead you will need to combine both. There are some aspects that are hard to achieve without manual testing. For example, performing security testing, identifying errors in the text, and checking usability. Mobile app usability testing essentially requires human involvement.

With automation testing, you can find more bugs faster and eliminate human error. It will not only save you time and human resources, but will also give you greater coverage because with automated tools, you can test more scenarios.

Automated testing uses different data sets to run tests on several devices or even platforms simultaneously. It also verifies functionality that was already implemented. It can even be applied to design elements owing to the range of graphics libraries: these libraries compare a graphics file with a screenshot of a working app. Since GUI scripts can be recorded, you can reuse them in many similar operations. All in all, test automation is more exciting for engineers.

On the other side, there are some complications which come hand in hand with automation testing. First of all, it requires the permanent assistance of a test engineer because of the dynamics of development. After all, as the project changes, tests will need to be changed as well. Even the slightest change of one element in an app requires the writing of new tests.

Big projects will need additional funding for automated mobile app testing services, but if implemented from the very beginning, any additional costs will more than pay for themselves.

There are many automation tools to choose from and lots of open source ones. At MadAppGang we use Appium, a framework for testing mobile applications including hybrid and web apps. It’s a cross-platform mobile app testing solution which supports real devices and emulators. We write test scripts in JavaScript, although with Appium, you can also write tests in other programming languages.

There are several other automation tools we implement in our projects. These include:

• Mocha, a framework functioning on Node.js which runs tests sequentially and allows for flexible and accurate reporting. It also maps uncaught exceptions. It has functions which help describe a test that was run.
• Chai, a BDD / TDD assertion library for Node.js. It can be paired with any JavaScript testing framework and is used for implementing tests. It has several interfaces from which a tester chooses the most suitable one.
• WebdriverIO, a framework based on the WebDriver package. It allows us to navigate the app and perform various functions inside it.

With the help of these automation tools, we achieve the following benefits for our clients:

1. Reduced software development time: End-to-end user interface tests take too long if performed manually.
2. Early bug fixing: By running tests after each iteration we detect issues that need to be fixed before the build goes to production.
3. Resolved fragmentation problems: We are able to test an app on different devices and OS versions.
4. Resolved integration problems: We detect how the new code interacts with the pre-existing one.
5. Better test coverage: We can write all possible test scenarios and run them on various devices.
   

Automation can be used for mobile app functional testing and non-functional testing. The former makes sure that a specific part of an app works as expected, while the latter is used to check non-functional parts of applications such as the design, performance, and security.

The following are the phases of automated testing:

• Unit testing: Developers write tests to check particular modules or functions.
• API testing: Applied during the integration phase and is designed to determine whether the API returns a correct response to possible requests.
• GUI testing: Utilised during the execution phase, testers write scripts to replicate the actions of users in order to check the user interface.

There are some other types of testing that can be handled with automation tools:

• Smoke testing: A short cycle of tests confirms that after assembling a new build, an application performs the essential functions. After an app successfully passes smoke tests, a full test cycle begins.
• Integration testing: Components of an app are combined to produce an output and the interaction between the software and hardware is checked. In our project SmartRun, a pulse sensor had to report a correct heart rate to the app. AppleWatch could function as a sensor and report to both applications on a mobile device and on AppleWatch, so we had to check how our software solution interacted with the given hardware.
• Regression testing: It is carried out to check how changes in code affect the functionality and to make sure that none of the existing modules have been affected by new ones. It allows a focus on new features instead of constantly turning back to fix existing ones.
• Mobile app performance testing: A type of non-functional testing which determines the app’s performance under a certain workload in a given environment.
• Acceptance testing: It belongs to functional testing and checks whether the acceptance criteria determined by the client are fulfilled. Let’s say you want your app to have real-time chat and the option to send files, as testers, our goal is making sure the chat works the way you described it.

No application is finished until it’s tested. If you wonder how to test mobile apps and make the most of the process, run tests on the earliest stages of development instead of doing that at the very end.

How can automation testing facilitate the process? It’s not an across-the-board solution as some testing types should still be completely manually. But for many cases, it’s a reliable option to consider.

Above all, it’s fast and can be done remotely. It’s also valid for any type of development: cross-platform, native, or hybrid. It covers different OS versions, various devices, and emulators with different configurations. Last but not least, it can save you time and money, particularly in the long run, and who doesn’t want that?

09 May 2019