██FR█████ █INTELL███████████

An Extensible React Native App Automation Framework


This content originally appeared on DEV Community and was authored by Felicia Walker

NOTE: I’ve had the bulk of this in draft forever so it may be a bit dated now. However, the overall approach is still useful and shows how you can combine multiple technologies together for a cohesive yet flexible solution.

I’m going to keep this high level and cover parts of it in other posts. I will also defer detailed framework code stuff to future articles going over what this eventually evolved into.

I was hired at a previous company to create an automation framework to do UI testing on a React native mobile app. Initially simulators would be used but support for real devices in the AWS device farm was be ideal.

Cucumber needed to be integrated so tests would be easier for developers to write and non-technical people to understand. It needed to be written in Typescript and use a tool called Detox that they had been experimenting with.

Components

Here is a brief description of the various components and packages that the framework uses:

  • Cucumber – Specifically we use cucumber.js which is a runner that links plain language tests, written in the Gherkin syntax, to actual test execution code
  • Detox – Drives mobile apps by hooking into their underlying code. This allows Detox to know when operations are complete, which makes it more efficient in finding elements. This is what is normally used for testing since it is faster and easier to set up.
  • Appium – A server that allows you to drive mobile apps externally. It has more functionality than Detox, but is slower and harder to set up. This is needed because the AWS Device Farm does not support Detox. Also, some tests simply cannot be done with Detox’s limitations.
  • WebdriverIO – A driver that talks to the Appium server and configures it to make using Appium much easier.
  • AWS Device Farm – What the company wanted to use eventually. It supports Appium tests, and it should be possible to package up the framework along with the tests to run in their Node/Appium environment.

Architecture Overview

The main design focus was to make things easy to use an maintainable while being flexible. This means most of the complexity is encapsulated inside Cucumber steps and in the lower code that supports it. Also, the specifics for executing against Detox or Appium exists only in their own encapsulated classes that can be switched in and out. This was most everything can remain the same, including the actual tests, no matter what OS is used or test runner.

Another aspect is the system for identifying elements inside the app. React supports an attribute called testID. These needed to be added in the app code using a system to ensure they are unique and manageable. In the test framework, these are defined and associated with plain language phrases used in the Cucumber tests. To ensure that this is flexible, a system was made to allow for multiple phrases to be used and support for regular expressions.

The architecture does allow for the possibility of adding web in future, via Appium or adding in a new Selenium driver. A second set of tests would need to be created, however, since the web version has a totally different look and feel than the mobile version.

System Level Architecture

Here is a diagram of the system level with all packages and support files involved:

System diagram for the framework

When a Detox test is run

  1. Yarn is used to run Cucumber.js along with various arguments
  2. Cucumber.js executes tests.setup.js, which creates a shared JS context object capturing command line options and pulls in the cucumber init files
  3. Cucumber.init.ts setups up Detox, configures and runs a server, and hooks into Cucumber for results reporting
  4. Tests are now run with Detox executing actions against an appropriate simulator. It can do this since the installed app has been built with Detox instrumentation.
  5. Results are reported to the console

When an Appium test is run

  1. Yarn is used to run WebdriverIO along with various arguments, including the os type which is required
  2. WebdriverIO executes tests.setup.js, which creates a shared JS context object capturing command line options and pulls in the WebdriverIO config file
  3. Wdio.conf.ts sets the capabilities of the device to run, sets up Cucumber, and configures/starts the Appium server
  4. Appium talks to the device, installs an app for executing actions against the app under test
  5. Tests are now run with WebdriverIO talking to the Appium server, which executes actions on the device
  6. Results are reported to the console

Things to note

  • The user can manually start an Appium server instead of the framework. This is useful to see what it is doing, and can be faster when running multiple tests.
  • Local real devices can be used instead of simulators. This is trickier with Detox, and has not been fully investigated.
  • Running in AWS is left to the future, and will require packaging everything up and uploading it there

When to use Detox or Appium

You will normally want to use Detox locally because:

  • Detox is faster
  • Detox is easier to work when developing
  • Detox seems to be a bit less temperamental when running, probably due to fewer moving parts

However, Appium does have some advantages:

  • Appium has more capabilities than Detox.
    • One big one is Detox can only find elements but not get any info about them. You can’t get the text of an element to verify with Detox, but you can with Appium.
    • Appium is much better at dealing with native OS things like action sheets, and gives more control over them
  • Appium is probably better for running on real devices
  • Appium can be used to the web (future consideration)
  • Appium is supported by the AWS Device Farm and probably others since it is a standard

Code Level Architecture

Here is a diagram of the internals of the actual framework:

Class diagram of the framework internals

  • Features – The tests are written here as plain language using the Gherkin syntax. Each feature file should focus on one particular area.
  • Step Definitions – These are cucumber files that link the feature keywords to actual code. There is one file per feature file with a common shared one.
  • Data – Test data is pulled into the steps using factories, but only one for user credentials currently exists.
  • Pages – Each page file represents one screen of the app. It defines the various elements, their testIDs, and the phrases used to identify them in the feature tests. To ease use, a factory exists that can provide a page object based on a phrase.
  • Actions – These classes implement a set of standard actions in a driver specific format. A factory exists to provide the proper one based on if Detox or Appium is used.
  • World Context – This is a global object that captures any command line arguments and other info such as the framework type and os. Cucumber has a native version of this, but it is only accessible by the step definitions.
  • Loggers – Global loggers to give details about Cucumber, Detox, and Appium actions. The normal level is DEBUG since that provides enough info to troubleshoot failures.

How it works

There are differences in the specifics, but generally Detox and Appium work the same. The tester runs a yarn command (predefined or ad-hoc) which launches packages that pulls configuration files in to set up Detox/Appium and also interpret Cucumber tests. In each test, the plain language of the feature file is linked to actual code via step definitions. The actual code is organized per screen with a standard set of actions available. During setup, the proper concrete implementations of the actions using Detox or Appium commands are linked to these page objects. Both Detox and Appium use their own servers to interact with the simulator or device under test, but via different mechanisms. Their results are interpreted by the Cucumber runner and reported back via the console.

When a command is executed Detox/Appium examines the current screen and tried to locate the element to act on by the testID attribute. This is a special React attribute for the specific purpose of automation. It is a unique identifier defined in the product code for elements used by automation. Since this testID is not very user friendly, a mechanism exists in the page objects that links plain language phrases to the testIDs. The same is also true for identifying which screen (and associated page object) to use in the actual tests.

Conclusion

This may not be the most in depth article, but hopefully it shows how you can build an automation framework that incorporates different technologies in a decoupled way. This allows for flexibility in the vehicles that actually drive an app but also how you can write the tests above.

As mentioned in the note at the beginning, some details will be covered in other posts and the code details in a series on what this eventually evolved into.

References

  • Mapping Gherkin phrases to page locators – Blog post
  • System to add automatic testIds to React code – Blog post


This content originally appeared on DEV Community and was authored by Felicia Walker