Suppose you are to setup test automation. Building and maintaining a whole test infrastructure all by your team can be painful. Although some cloud services like Sauce Labs do cover DevOps, you may have hesitated to use them due to security issues or budget concerns in your company. Docker is a good tool for you to setup and maintain servers for test automation, especially if you are just starting to build an automation test infrastructure with open source solutions.

Of the open source solutions, Selenium is an open source framework for automating UI tests. It allows you to simulate user flow by executing scripts on different browsers. Selenium-Grid consists of hub servers and node servers to let you execute tests in parallel to speed up testing. When a hub receives requests from clients, the requests are rerouted to an appropriate node server, based on the capability parameters.

Appium is a mobile test framework that supports native apps, webviews and hybrid apps on iOS and Android. Appium connects Android devices through Android Debug Bridge (ADB) and uses the Android UiAutomator framework, so you can make interactions on mobile apps to simulate user action flow, and send system commands to the device. Appium also allows you to test with multiple devices at the same time by configuring capability parameters in the request payload as desired.

Selenium and Appium provide APIs for clients to send requests to Selenium servers and Appium servers. Once client requests are received, the servers send commands to the corresponding drivers to control your testing devices or browsers according to the scenario you designed.

The following diagram illustrates a test system infrastructure for testing both web based UI and mobile. To support both web based UI and mobile testing, you need to prepare the remote servers—Selenium-hub servers and Appium servers—before executing test scripts.

Docker containers are easy to create and destroy. For those of you who are unfamiliar with Docker, Docker is a platform which allows you to pack your app in a container with all its dependencies. In the following sections, we would like to talk about how to manipulate Docker to build up a test runner, Selenium-Grid, and Appium servers to compose a flexible and disposable test infrastructure.

Test runner container

Whichever test framework or test library you are using, you need to consider the library version, environment variable settings and other dependencies when implementing and executing a test. A test script is a program that sends requests to the server to execute commands on the driver you want to test against. To enable anyone in your team execute the test on any platform, you can use Docker to pack all test scripts and libraries as a test runner container. The container is an isolate environment that includes all environment settings and libraries, so the test scripts can be run on any platform. Here is an example of a Dockerfile to make a simple test runner Docker image. Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.

# Use Alpine as the base image
FROM python:alpine

# Add required libraries
RUN apk --update add 
    ca-certificates 
    py-pip 
    curl 
   && pip2 install --upgrade pip

# Set environment variables
ENV PROJECT_NAME=myproject
ENV ENV_FILES=/${PROJECT_NAME}/env

# Add required libraries
COPY ./requirements.txt /${PROJECT_NAME}
RUN pip2 install -r /${PROJECT_NAME}/
WORKDIR /${PROJECT_NAME}
COPY / /${PROJECT_NAME}

The parts of the Dockerfile are as the following:

• Base image: We are using Alpine as base image based on the best practice of designing Dockerfile.
• Required libraries: Define all of the required libraries. Since Python is our primary language of test framework, to install required Python libraries, we are using the pip install command with the requirements.txt file which contains specific versions of the Python libraries.
• Environment variables (optional): Configure environment variables with the ENV command, if there are any environment variable dependencies in the test.
• Scripts (optional): Optionally, you can put test scripts in the container and ship it to any environment. Or, you can run the test by mounting the script files folder at runtime. To run the test after you build your own test runner container, run the container with the following command.

  docker run --rm -v $(pwd)/report:/myproject/report --name test_runner test-runner:latest pybot -V env/beta.py -d report -v REMOTE_URL:http://$SELENIUM_SERVER_IP:4444/wd/hub testsuites/ui_test/

  With the --rm option, the test runner container will be removed after completing the test. In addition, you can keep the test result even after the container is destroyed by mounting the folder $(pwd)/report on your host machine.

Selenium-Grid container

Selenium-Grid container consists of a Selenium Hub server and node servers. It allows you to run your tests in parallel; different tests can be run at the same time on different node servers. Selenium Hub is the main command center for managing which machines your Selenium test will run on. To run a Selenium test, you specify capability parameters in the payload of the request and send the request to the Hub. The hub, then reroutes the test to the node server according to the browser version you have specified to run the test against.

In general, you would install Selenium Hub and node servers over multiple machines. Scaling up, i.e. adding more node servers to support increasing test cases is not easy. However, Selenium provides an official Docker image for Selenium Hub and node servers respectively. By running Selenium servers (hub and node) in containers, you can easily setup and configure the Hub server, and also scale up the number of node server containers. You can see the details in their Github repository on how to setup Selenium-Grid with Docker. Furthermore, there are some open source projects that provide extra functions for Selenium-Grid by extending the Selenium Docker image. Zalenium is an open source project contributed by Zalando, announced in the SeleniumConf Austin 2017. Zalenium aims to provide a flexible and disposable Selenium-Grid server that anyone can use. Here is an example on deploying Zalenium with Docker command on Linux.

docker run --rm -ti --name zalenium -p 4444:4444 
  -v $(which docker):/usr/bin/docker 
  -v /var/run/docker.sock:/var/run/docker.sock 
  -v /tmp/videos:/home/seluser/videos 
  --privileged dosel/zalenium start

Zalenium uses the Docker on Docker technique to create node containers on demand. For example, by sharing a docker socket, Zalenium is able to create node containers with appropriate Selenium-WebDriver when the Selenium Hub receives multiple requests at the same time. All node containers will be destroyed after completing the test. In addition, Zalenium includes VNC (Virtual Network Computing) and video recording which are really convenient for troubleshooting and makes the test just running on the local.

If you are interested in how to parallel Selenium test with Zalenium, you can get more information in our DevDay 2017 talk - Parallel Selenium Test With Docker.

Appium container

Appium provides REST APIs to let clients (the test runner) send requests to the server. When the server receives a request, the server sends a corresponding command to the Android device. Suppose an Appium server receives a request to simulate user actions on an Android app running on an Android device. The Appium server will connect to the Android device through ADB, and then start the UiAutomator process and send action requests to it.

There are some public Docker images in Docker Hub such as appium/appium which contains Appium core server, ADB and all of Appium's dependencies. You can easily set up an Appium server as a remote server to perform mobile tests, by running the following command on Unix.

docker run --rm --privileged -d -p 4723:4723  -v /dev/bus/usb:/dev/bus/usb --name container-appium appium/appium

By mounting a USB port on the host machine, the ADB inside the container is able to connect with the device connected to the host machine. The container will be destroyed if you run the container with the --rm option in command line.

On OS X, you can run a container inside a docker-machine, which allows adb to connect to the Android device through USB as well.

Such isolated environment enables manipulating building a simple device farm with multiple devices. The concept is that each docker-machine has one Appium server and ADB running in the Docker container, and the ADB inside the container connects to one mobile device through USB. You can make use of the Appium server IP that corresponds to the Android device that matches the test criteria such as Android API version and device brand. In addition, you can connect the device via remote tools like Vysor through WiFi connection, so you can have a live view of the device while running a test on the device.

Conclusion

Docker is not only for software development but can be manipulated for software testing. It lets you to setup and scale out remote servers either for web UI or mobile testing. By having an isolated and stable environment, everyone can perform the test inside a container to verify the system functions at any development stage. All containers in this infrastructure can be created on demand and destroyed when the test is finished. It makes the test infrastructure flexible and maximize the machine resource. Try to use Docker to build a flexible and disposable test infrastructure for web UI and mobile testing.