How To Run Your First Playbook

Creating Environment

For executing prepared tests you need to configure your environment first.

Following packages should be installed:

sudo add-apt-repository ppa:gijzelaar/opencv2.4

sudo apt-get update
sudo apt-get install maven xvfb openjdk-8-jdk recordmydesktop wmctrl xdotool build-essential libopencv2.4-java libcv-dev libtesseract3

Firefox Version

Unfortunately latest versions of Mozilla Firefox are not supported by Selenium tests.

We have to use Firefox with version 45 with our current test setup.

It should be fixed in the future version.

Running Tests

If you want to start a prepared test, you need to use following script:

$ ./run_tests_qa.sh
======================================================================
Script for run the acceptance tests!

Parameters:
  -m          Management Host First:  IP
  -M          Management Host Second: IP
  -l          Observe List of All Playbooks
  -L          Observe List Playbooks for run
  -s          Choice of Playbooks for run
  -s all      Start all Playbooks
  -s "playbook1.story playbook2.* ... " Start a few Playbooks
  -r          Start acceptance tests
  -h          Help
======================================================================

For example, if you have installed Subutai Social management host on 192.168.0.221, you need to point this script to this server with following command:

$ ./run_tests_qa.sh -m 192.168.0.221

It will add your IP-address to the configuration of tests.

Then you need to choose the test to be executed, or just configure all tests to be executed: ./run_tests_qa.sh -s all

image

image

You're all set now with everything you need. To start testing use the following command: ./run_tests_qa.sh -r

image

image

A new Firefox browser instance should be started and the test will be executed based on the steps in your Story file. After that the results will appear in the console output.

Getting Test Results

When test execution completes you can observe a detailed report generated by Serenity.

The report will be placed in the following file: target/site/serenity/index.html.

image

image

And you can get detailed information including timings, screenshots, and errors if any occur by using the navigation controls in this report:

image

image

Automation

After automated tests execute you can get an exit code to indicate whether the test ran correctly or not to use it for downstream logic: * 0 - if everything finished successfully; * non 0 - if you got some error.

Based on the exit code you can build any automated sequence of acceptance tests by combining the list of playbooks and analyzing their results.

Regression tests can use a standard list of tests to approve each new build. Based on test results the automation system can assess whether or not to accept or decline a build. Using this approach any CI system (i.e. Jenkins) can be used to drive behavioral (functional) testing in an automated fashion.