How To Run Your First Playbook¶
For executing prepared tests you need to configure your environment first.
Following packages should be installed:
sudo add-apt-repository ppa:gijzelaar/opencv2.4 sudo apt-get update sudo apt-get install maven xvfb openjdk-8-jdk recordmydesktop wmctrl xdotool build-essential libopencv2.4-java libcv-dev libtesseract3
Unfortunately latest versions of Mozilla Firefox are not supported by Selenium tests.
We have to use Firefox with version 45 with our current test setup.
It should be fixed in the future version.
If you want to start a prepared test, you need to use following script:
$ ./run_tests_qa.sh ====================================================================== Script for run the acceptance tests! Parameters: -m Management Host First: IP -M Management Host Second: IP -l Observe List of All Playbooks -L Observe List Playbooks for run -s Choice of Playbooks for run -s all Start all Playbooks -s "playbook1.story playbook2.* ... " Start a few Playbooks -r Start acceptance tests -h Help ======================================================================
For example, if you have installed Subutai Social management host on
192.168.0.221, you need to point this script to this server with
$ ./run_tests_qa.sh -m 192.168.0.221
It will add your IP-address to the configuration of tests.
Then you need to choose the test to be executed, or just configure all
tests to be executed:
./run_tests_qa.sh -s all
You're all set now with everything you need. To start testing use the
A new Firefox browser instance should be started and the test will be executed based on the steps in your Story file. After that the results will appear in the console output.
Getting Test Results¶
When test execution completes you can observe a detailed report generated by Serenity.
The report will be placed in the following file:
And you can get detailed information including timings, screenshots, and errors if any occur by using the navigation controls in this report:
After automated tests execute you can get an exit code to indicate whether the test ran correctly or not to use it for downstream logic: * 0 - if everything finished successfully; * non 0 - if you got some error.
Based on the exit code you can build any automated sequence of acceptance tests by combining the list of playbooks and analyzing their results.
Regression tests can use a standard list of tests to approve each new build. Based on test results the automation system can assess whether or not to accept or decline a build. Using this approach any CI system (i.e. Jenkins) can be used to drive behavioral (functional) testing in an automated fashion.