Setting Up Continuous Integration
Once the Build File is ready and Version Control has been set up (if wanted), tests can be scheduled using a Continuous Integration Server such as Jenkins.
The role of the CI Server is to run the regression pack (from a local machine or via a version control repository). The CI Server will trigger the Provar automation pack on a defined schedule or based on an external trigger, such as a new build or deployment.
Start by downloading and installing Jenkins.
To download Jenkins, visit the Jenkins website. Click the ‘Download’ button and select the file which suits your operating system.
To install Jenkins, first move the downloaded Jenkins.war file to an appropriate directory, e.g. C:\Program Files\Jenkins:
Then open command prompt in this directory by clicking on the filepath and typing ‘cmd’:
Once command prompt has opened, type and run the command java –jar Jenkins.war.
Once the command has run, a password will be generated for logging into Jenkins:
In this example the password is cd3f375e0baa47f4b4a5591f1e920bfe.
Note that this password can also be checked by referring to the InitialAdminPassword.txt file which has been created in the \.jenkins\secrets folder:
Next, open your browser and navigate to URL: localhost:8080:
The following screen will appear:
Enter your password then click the ‘Continue’ button.
On the next screen, choose the option ‘Select plugins to install’:
Some plugins will be defaulted already. These can be left as defaulted.
Make sure that the following plugins are ticked:
Ant Plugin (under Build Tools):
JUnit Plugin (under Build Analysis and Reporting):
If you want Jenkins to download the regression pack from your Version Control repository, also tick the relevant plugin under Source Code Management (e.g. the Git plugin):
Then click the ‘Install’ button.
When this is completed, click the ‘Continue as admin’ button on the following screen (instead of filling out ‘Create First Admin User’):
Note that, the next time you log in, your username will be ‘admin’ and your password will be the one generated earlier.
On the next screen, click the button ‘Start using Jenkins’ to complete the setup.
Scheduling a Run
To schedule test in Jenkins, start by clicking the ‘New Item’ option on the left-hand side:
Provide a name for the new project, e.g. ‘ProvarProject’, then select the ‘Freestyle project’ option and click the ‘OK’ button.
If you are using version control, navigate to Source Code Management and select ‘Git’. (If you are not using version control, skip ahead to Build Triggers).
Enter the ‘Repository URL’, which can be copied by clicking the ‘Clone or Download’ button against the repository.
Then provide Credentials (if applicable – this is only needed for private repositories):
To define the execution schedule, navigate to Build Triggers. Here you can define when you want Provar tests to be run. Choose ‘Build periodically’ if you want to set up a schedule.
(Alternatively, you can also select ‘Trigger builds remotely’ or ‘Build after other projects are built’ if you want to trigger the execution using a script or following completion of other project builds.)
If you choose the ‘Build periodically option, you will then need to enter an expression in ‘Schedule’ to define when the tests should be run. This is done using a CRON expression:
To create this expression, you can write your own from scratch or copy or adapt one of the examples below (taken from here):
H H(3-4) * * *
Schedule: Every Day of the week, every Month in a year, every Day in a month at a time between 3 – 4 am in the morning at any minute.
H (30 -45) 3 * * *
Schedule: every Day of the week, every Month in a year, every Day in a month at 3 am in the morning between 30 -45 minutes.
*/5 * * * * :
Schedule: every 5 minutes.
0 8 * * *
Schedule: every day at 8h 00.
Enter your preferred syntax in ‘Schedule’, then check the validation message underneath to confirm the syntax is correct and the schedule has been defined as expected.
The ‘Would last have run at…’ information will show the most recent (past) run and next upcoming run, based on your syntax:
Finally, navigate to Build and click the ‘Add build step’ dropdown, then choose ‘Invoke ANT’.
Leave ‘Targets’ blank, unless you know that your Build File has multiple targets. (If your Build File was generated using the default settings supplied in Run Under ANT, leaving ‘Targets’ blank will work fine.)
If you have modified the Build File so that there are multiple targets, enter the name of the desired target, e.g. ‘runtests’.
Then choose the ‘Advanced…’ option and provide the build.xml path.
If the project is fetched from git, it should use the JENKINS_HOME variable (see screenshot above). If the project is local, it can be a filepath, e.g. C:\Users\Hetty\Documents\GitHub\dev\DevOps\ANT\build.xml:
Click the ‘Save’ button.
Test your Build by clicking the ‘Build Now’ option in the left-hand sidebar:
Build progress will be shown within Jenkins; if the build is successful, results will also be emailed as defined in the original Build File.
Configuring Test Reports
Once your build generates test results, Jenkins’ JUnit plugin can be used to display these results graphically:
When Provar tests are executed via Continuous Integration, Provar automatically generates a JUnit.xml file alongside its HTML and PDF reports. Using the JUnit plugin, this JUnit.xml file can be converted into a graphical visualization of the test results. The plugin also provides a web UI for viewing test reports and tracking failures.
Since Provar generates the JUnit.xml file automatically, very little further setup is required. One thing to keep in mind that the Results folder, which contains this JUnit.xml, should be generated or placed in Jenkins’ workspace folder.
For example: If the project is inside Jenkins’ workspace, define the results path as:
Follow the steps below to get the JUnit plugin up and running.
In Jenkins, click on the dropdown arrow on your Job Name and select ‘Configure’:
Navigate to ‘Post-build Actions’, then click the ‘Add post-build action’ dropdown and select ‘Publish JUnit test result report’.
In ‘Test report XMLs’, set the path of the JUnit.xml file (note that this is generated inside your Results folder). This XML file should also be inside the workspace of the job of Jenkins. Enter the relative path of the results folder generated after execution:
Note that, before the first execution of your build, this field will show an error, since the file is not generated until after the execution.
Then save and trigger the job.
Once execution completes, you can view JUnit results by clicking on Test Result on the Execution View screen:
On the next screen, a ‘History’ link is available to show the historical view or trend of execution. Note that historical data only be visible until two or more execution cycles.
This completes your Continuous Integration setup.