Handling global failure in CI
When integrating Hiptest with a CI tool, it may happen that the test run is not fully synchronised with the status of the last build. This mainly happens when the tests have not been ran (for example due to a installation or compilation error during the setup process).
In such cases, as there is no test results to push at the end of the build, Hiptest will consider only the status from the previous build. So it might still show green statuses even if the last build failed.
There is a simple way to fix this kind of issues.
If you use hiptest-publisher to generate the test scripts and push the results, you can simply add the following line in you config file:
global_failure_on_missing_reports = true
This will ensure that, when no results files have been generated, Hiptest will be notified that there was a global failure and the tests will all be marked as fail.
Note that you should only add the line once the full process has been shown as reliable (so once you have a few successful builds that pushed results back to Hiptest).
If you want to push results for tests that were not written in Hiptest, you can use another cUrl command to notify a global failure of the build.
Below is an example of a post-failure action written with bash:
#!/bin/bash FILE=<path to you result file> SECRET_TOKEN=<your hiptest secret token> TEST_RUN_ID=<the test run to which you want to push results> if [ -f $FILE ]; then curl -X POST -F file=@$FILE https://hiptest.net/import_test_reports/$SECRET_TOKEN/$TEST_RUN_ID/junit else curl -X POST https://hiptest.net/report_global_failure/$SECRET_TOKEN/$TEST_RUN_ID/ fi
This script ensures that, if the result file is present, it will be pushed to Hiptest to update the results in the test run and otherwise, declare a global failure.