CI Integration
Test Runs can be automatically started from your CI tool (Github Actions, Gitlab Runner, Jenkins etc) by using HyperTest CLI.
To achieve this:
- 1.Edit or add a job/pipeline in your CI tool which triggers on every PR event
- 2.Build, Deploy and test your test application with new changes from PR
- 3.Expose env variables required for Hypertest CLI to run a test (Check step 2 for this)
- 4.Download HyperTest CLI
- 5.Trigger a new test run from CLI
The above flow will ensure that every time a new PR is raised, tests are automatically triggered in HyperTest.
Note: you should add this as a cacheable step in your CI tool
Expose Variables and run below commands:
wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz
./hypertest can-run-test
./hypertest start-new-test
Refer below sample CI files for different platform pipelines
wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz
- 1.
HT_BASE_URL``HT_BASE_URL is the url of your Hypertest Dashboard
HT_BASE_URL=http://hypertest-ip-or-hostname:dashboard-port You can also export HT_BASE_URL directly in the command as well. Example:HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test
- 2.HT_API_TOKEN (for kubernetes setup) HT_API_TOKEN is the api token required for authentication while starting test runs. You can find it in your central dashboard under user management. Go to API Tokens and download the default cli token to get the token
Although we automatically pick up these parameters if you're using GitHub Actions or Gitlab Runner, in case you use Jenkins or trigger your CI jobs manually then you should set these env variables:
- 1.
CIShould be set to true for enabling CI integration - 2.
HT_COMMIT_HASHIt's a unique identifier hash generated for your commit - 3.
HT_BRANCHThe source branch name of the pull request - 4.
HT_PR (Required for Gitlab and BitBucket)Pull request number, in the case of Gitlab Internal Merge Request Id should be passed \
export HT_BASE_URL='<http://hypertest-vm-ip:dashboard-port>'
export CI='true'
export HT_COMMIT_HASH='<your commit hash>'
export HT_BRANCH='<your source branch name of the PR>'
export HT_PR='<PR number (this is required only for gitlab and bitbucket)>'
export HT_API_TOKEN='<you api token>'
Hypertest CLI is ready to be used for automatically starting tests.
Below are some of the commands that you can use to verify settings, start tests, see the progress of tests, etc
Check if a new test can be started on hypertest.
Verify if all parameters are set and you can start running the test on Hypertest
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test
./hypertest can-run-test --help
Usage: hypertest can-run-test [options]
checks if a new test can be started on hypertest. Using --poll will check until a new test can be started
Options:
--poll polls hypertest until a new test can be started (default: false)
--interval <interval> poll interval in seconds (default 60) (default: 60)
--time-limit <time-limit-in-seconds> max duration to wait for execution (default 3600) (default: 3600)
--git-dir <path/conatinting/.git/directory> useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
-h, --help display help for command
Start a new test on HyperTest
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest start-new-test
./hypertest start-new-test --help
Usage: hypertest start-new-test [options]
starts a new test on hypertest
Options:
-d, --delay <delay-time-in-seconds> delay in seconds to wait before starting a new test (default: 0)
-p, --primary <primary-base-url> base url for stable(primary) version
-c, --candidate <candidate-base-url> base url for test(candidate) version
-i, --initial-timestamp <initial-timestamp> traffic would be considered from here. (calculated from test_window by default) (Should be parsable by new Date(val) in javascript)
-f, --final-timestamp <final-timestamp> traffic would be considered until here. (current time by default) (Should be parsable by new Date(val) in javascript) (default: "2022-09-13T12:20:29.979Z")
-t, --testcase-ids <testcase-ids> run test for selected testcases. (comma separated testcase ids)
--no-stop-running-tests dont start a suite if one is already running. (default: terminates old and starts new)
--stop-on-first-failure stops test execution on first failure (default: complete entire test execution) (default: false)
--smoke-only only run test cases saved as smoke tests (default: false)
--saved-only only run saved test cases (default: false)
--play-duplicate-requests run duplicate requests within a session (default: false)
--play-duplicate-sessions run duplicate sessions (default: false)
--description <description> test case description
--interval <interval> poll interval in seconds (default 60) (default: 60)
--skip-wait-for-old-tests does not wait for old running tests to complete. Running tests terminated by default or program fails if --no-stop-running-tests is passed (default: false)
--wait-for-result wait for test to complete (just starts the test by default) (default: false)
--ht-env <ht-env> test ht env
--minimum-severity <minimum-severity> minimum severity for differences
--signoff-required Is accepting/rejecting regressions required after test results are generated. (Useful in CI integration) (default: true)
--git-dir <path/conatinting/.git/directory> useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
--time-limit <time-limit-in-seconds> max duration to wait for execution (default 3600) (default: 3600)
-h, --help display help for command
Find the test progress of a given testId
# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-test-progress -wait-for-result
./hypertest get-test-progress --help
Usage: hypertest get-test-progress [options]
find test progress of given testId
Options:
-t --test-id <test-id> test id to get test result
--interval <interval> poll interval in seconds (default 60) (default: 60)
--time-limit <time-limit-in-seconds> max duration to wait for execution (default 3600) (default: 3600)
--wait-for-result wait for test to complete (just display current progress by default) (default: false)
-h, --help display help for command
Find signoff state of last test
# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-signoff-status --wait-for-signoff
./hypertest get-signoff-status --help
Usage: hypertest get-signoff-status [options]
find signoff state of last test
Options:
-t --test-id <test-id> test id to get signoff state
--interval <interval> poll interval in seconds (default 60) (default: 60)
--time-limit <time-limit-in-seconds> max duration to wait for execution (default 604800) (default: 604800)
--wait-for-signoff wait for test to signoff (just starts the test by default) (default: false)
-h, --help display help for command
Access the help options for all commands by using
-h or --help options./hypertest --help
Usage: hypertest [options] [command]
Options:
-v, --version output the current version
-h, --help display help for command
Commands:
start [options] start new hypertest instance
stop [options] stop hypertest
update [target-version] update hypertest
update-cli [target-version] update hypertest
restart restarts hypertest
bg-start forcefully starts a new background service by killing previous one if running
bg-test test if background service is running
bg-kill forcefully kills background service if running
can-run-test [options] checks if a new test can be started on hypertest. Using --poll will check until a new test can be started
start-new-test [options] starts a new test on hypertest
get-test-progress [options] find test progress of given testId
get-signoff-status [options] find signoff state of last test
get-signoff-status-by-commit [options] find signoff state in last test of given commit id
tunnel [options] establish a tunnel for your service without exposing it on the public internet
help [command] display help for command
Last modified 2mo ago