CI Integration

Test Runs can be automatically started from your CI tool (Github Actions, Gitlab Runner, Jenkins etc) by using HyperTest CLI.
Brief Steps:
To achieve this:
1.Edit or add a job/pipeline in your CI tool which triggers on every PR event
2.Build, Deploy and test your test application with new changes from PR
3.Expose env variables required for Hypertest CLI to run a test (Check step 2 for this)
4.Download HyperTest CLI
5.Trigger a new test run from CLI
The above flow will ensure that every time a new PR is raised, tests are automatically triggered in HyperTest.
Note: you should add this as a cacheable step in your CI tool
Expose Variables and run below commands:
wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz
./hypertest can-run-test
./hypertest start-new-test
Sample files for CI/CD
Refer below sample CI files for different platform pipelines
Github-actions can be found here​
Gitlab CI/CD can be found here​
Bitbucket Pipelines can be found here​
Jenkins CI/CD can be found here​
Getting Started
1. Download HyperTest CLI
wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz
2. Set env variable in your CI job
Required env variables
1.HT_BASE_URL
``HT_BASE_URL is the url of your Hypertest Dashboard
HT_BASE_URL=http://hypertest-ip-or-hostname:dashboard-port
You can also export HT_BASE_URL directly in the command as well.
Example: HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test
2.HT_API_TOKEN (only required for HT setup that are on Kubernetes, not required for linux VM based HT setups)
HT_API_TOKEN is the api token required for authentication while starting test runs. You can find it in your central dashboard under user management. Go to API Tokens and download the default cli token to get the token


Other env variables
Although we automatically pick up these parameters if you're using GitHub Actions or Gitlab Runner, in case you use Jenkins or trigger your CI jobs manually then you should set these env variables:
1.CI
Should be set to true for enabling CI integration
2.HT_COMMIT_HASH
It's a unique identifier hash generated for your commit
3.HT_BRANCH
The source branch name of the pull request
4.HT_PR (Required for Gitlab and BitBucket)
Pull request number, in the case of Gitlab Internal Merge Request Id should be passed
\
export HT_BASE_URL='<http://hypertest-vm-ip:dashboard-port>'
export CI='true'
export HT_COMMIT_HASH='<your commit hash>'
export HT_BRANCH='<your source branch name of the PR>'
export HT_PR='<PR number (this is required only for gitlab and bitbucket)>'
export HT_API_TOKEN='<you api token> (only required for K8s based setup)'
Hypertest CLI is ready to be used for automatically starting tests.

Below are some of the commands that you can use to verify settings, start tests, see the progress of tests, etc
3. Check if HyperTest can run a test (optional)
Check if a new test can be started on hypertest.
Verify if all parameters are set and you can start running the test on Hypertest
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test
./hypertest can-run-test --help
./hypertest can-run-test --help
Usage: hypertest can-run-test [options]
​
checks if a new test can be started on hypertest. Using --poll will check until a new test can be started
​
Options:
  --poll                                       polls hypertest until a new test can be started (default: false)
  --interval <interval>                        poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>         max duration to wait for execution (default 3600) (default: 3600)
  --git-dir <path/conatinting/.git/directory>  useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
  -h, --help                                   display help for command
4. Start a new test
Start a new test on HyperTest
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest start-new-test
./hypertest start-new-test --help
./hypertest start-new-test --help
Usage: hypertest start-new-test [options]
​
starts a new test on hypertest
​
Options:
  -d, --delay <delay-time-in-seconds>          delay in seconds to wait before starting a new test (default: 0)
  -p, --primary <primary-base-url>             base url for stable(primary) version
  -c, --candidate <candidate-base-url>         base url for test(candidate) version
  -i, --initial-timestamp <initial-timestamp>  traffic would be considered from here. (calculated from test_window by default) (Should be parsable by new Date(val) in javascript)
  -f, --final-timestamp <final-timestamp>      traffic would be considered until here. (current time by default) (Should be parsable by new Date(val) in javascript) (default: "2022-09-13T12:20:29.979Z")
  -t, --testcase-ids <testcase-ids>            run test for selected testcases. (comma separated testcase ids)
  --no-stop-running-tests                      dont start a suite if one is already running. (default: terminates old and starts new)
  --stop-on-first-failure                      stops test execution on first failure (default: complete entire test execution) (default: false)
  --smoke-only                                 only run test cases saved as smoke tests (default: false)
  --saved-only                                 only run saved test cases (default: false)
  --play-duplicate-requests                    run duplicate requests within a session (default: false)
  --play-duplicate-sessions                    run duplicate sessions (default: false)
  --description <description>                  test case description
  --interval <interval>                        poll interval in seconds (default 60) (default: 60)
  --skip-wait-for-old-tests                    does not wait for old running tests to complete. Running tests terminated by default or program fails if --no-stop-running-tests is passed (default: false)
  --wait-for-result                            wait for test to complete (just starts the test by default) (default: false)
  --ht-env <ht-env>                            test ht env
  --minimum-severity <minimum-severity>        minimum severity for differences
  --signoff-required                           Is accepting/rejecting regressions required after test results are generated. (Useful in CI integration) (default: true)
  --git-dir <path/conatinting/.git/directory>  useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
  --time-limit <time-limit-in-seconds>         max duration to wait for execution (default 3600) (default: 3600)
  -h, --help                                   display help for command
5. Wait for results (waits for test to be over)
Find the test progress of a given testId
# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-test-progress -wait-for-result
./hypertest get-test-progress --help
./hypertest get-test-progress --help
Usage: hypertest get-test-progress [options]
​
find test progress of given testId
​
Options:
  -t --test-id <test-id>                test id to get test result
  --interval <interval>                 poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>  max duration to wait for execution (default 3600) (default: 3600)
  --wait-for-result                     wait for test to complete (just display current progress by default) (default: false)
  -h, --help                            display help for command
6. Wait for qa signoff (waits until results are accepted/rejected)
Find signoff state of last test
# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-signoff-status --wait-for-signoff
./hypertest get-signoff-status --help
./hypertest get-signoff-status --help
Usage: hypertest get-signoff-status [options]
​
find signoff state of last test
​
Options:
  -t --test-id <test-id>                test id to get signoff state
  --interval <interval>                 poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>  max duration to wait for execution (default 604800) (default: 604800)
  --wait-for-signoff                    wait for test to signoff (just starts the test by default) (default: false)
  -h, --help                            display help for command
Other CLI options
Access the help options for all commands by using -h or --help options
./hypertest --help
./hypertest --help
Usage: hypertest [options] [command]
​
Options:
  -v, --version                           output the current version
  -h, --help                              display help for command
​
Commands:
  start [options]                         start new hypertest instance
  stop [options]                          stop hypertest
  update [target-version]                 update hypertest
  update-cli [target-version]             update hypertest
  restart                                 restarts hypertest
  bg-start                                forcefully starts a new background service by killing previous one if running
  bg-test                                 test if background service is running
  bg-kill                                 forcefully kills background service if running
  can-run-test [options]                  checks if a new test can be started on hypertest. Using --poll will check until a new test can be started
  start-new-test [options]                starts a new test on hypertest
  get-test-progress [options]             find test progress of given testId
  get-signoff-status [options]            find signoff state of last test
  get-signoff-status-by-commit [options]  find signoff state in last test of given commit id
  tunnel [options]                        establish a tunnel for your service without exposing it on the public internet
  help [command]                          display help for command
Automate Starting a Test
GitHub Checks Integration
Last modified 11d ago