CI Integration

Test Runs can be automatically started from your CI tool (Github Actions, Gitlab Runner, Jenkins etc) by using HyperTest CLI.

Brief Steps:

To achieve this:

Edit or add a job/pipeline in your CI tool which triggers on every PR event
Build, Deploy and test your test application with new changes from PR
Expose env variables required for Hypertest CLI to run a test (Check step 2 for this)
Download HyperTest CLI
Trigger a new test run from CLI

The above flow will ensure that every time a new PR is raised, tests are automatically triggered in HyperTest.

Note: you should add this as a cacheable step in your CI tool

Expose Variables and run below commands:

wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz
./hypertest can-run-test
./hypertest start-new-test

Sample files for CI/CD

Refer below sample CI files for different platform pipelines

Github-actions can be found here
Gitlab CI/CD can be found here
Bitbucket Pipelines can be found here
Jenkins CI/CD can be found here

Getting Started

1. Download HyperTest CLI

wget -O ht-cli.tar.xz https://hypertest-binaries-1.s3.ap-south-1.amazonaws.com/ht-cli/ht-cli-latest.tar.xz
tar xvf ht-cli.tar.xz

2. Set env variable in your CI job

Required env variables

HT_BASE_URL ``HT_BASE_URL is the url of your Hypertest Dashboard

HT_BASE_URL=http://hypertest-ip-or-hostname:dashboard-port You can also export HT_BASE_URL directly in the command as well. Example: HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test

HT_API_TOKEN (only required for HT setup that are on Kubernetes, not required for linux VM based HT setups) HT_API_TOKEN is the api token required for authentication while starting test runs. You can find it in your central dashboard under user management. Go to API Tokens and download the default cli token to get the token

Other env variables

Although we automatically pick up these parameters if you're using GitHub Actions or Gitlab Runner, in case you use Jenkins or trigger your CI jobs manually then you should set these env variables:

CI
Should be set to true for enabling CI integration
HT_COMMIT_HASH
It's a unique identifier hash generated for your commit
HT_BRANCH
The source branch name of the pull request
HT_PR (Required for Gitlab and BitBucket)
Pull request number, in the case of Gitlab Internal Merge Request Id should be passed
SCM Specific Variables These variables are required to give us access to write checks on your PR in your SCM. You can either pass these SCM related Variables in your CI pipelines as enviornment variables or you can add them in HyperTest Dashboard's Service Configuration Page. Please refer SCM intergration (gitlab, github, bitbucket) guides for more details.

export HT_BASE_URL='<http://hypertest-vm-ip:dashboard-port>'
export CI='true'
export HT_COMMIT_HASH='<your commit hash>'
export HT_BRANCH='<your source branch name of the PR>'
export HT_PR='<PR number (this is required only for gitlab and bitbucket)>'
export HT_API_TOKEN='<you api token> (only required for K8s based setup, not for Linux based HT Setup)'

# SCM Specific Configuration details to give HyperTest access to publish reports on your PR
# If you are using Github SCM, please provide below variables, refer Github Checks Intergrations page for details
export GITHUB_CONFIG_REPO_URL=<github_url>
export GITHUB_CONFIG_INSTALLATION_ID=<installation_id>

# If you are using Gitlab SCM, please provide below variables, refer Gitlab Integration page for details
# export GITLAB_CONFIG_BASE_URL=<Base domain or https://gitlab.com> ## This is just the base domain and not complete URL of yuour gitlab repo
# export GITLAB_CONFIG_PROJECT_ID=<Gitlab_project_id> ## Project ID of your gitlab Project
# export GITLAB_CONFIG_ACCESS_TOKEN=<User_Access_token> ## Gitlab Access Token of User which has access to Repo - Scope -api

# If you are using BitBucket SCM, please provide below variables, refer BitBucket Intergrations page for details
# export BITBUCKET_CONFIG_WORKSPACE=<bitbucket-workspace>
# export BITBUCKET_CONFIG_REPO_SLUG=<bitbucket repo name>
# export BITBUCKET_CONFIG_USER_NAME=<bitbucket user-name>
# export BITBUCKET_CONFIG_APP_PASSWORD=<bitbucket app-password>

Hypertest CLI is ready to be used for automatically starting tests.

Below are some of the commands that you can use to verify settings, start tests, see the progress of tests, etc

3. Check if HyperTest can run a test (optional)

Check if a new test can be started on hypertest. Verify if all parameters are set and you can start running the test on Hypertest

HT_BASE_URL=http://1.2.3.4:5678 ./hypertest can-run-test

./hypertest can-run-test --help

./hypertest can-run-test --help
Usage: hypertest can-run-test [options]

checks if a new test can be started on hypertest. Using --poll will check until a new test can be started

Options:
  --poll                                       polls hypertest until a new test can be started (default: false)
  --interval <interval>                        poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>         max duration to wait for execution (default 3600) (default: 3600)
  --git-dir <path/conatinting/.git/directory>  useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
  -h, --help                                   display help for command

4. Start a new test

Start a new test on HyperTest

HT_BASE_URL=http://1.2.3.4:5678 ./hypertest start-new-test

./hypertest start-new-test --help

./hypertest start-new-test --help
Usage: hypertest start-new-test [options]

starts a new test on hypertest

Options:
  -d, --delay <delay-time-in-seconds>          delay in seconds to wait before starting a new test (default: 0)
  -p, --primary <primary-base-url>             base url for stable(primary) version
  -c, --candidate <candidate-base-url>         base url for test(candidate) version
  -i, --initial-timestamp <initial-timestamp>  traffic would be considered from here. (calculated from test_window by default) (Should be parsable by new Date(val) in javascript)
  -f, --final-timestamp <final-timestamp>      traffic would be considered until here. (current time by default) (Should be parsable by new Date(val) in javascript) (default: "2022-09-13T12:20:29.979Z")
  -t, --testcase-ids <testcase-ids>            run test for selected testcases. (comma separated testcase ids)
  --no-stop-running-tests                      dont start a suite if one is already running. (default: terminates old and starts new)
  --stop-on-first-failure                      stops test execution on first failure (default: complete entire test execution) (default: false)
  --smoke-only                                 only run test cases saved as smoke tests (default: false)
  --saved-only                                 only run saved test cases (default: false)
  --play-duplicate-requests                    run duplicate requests within a session (default: false)
  --play-duplicate-sessions                    run duplicate sessions (default: false)
  --description <description>                  test case description
  --interval <interval>                        poll interval in seconds (default 60) (default: 60)
  --skip-wait-for-old-tests                    does not wait for old running tests to complete. Running tests terminated by default or program fails if --no-stop-running-tests is passed (default: false)
  --wait-for-result                            wait for test to complete (just starts the test by default) (default: false)
  --ht-env <ht-env>                            test ht env
  --minimum-severity <minimum-severity>        minimum severity for differences
  --signoff-required                           Is accepting/rejecting regressions required after test results are generated. (Useful in CI integration) (default: true)
  --git-dir <path/conatinting/.git/directory>  useful to obtain branch name. (Default = current working directory) (default: "/opt/hypertest_neha")
  --time-limit <time-limit-in-seconds>         max duration to wait for execution (default 3600) (default: 3600)
  -h, --help                                   display help for command

5. Wait for results (waits for test to be over)

Find the test progress of a given testId

# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-test-progress -wait-for-result

./hypertest get-test-progress --help

./hypertest get-test-progress --help
Usage: hypertest get-test-progress [options]

find test progress of given testId

Options:
  -t --test-id <test-id>                test id to get test result
  --interval <interval>                 poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>  max duration to wait for execution (default 3600) (default: 3600)
  --wait-for-result                     wait for test to complete (just display current progress by default) (default: false)
  -h, --help                            display help for command

6. Wait for qa signoff (waits until results are accepted/rejected)

Find signoff state of last test

# last test id picked up by default
HT_BASE_URL=http://1.2.3.4:5678 ./hypertest get-signoff-status --wait-for-signoff

./hypertest get-signoff-status --help

./hypertest get-signoff-status --help
Usage: hypertest get-signoff-status [options]

find signoff state of last test

Options:
  -t --test-id <test-id>                test id to get signoff state
  --interval <interval>                 poll interval in seconds (default 60) (default: 60)
  --time-limit <time-limit-in-seconds>  max duration to wait for execution (default 604800) (default: 604800)
  --wait-for-signoff                    wait for test to signoff (just starts the test by default) (default: false)
  -h, --help                            display help for command

Other CLI options

Access the help options for all commands by using -h or --help options

./hypertest --help

./hypertest --help
Usage: hypertest [options] [command]

Options:
  -v, --version                           output the current version
  -h, --help                              display help for command

Commands:
  start [options]                         start new hypertest instance
  stop [options]                          stop hypertest
  update [target-version]                 update hypertest
  update-cli [target-version]             update hypertest
  restart                                 restarts hypertest
  bg-start                                forcefully starts a new background service by killing previous one if running
  bg-test                                 test if background service is running
  bg-kill                                 forcefully kills background service if running
  can-run-test [options]                  checks if a new test can be started on hypertest. Using --poll will check until a new test can be started
  start-new-test [options]                starts a new test on hypertest
  get-test-progress [options]             find test progress of given testId
  get-signoff-status [options]            find signoff state of last test
  get-signoff-status-by-commit [options]  find signoff state in last test of given commit id
  tunnel [options]                        establish a tunnel for your service without exposing it on the public internet
  help [command]                          display help for command

PreviousAutomate Starting a Test NextGitHub Checks Integration

Last updated 1 year ago