Regression Report

FAQs on HyperTest regression reports inside your version management tool when integrated with HyperTest

What is this Report?

HyperTest Reports in GitHub

• This is the git diff for all your APIs

• It contains desired and undesired changes detected due to regression.

• You may confirm these as bugs or desired changes by opening up the regression report in the HyperTest dashboard.

Why see this?

1. This report is the most exhaustive list of changes you are pushing to prod

2. If ignored, you can possibly miss a potential regression that can break your application.

How is it generated?

1. API traffic is recorded from your network and replayed at the same time on (a) Stable Version and (b) Test Version

2. API responses are generated from both versions at the same time, compared to track deviations like response codes, schema or data.

3. We also enable necessary modifications through middlewares while replaying requests, to handle and properly execute write (or non-idempotent) requests

If we mark a regression as not a bug does it show again for the same Merge Request?

1. Once a regression is marked as the desired change; as in when in you click “no” under “Is this a bug”, HyperTest creates a temporary rule which will ignore the same regression for any upcoming tests for a default number of 14 days. So for the current MR, this regression will not be reported.

Are responses from the two versions(primary and candidate) pre-recorded?

No, Only request parameters are stored in Hypertest, from the network traffic. HyperTest doesn't store responses. We will present the response once a test is run.