Once the grid is running, it is best practice to perform a calibration run first with a reduced load. This provides an opportunity to see how the test composition works and to see the response timings to make sure it runs as expected, before adding a more realistic load and introducing complication to the test.

Best Practice 10

Do a calibration run first before full load testing.

The calibration run would show the response timings so that you can calibrate that into the delay.

In the image below we can see that as the virtual user ramped up by about 100, the response times remain constant at about 250 ms. That can be built into the clip and the response time adjusted off of the delay. Previously a delay of one second was set, but with a 250ms response time, the delay has been changed to 750ms.

(Average response time vs. virtual users)

Once that is done proceed with the full load testing.

The images below show some analysis dashboards that come with CloudTest. CloudTest is able to visualize a dashboard in real time, and you can take some actions even when something happens during testing.

(Load analytics dashboard)

The top left graph in the image above shows that as the number of users increased, the response time suddenly jumped to 678ms. Because CloudTest has the capability to control the virtual user count during load testing, the load was brought back down. The maximum load can be determined by a sudden jump in response time, as shown in this example. Here, the maximum load was about 360 transactions per second.

The performance dashboard in the image below shows the transaction time for individual APIs. In the graph on the left, the registration API (blue) is slightly longer than the traditional login API (purple). The graph on the right shows the number of API calls completed per minute.

(Performance dashboard)