Test Center concepts

To use Test Center, you need to understand these concepts that the API makes available in its URL resources and data.

Functional testing

Use this type of testing to check if your configuration changes work as planned. These concepts all apply to functional tests:

  • Requirements. Requirements describe business goals you want to achieve with particular configuration changes. By associating test suites with them, you're saying that the functional test cases included in the test suites can be used as a unit test to confirm the correct implementation of the business requirement.

  • Config versions. Config versions refer to Property Manager property versions. All operations performed on config versions are saved only in Test Center. They don't affect properties in Property Manager. By adding a config version to Test Center and then associating a test suite with it, you're saying that the functional test cases in the test suite are regression test cases for the property version. To use a property in Test Center, you need to add it first to Property Manager. To learn how to do it, check the Property Manager API documentation: Create a new property version and Create a new activation or deactivation.

  • Test suites. Test suites combine several functional test cases and configure their settings.

  • Stateful test suites. In stateful test suites, a test run executes included test cases in the set order. Cookies and session information are retained for subsequent test cases.

  • Locked test suites. Locked test suites can be modified only by their editors and owners. Test Center users who create locked test suites automatically become their owners, but they can also designate other owners. Editors can modify locked test suites (except for the Locked status), add new functional test cases to them, and remove those already included. If the test suites are also stateful, editors can reorder included test cases. Owners can additionally manage the locked test suites' edit access and remove them. Every Test Center user can send a request for edit access to the test suite's owners. Owners are notified about the request with an email. Once they approve or reject the request, the user also gets a notification email.

  • Functional test cases. Functional test cases are the smallest units of testing. Each test case includes one test request, one condition, and one client profile. Test cases must be unique at the level of a test suite. It means that a test case can be included in only one test suite.

  • Test requests. Test requests combine fully qualified URLs and an optional set of custom request headers. By default, the test request URL is requested with whatever default headers are normally sent by the browser used by the client profile.

  • Client profiles. Client profiles combine a browser type, geographic region, and IP version that characterize the client used to make a test request. There are two client profiles provided that you can use. Both of them use Chrome as the browser type and the United States as the geographic region. They differ by IP versions — one is for IPv4 and the other one is for IPv6.

  • Conditions. Conditions are the criteria you want to evaluate on the HTTP response. They correspond to the test request or a specific config setting applied to the test request. Conditions have a sentence-like structure. For example: Response header 'foo' has a value that equals 'bar' or Caching option is no-store. Each element of such sentence is a value of a node. You can find all nodes and their descriptions including available values in the Test catalog template.

  • Test catalog template. You use the test catalog template to construct conditions. It lists all available nodes with their values and information about which nodes they trigger.

Comparative testing

Use this type of testing to find the side effects of configuration changes on the delivery of a hostname. Comparative testing compares the delivery of a hostname with the current configuration and the new one and lists all found differences (diffs).

It also enables you to compare how your website behaves on two production environments against each other. It's useful if you want to compare the configuration changes behavior on a test hostname which doesn't carry live traffic with the hostname that does carry the live traffic. These concepts all apply to comparative tests:

  • Test definitions. Test definitions are units of comparative testing. They associate hostnames impacted by config changes with IP versions for which you want to run a test.

  • Comparative test cases. Comparative test cases combine the hostname on which you want to run a test and, optionally, request headers.

  • Diffs. Diffs indicate a difference found between the hostnames' behaviors. A diff might either show expected behavior, behavior that you wanted to achieve with the configuration changes, or unexpected. You need to decide which one it is. You need to accept the expected diffs that are not likely to cause problems once you activate the changes. By not accepting the diff you indicate that the detected behavior is unexpected and needs to be further investigated.

Shared concepts

These concepts all apply to both, comparative and functional tests:

  • Test run. A test run executes one or more objects against a selected environment. Each test run can either execute the test for only one type of object (a requirement, config versions, test suites, functional test cases, or test definitions) or you can combine a requirement with config versions, config versions with test definitions, a requirement with test definitions, or a requirement with config versions and test definitions.

  • Execution. An execution is an instance of a resource used in a test run.

  • Test results. Test results for comparative testing list diffs found in how a hostname serves out to end users. For functional testing the results show you the expected and actual values.

  • Activity. An activity is any operation performed in Test Center.