How mPulse XMLHttpRequest (XHR), Fetch API and Single Page Application (SPA) Monitoring works

mPulse Real User Monitoring (commonly called RUM) uses the Boomerang JavaScript library running in browsers to collect and beacon back performance metrics.

When XHR instrumentation is enabled, Boomerang will monitor several events:

• XMLHttpRequest requests.

• Fetch API requests. Fetch API instrumentation can be enabled with Config Overrides for Boomerang version 1.609.0 and above.

• Mouse clicks for non SPA sites.

• History API calls and events for SPA sites. See Single page apps setup documentation.

When any of these events occur, Boomerang will start monitoring the page for DOM mutations and other networking activity.

As long as the event isn't determined to be background activity (i.e an XHR that wasn't followed by additional network requests), the event will be measured until all networking activity has completed.

This means that if an XHR is followed by a request to fetch more HTML that in turn added images to the page, the entire event will be measured from the XHR request start to the last image response completing.

By default Boomerang also groups concurrent XHR activity together. If you have one XHR that immediately triggers a second XHR, you will get a single XHR beacon. The u beacon parameter will be of the first XHR.

Compatibility and Browser Support

In general, Boomerang supports all browsers that support MutationObserver and XMLHttpRequest. If XHR is supported but MutationObserver is not then Boomerang will not be able to detect network activity from DOM mutations and there will be a much greater chance of the XHR being flagged as background activity and not be reported.

Supported Browsers and platforms:

• IE 9+ (not in quirks mode). XHR only.

• Edge

• Chrome 38+

• Firefox 25+

Note: Boomerang will not use MutationObserver in IE 11 due to several browser bugs.

Interesting Nodes Triggered by DOM Mutations

MutationObserver is used to detect "interesting" nodes. Interesting nodes are new IMG/IMAGE/IFRAME/LINK (rel=stylesheet) nodes or existing nodes that are changing their source URL.

Boomerang considers the following "uninteresting" nodes:

• Nodes that have either a width or height <= 1px.

• Nodes that have display:none.

• Nodes that have visibility:hidden.

• Nodes that update their source URL to the same value.

• Nodes that have a blank source URL.

• Nodes that have a source URL starting with about:, javascript: or data:.

• SCRIPT nodes because there is no consistent way to detect when they have loaded.

• Existing IFRAME nodes that are changing their source URL because is no consistent way to detect when they have loaded.

• Nodes that have a source URL that matches a XHR exclude filter rule.

Note: Resources requested from CSS stylesheets cannot be detected using MutationObserver and are not tracked. Eg. Resources initiated from @font-face section or background-image directive and cannot be detected.

XMLHttpRequest (XHR) and Fetch API Monitoring

• Back-End time for XHR requests is the time of the first XHR request (startTime to responseEnd).

• Total time starts at the first XHR request startTime and includes all interesting nodes and XHR/Fetch requests that follow.

• Front-End time is the Total time minus the Back-End time.

By default, XHR events will wait up to 50ms after loading for interesting requests to start.

Examples

Example 1 - XHR not followed by an interesting mutation or XHR/Fetch (no beacon)

In this example, the XHR is not followed by an interesting resource or another XHR/Fetch request within 50ms of the XHR onload event or during XHR onload event handlers. Boomerang determines that this is a background XHR and no beacon is sent.

Example 2 - XHR followed by an interesting mutation within 50ms

In this example, the XHR is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 3 - XHR with an interesting mutation within 50ms with slow XHR onload handlers

In this example, the XHR is followed by an interesting resource that initiated during slow XHR onload event handlers (JavaScript is executing for more than 50 ms). Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 4 - XHR followed by an interesting mutation after 50ms (no beacon)

In this example, the XHR is not followed by an interesting resource within 50ms of the XHR onload event or during XHR onload event handlers. Boomerang determines that this is a background XHR and no beacon is sent.

Example 5 - XHR with an interesting mutation within 50ms followed by another interesting mutation within 50ms of that finishing

In this example, the XHR is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. Boomerang detects image2.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 6 - XHR with an interesting mutation within 50ms followed by another interesting mutation after 50ms of that finishing

In this example, the XHR is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 7 - Concurrent XHRs with an interesting mutation within 50ms of one of XHRs loading

In this example, Boomerang detects the api-example2 that is being fetched concurrently to the initially tracked api-example1 XHR. When Boomerang tracks concurrent XHRs then mutation monitoring starts after the any of the tracked XHRs loads. The Back-End time is always the duration of the initial XHR. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 8 - XHR with an interesting mutation within 50ms. Several resources are not tracked

In this example, The XHR is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects style.css and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.
Several resources are not tracked in this example:

• random.jpg is not tracked because it initiated before the XHR loads.

• SCRIPT nodes, like example.js, are not tracked because there is no consistent way to detect when they have loaded.

• pixel.jpg is not tracked. Nodes that have either a width or height <= 1px, nodes that have display:none or nodes that have visibility:hidden are not tracked.

• style.css is detected and tracked but resources initiated from stylesheets cannot be detected or tracked. font.woff was initiated from a CSS @font-face section and cannot be detected. background.jpg was initiated from a CSS background-image directive and cannot be detected.

Example 9 - XHR matching an exclude filter

In this example, Boomerang ignores analytics XHR because it matches an XHR exclude filter rule defined by the mPulse app configuration. The api-example1 XHR which does not match an XHR exclude filter rules is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Below are sample mPulse app configurations that could include api-example1 and exclude analytics from XHR monitoring.

Using the Only Matching option of Capture XHRs to allowlist monitoring of api-example and ignore analytics by default:

Using All option of Capture XHRs to blocklist monitoring of analytics and include api-example1 by default:

Example 10 - XHR matching an exclude filter

In this example, Boomerang ignores analytics XHR because it matches an XHR exclude filter rule defined by the mPulse app configuration. The api-example1 XHR which does not match an XHR exclude filter rules is followed by an interesting resource within 50ms of the XHR onload event. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent. See Example 9 above for sample mPulse app configurations.

Example 11 - Concurrent XHRs not followed by an interesting mutation

In these examples, Boomerang detects the api-example2 that is being fetched concurrently to the initially tracked api-example1 XHR. When Boomerang tracks concurrent XHRs then mutation monitoring is enabled after any of the tracked XHRs loads. The Back-End time is always the duration of the initial XHR. Once the all resources are loaded Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 12 - XHR followed by an XHR

Example 13 - Fetch followed by interesting mutations

In this example, a Fetch API request is followed by interesting resources. Boomerang enables mutation detection once the Fetch Promise resolves. Boomerang detects image1.jpg and image2.jpg and waits until the resources are loaded. Once the last resource is loaded Boomerang waits another 50ms for new interesting resources. No new resources are detected and a beacon is sent.

Example 14 - XHR in a browser that does not support the MutationObserver API

In browsers that do not support the MutationObserver API (eg. all versions of IE), Boomerang will only include XHR and Fetch requests in XHR monitoring. Although IE 11 supports MutationObserver, it is not used by Boomerang due to browser bugs making it unreliable. In this example, the XHR is not followed by another XHR/Fetch request within 50ms of the XHR onload event or during XHR onload event handlers. No beacon is sent because no resources were detected.

SPA Hard Navigations

A SPA hard navigation is the first navigation to the site, plus any of the work required to build the initial view. The SPA hard navigation will track at least the length of onload​, but may also include the additional time required to load the framework (for example, Angular) and the first view.

• All static resources loaded by a page extend the page’s onload event.

• Resources added to a page dynamically before the onload event also extend the page’s onload event.

• XHR/Fetch requests do not extend onload.

• SPA apps will typically load many resources after the page’s onload event has fired.

The goal of measuring SPA hard navigations is to capture the time of resources added dynamically after the onload event and XHR/Fetch resources that weren’t included in the page onload event time. In order for Boomerang to achieve this goal it needs to be loaded and configured before those said resources are requested. If Boomerang is loaded or configured too late then resources may not be tracked. If Boomerang does not detect any resources after onload then it falls back to reporting onload time like a non-SPA navigation.

For SPA hard navigations,

• Back-End time is calculated with the same timers as a traditional navigation, where the root HTML's timestamps are used for calculation: NavigationTiming navigationStart until responseStart.

• Front-End time is the Total time minus the Back-End time.

By default, SPA events will wait up to 1s after initiating for interesting requests to start.

Examples

Example 1 - SPA hard navigation with Boomerang and config loaded before browser’s onload. Interesting mutations and XHR/Fetch requests detected.

In this example, Boomerang is loaded and configured before onload and is able to detects interesting resources after onload. Boomerang detects api-example1, template.html, api-example2, image2.jpg and api-example3. Boomerang waits until the resources are loaded. Once the resources are loaded Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.

Example 2 - SPA hard navigation with Boomerang and/or config loaded after browser’s onload. Interesting mutations or XHR/Fetch requests detected.

In this example, Boomerang is loaded and configured after onload and is able to detects some but not all interesting resources after onload. Boomerang does not detect api-example1, template.html, api-example2 because they start before the config response. Boomerang detects image2.jpg and api-example3. Boomerang waits until the resources are loaded. Once the resources are loaded Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.

Example 3 - SPA hard navigation with Boomerang and/or config loaded after browser’s onload. No interesting mutations or XHR/Fetch requests detected.

SPA monitoring starts once config response is applied to Boomerang.
In this example, Boomerang is loaded and configured after onload and is not able to detects interesting resources after onload. Boomerang does not detect api-example1, template.html, image2.jpg and api-example2 because they start before the config response. Boomerang waits 1s for new interesting resources. No new resources are detected and a beacon is sent. Boomerang does not detect api-example3 because it happened too late. Since no resources are detected by the SPA monitoring, Boomerang falls back to reporting NavigationTiming onload time like a non-SPA navigation.

Example 4 - SPA hard navigation with Boomerang and/or config loaded after browser’s onload. Interesting mutations and XHR/Fetch requests detected. Several resources are not tracked

In this example, Boomerang is loaded and configured after onload and is able to detects some but not all interesting resources after onload. Boomerang does not detect api-example1, template.html, api-example2 because they start before the config response. Boomerang detects image2.jpg, style.css and api-example3. Boomerang waits until the resources are loaded. Once the resources are loaded Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.
Several resources are not tracked in this example:

• SCRIPT nodes, like example.js, are not tracked because there is no consistent way to detect when they have loaded. However, if the ResourceTiming API is available then the request time of the SCRIPT node will be included in the Back-End time since it started and ended within the Total time of the SPA navigation.

• pixel.jpg is not tracked. Nodes that have either a width or height <= 1px, nodes that have display:none or nodes that have visibility:hidden are not tracked.

• style.css is detected and tracked but resources initiated from stylesheets cannot be detected or tracked. font.woff was initiated from a CSS @font-face section and cannot be detected. background.jpg was initiated from a CSS background-image directive and cannot be detected.

Example 5 - SPA hard navigation with XHRs matching an exclude filter

In this example, Boomerang ignores analytics XHR because it matches an XHR exclude filter rule defined by the mPulse app configuration. Boomerang detects api-example1 XHR which does not match an XHR exclude filter and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.

Below are sample mPulse app configurations that could include api-example1 and exclude analytics from XHR monitoring.

Using the Only Matching option of Capture XHRs to allowlist monitoring of api-example and ignore analytics by default:

Using All option of Capture XHRs to blocklist monitoring of analytics and include api-example1 by default:

Example 6 - SPA hard navigation with XHR monitoring not enabled

In this example, XHR resources are not detected due to XHR exclude filters in the mPulse app configuration. Since no resources are detected by the SPA monitoring, Boomerang falls back to reporting NavigationTiming onload time like a non-SPA navigation.

Below are sample mPulse app configurations that could exclude api-example1 and exclude analytics from XHR monitoring.

Using the None option of Capture XHRs to disable XHR monitoring.

Using the Only Matching option of Capture XHRs to ignore api-example1 and analytics by default:

Example 7 - SPA hard navigation in a browser that does not support the MutationObserver API

In browsers that do not support the MutationObserver API (all versions of IE), Boomerang will only include XHR and Fetch requests in SPA navigation time. Although IE 11 supports MutationObserver, it is not used by Boomerang due to browser bugs making it unreliable. Boomerang does not detect image2 since MutationObserver is not supported. Boomerang detects api-example1 XHR and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 1s for new interesting resources. Boomerang does not detect image3, no new resources are detected and a beacon is sent.

SPA Soft Navigations

A SPA soft navigation is any navigation after the first hard navigation. A soft navigation is an “in-page” navigation where the view changes, but the browser does not actually fully navigate.

Once a SPA soft navigation is detected, the Boomerang SPA monitoring starts monitoring the page’s markup (DOM) for changes. If any of these changes trigger a download, such as a XHR, image, CSS, or JavaScript, then the Boomerang SPA plugins monitor those resources as well. Only once all of these new resources have been fetched do the Boomerang SPA plugins consider the SPA navigation complete.

For SPA soft navigations,

• Back-End time is any time with an XHR, fetch or SCRIPT request in progress.

• Front-End time is the Total time minus the Back-End time.

For browsers that do not support the ResourceTiming API, Back-End and Front-End time will not be available.

By default, SPA events will wait up to 1s after initiating for interesting requests to start.

Examples

Example 1 - SPA soft navigation with no URL change not followed by an interesting mutation or XHR/Fetch within 1s (no beacon)

In this example, the SPA soft navigation is not followed by an interesting resource or a XHR/Fetch request within 1s of the route change. No beacon is sent because no resources were detected and the URL did not change.

Example 2 - SPA soft navigation with a URL change not followed by an interesting mutation or XHR/Fetch within 1s (beacon total time ~1ms)

In this example, the SPA soft navigation is not followed by an interesting resource or a XHR/Fetch request within 1s of the route change. A beacon is sent because the URL changed even though no resources were detected. The reported navigation total time will be ~1ms.

Example 3 - SPA soft navigation followed by interesting mutations and XHR/Fetch requests

In this example, the SPA navigation is followed by an interesting resource that initiated with 1s of the route change. Boomerang detects api-example1, api-example2, image1.jpg and api-example3. Boomerang waits until the resources are loaded. Once the resource is loaded, Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent. Back-End is any time with an XHR, fetch or SCRIPT request in progress. SCRIPT nodes, like example.js, are not tracked because there is no consistent way to detect when they have loaded. However, the request time of the SCRIPT node will be included in the Back-End time since it started and ended within the Total time of the SPA navigation. Front-End time is the Total time minus the Back-End time. For browsers that do not support the ResourceTiming API, Back-End and Front-End time will not be available.

Example 4 - SPA soft navigation with an interesting mutation within 1s. Several resources are not tracked

In this example, The SPA soft navigation is followed by an interesting resource within 1s. Boomerang detects api-example, random.jpg and style.css and waits until the resources are loaded. Once the resources are loaded Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.
Several resources are not tracked in this example:

• SCRIPT nodes, like example.js, are not tracked because there is no consistent way to detect when they have loaded. However, if the ResourceTiming API is available then the request time of the SCRIPT node will be included in the Back-End time since it started and ended within the Total time of the SPA navigation.

• pixel.jpg is not tracked. Nodes that have either a width or height <= 1px, nodes that have display:none or nodes that have visibility:hidden are not tracked.

• style.css is detected and tracked but resources initiated from stylesheets cannot be detected or tracked. font.woff was initiated from a CSS @font-face section and cannot be detected. background.jpg was initiated from a CSS background-image directive and cannot be detected.

Example 5 - SPA soft navigation with XHRs matching an exclude filter

In this example, Boomerang ignores analytics XHR because it matches an XHR exclude filter rule defined by the mPulse app configuration. The api-example1 XHR which does not match an XHR exclude filter rule is followed by an interesting resource within 1s of the loading. Boomerang detects image1.jpg and waits until the resource is loaded. Once the resource is loaded, Boomerang waits another 1s for new interesting resources. No new resources are detected and a beacon is sent.

Below are sample mPulse app configurations that could include api-example1 and exclude analytics from XHR monitoring.

Using the Only Matching option of Capture XHRs to allowlist monitoring of api-example and ignore analytics by default:

Using All option of Capture XHRs to blocklist monitoring of analytics and include api-example1 by default:

Example 6 - SPA soft navigation with URL change not followed by an interesting mutation. within 1s. XHR monitoring not enabled. (beacon total time ~0ms)

In this example, the SPA soft navigation is not followed by an interesting resource within 1s of the route change, XHR resources are not detected due to XHR exclude filters in the mPulse app configuration. A beacon is sent because the URL changed even though no resources were detected.

Below are sample mPulse app configurations that could exclude api-example1 and exclude analytics from XHR monitoring.

Using the None option of Capture XHRs to disable XHR monitoring.

Using the Only Matching option of Capture XHRs to ignore api-example1 and analytics by default:

Example 7 - SPA soft navigation in a browser that does not support the MutationObserver API

In browsers that do not support the MutationObserver API (all versions of IE), Boomerang will only include XHR and Fetch requests in SPA navigation time. Although IE 11 supports MutationObserver, it is not used by Boomerang due to browser bugs making it unreliable. Boomerang does not detect image1 since MutationObserver is not supported. No new resources are detected and a beacon is sent.