What is in an mPulse beacon

What's in an mPulse Beacon (Web Apps)

The content of this page applies to mPulse web apps. Documentation for mobile apps is available here.

mPulse is built on top of the boomerang JavaScript library which collects web performance data from a user’s web browser and sends the data back to the mPulse servers via a beacon.

Beacons are invisible network requests that contain performance data and other page-load characteristics. All of this data is included on the beacon via HTTP headers or as part of the query string or form payload.

Within the web performance community, this data is commonly called RUM or Real User Measurement data because it involves measuring the experience of real users.

In this document, we will describe the various parameters collected via the boomerang beacon as well as how they are mapped to other parts of mPulse such as S3, Asgard and DSWB.

As of April 2020, all IP addresses are truncated. IPv4 IPs are truncated to /24 (e.g. 1.2.3.4 to 1.2.3.0) and IPv6 IPs are truncated to /64 (e.g. aaaa:bbbb:cccc:dddd:eeee:ffff:gggg:hhhh to aaaa:bbbb:cccc:dddd::).

Top Level Fields

FieldQuery String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
Log Type--type----beacon batchmPulse log type
Beacon ID--idbeaconId--xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxBeacon GUID
Domainh.ddomain--domainwebsite.comRegistered domain or registered dev domain in mPulse of the page sending the beacon
Domain ID--domainidappid--123Registered domain ID
Timestamp--timestamptimestamptimestamp1512086400000Server timestamp when beacon arrived (Unix epoch)
Page IDpidpage_idpageId--abcd1234Page ID
Beacon Numbernnadditionalfieldsjson--1Beacon number for current page
IP--remote_ipremoteipremoteIP1.2.3.0IP address of the end user (or the proxy they are connected through) (truncated)
API Keyh.keykey--keyXXXXX-XXXXX-XXXXX-XXXXX-XXXXXmPulse API Key assigned to this domain
Beacon HTTP Method--http_method--httpMethodGET POSTHTTP Request method used to send the beacon (for internal use only)
Beacon HTTP Version--http_version--httpVersionHTTP/1.1HTTP Version used to send the beacon (for internal use only)
Beacon Referrer--http_referrer--beaconReferrerhttp://referrer.comURL of the page that sent the beacon. This may be blank depending on browser privacy settings.
Beacon IPv6--ipv6ipv6ipv6trueSet to true if the beacon was sent using IPv6 (for internal use only)
Proxy Address--proxy_address--proxyAddress1.2.3.4IP address of the CDN provider that handled the edge connection for this beacon (for internal use only) (not truncated)
Beacon Typehttp.initiatorbeacon_typebeacontypenamebeaconTypepage view xhr unload spa spa_hard interaction
client_batch api_custom_metric api_custom_timer api_network_request
Type of beacon: page view, XHR, SPA hard or soft navigation, etc
Page Grouph.pgpage_grouppagegroupnamepageGroupHome Orders PDP (No Page Group)Page Group for the page (Page Groups are defined in the mPulse App Config dialog)
A/B Test nameh.abab_testabtestnameabTestNameA BIf the site uses A/B Testing, this variable specifies the test bucket the user was in when they sent the beacon
Site/App Versionh.vsite_versionsiteversionnamesiteVersion1.0 2.5bSite owners may use this to specify a release number for their site.
Filtered URL--urlurlurlhttp://website.com/URL of page according to any URL patterns defined within the app.
If no URL patterns are defined, this is the top level URL of the site.
Warnings--warningswarningswarnings["clock:askew"]Array of warnings added by the collector when parsing this beacon
SPDYnt_spdyspdyspdyspdytrueSet to true if the page was served using SPDY
SSL--sslsslssltrueSet to true if the page was served using SSL
API Statusapiapi----trueSet to true if this beacon was sent from an API call
Mobile Connection Typemob.ctmobile_connection_typemobileConnectionType--Ethernet WiFi 2G 3G 4G LTEMobile connection type
Navigation Typent_nav_typenavigation_typenavigationTypeName--navigate reload back_forwardNavigation Type

Session

The following fields live under the session: {} object in S3.

SessionQuery String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
Session IDrt.siIDsessionidsessionIdxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-nnnnnnSession ID generated by mPulse and stored in a cookie by Boomerang.

The first five chunks are a UUID, while the last chunk is the base36 representation of the session start time based on the user's clock.
Session Start Timert.ssstartsessionstartsessionStart1512086400000Session start time (Unix epoch) based on the user's browser's clock.

This clock may not be in sync with the server clock, so only compare it with other browser timestamps.
Session Latest Timert.endlatestsessionlatestsessionLatest1512086400000The timestamp (Unix epoch) of the current beacon based on the user's browser's clock.

It is not the timestamp of the latest beacon in the session.

To find out actual session duration, look at MAX(latest) - MIN(start) across all beacons grouped by Session ID.
Session Length
(in Pages)
rt.slpagessessionpagessessionLength10The number of pages the user viewed in this session up to and including the current page, but not including any pages viewed after this page in the same session.

To find out actual Session Length in pages, look at MAX(pages) across all beacons grouped by Session ID.

If MAX(pages) == 1, then the session is considered a bounce.
Uncounted pagesrt.obooboPagessessionobopagessessionUncountedPages0The number of pages in the session for which we were unable to calculate a load time.

We need this to get a closer estimate of the actual average load time of all pages in the session.
Total Load Timert.tttotalLoadTimesessiontotalloadtimesessionTotalLoadTime2000The sum of load times (in milliseconds) of all pages within the session where we calculated load time.

The average load time of a session is MAX(totalLoadTime) / (MAX(pages) - MAX(oboPages)).
Unload flagrt.quitisUnloadsessionisunloadsessionIsUnloadtrue or missingThis field specifies if the beacon was part of the unload event of the page or not.

Unload beacons give us a better idea of how long a session actually lasted.

User Agent Fields

The following fields come from the User-Agent HTTP Request Header. There are no query string parameters.

The following fields live under the user_agent: {} object in S3.

User Agent FieldsS3AsgardDSWB BEACON_TABLE varExamplesDescription
Browser FamilyfamilyuseragentnameuserAgentFamilyChrome Firefox IE Safari Mobile SafariThe browser family extracted from the user agent header
Major versionmajoruseragentversionuserAgentMajor42 13 11The browser major version
Minor versionminoruseragentminoruserAgentMinor0 5The browser minor version
OS NameosoperatingsystemnameuserAgentOsWindows Android OS Mac OS X iOSThe Operating System name for the user
OS VersionosversionoperatingsystemversionuserAgentOsVersion8.1The Operating System version for the user
ModelmodeldevicenameuserAgentModeliPhone iPadDevice model for mobile devices
ManufacturermanufacturerdevicemanufacturernameuserAgentManufacturerApple Samsung LGDevice manufacturer for mobile devices
Device TypetypedevicetypenameuserAgentDeviceTypeDesktop Mobile TabletDevice type
Syntheticsynthetic----WebPagetestDetected synthetic user agent name
Botbot----cURL GooglebotDetected bot name
Raw User Agent StringrawuseragentrawuserAgentRawMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Safari/537.36The raw User-Agent HTTP header

Geo Information

The following fields come from doing a MaxMind GeoIP lookup on the source IP address. There are no query string parameters.

The following fields live under the geo: {} object in S3.

Geo InformationS3AsgardDSWB BEACON_TABLE varExamplesDescription
CountrycccountrycodecountryCodeUSThe user's country (L1 geo info)
Region/State/TerritoryrgregioncoderegionCodeDCThe user's region, state, province, or territory (L2 geo info)
CitycitygeocitycityNameWashingtonThe user's city (L4 geo info)
Postal CodepostalcodegeopostalcodepostalCode20009The user's postal/zip code
Latitudelatgeolatlatitude10.0The user's approximate latitude (as granular as the zip code of the ISP the user is connecting from)
Longitudelongeolonlongitude10.0The user's approximate longitude (as granular as the zip code of the ISP the user is connecting from)
OrganizationorggeoorgorganizationComcast Cable Verizon WirelessThe user's organization, if connected over a corporate/university network
ISPispgeoispispComcast Cable Virgin MediaThe user's ISP
Network speednetspeedconntypenamenetworkSpeedCellular Ethernet Cable/DSLThe advertised network grade of the user's internet connection
Overwritten Locationovr----true falseWhether the mobile device's location API was used to determine location (rather than the source IP address)

Bandwidth

The following fields live under the bandwidth: {} object in S3.

Also see Bandwidth & Latency details in the Other Parameters section below.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
bwkbpsbandwidthkbpskbps1000Bandwidth (in kbps) as measured by the bandwidth test
bw_errerror_pcbandwidtherrorpckpbsError10Error in measuring the bandwidth based on standard deviation
blockbandwidthblockcodebandwidthBlock1 5Bandwidth block (calculated by the server)
bw_debugbw_debug----5_20Bandwidth debugging info

Timers

All timers are in milliseconds unless otherwise specified.

The following fields live under the timers: {} object in S3 unless otherwise specified.

Standard Timers

TimersQuery String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
Base TimersThe following fields are sent on the beacon as-is
Unload Time--unload_timeunloadTime--100Time unloading the previous page
Redirect Time--redirect_timeredirectTime--100Time from redirect start to redirect end
Waiting Time--waiting_timewaitingTime--100Time from request start to response start
Download Time--download_timedownloadTime--100Time from response start to response end
Back-End Time (TTFB)t_respt_respfirstbytetimertimersResponse100Time from navigation start to first byte
Front-End Timet_paget_pagefirstlastbytetimertimersPage100Time from first byte to onload (or whenever the page is considered ready)
Page Load timet_donet_donepageloadtimetimersDone100Time from navigation start to onload (or whenever the page is considered ready)
DOM Loadedt_domloadedt_domloadedtimerstdomloadedtimersDomLoaded100Time from navigation start to when the DOMContentLoaded event fired
config.json first bytet_configfbt_configfbtimerstconfigfbtimersConfigFirstByte100Time from config.json request to config.json first byte
config.json load timet_configjst_configjstimerstconfigjstimersConfigJs100Time from config.json request to config.json load complete
Worker Start--worker_startworkerStart--100Time for Service Worker start
Other TimersThe following are sent on the beacon as part of t_other
Boomerang loaderboomr_ldboomr_ldtimersboomrldtimersBoomerangLoader100Time from navigation start to when the Boomerang loader initiates Boomerang download
Boomerang first byteboomr_fbboomr_fbtimersboomrfbtimersBoomerangFirstByte100Time from navigation start to first byte of Boomerang
Boomerang latencyboomr_latboomr_lattimersboomrlattimersBoomerangLatency100Time from Boomerang request to Boomerang first byte
Boomerang download timeboomerangboomerangtimersboomerangtimersBoomerang100Time from first byte of Boomerang until Boomerang completes initialization
Detailed Boomerang TimesThe following are sent on the beacon compressed using Resource Timing compression
Detailed Boomerang timesrt.bmrparams["rt.bmr"]paramsRtBmr*--145,124,2Resource Timing values for Boomerang, compressed using the Resource Timing compression algorithm
Detailed config.json timesrt.cnfparams["rt.cnf"]paramsRtCnf*--301,100,3Resource Timing values for config.json, compressed using the Resource Timing compression algorithm

Prerender States

These are only supported on Chrome at this time.

See Chromium Prerender docs & Prerendering in Chrome

Prerender statesQuery String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
Prerender completet_loadt_loadtimerstloadtimersLoad100Time from navigation start to onload, if the page was in a pre-rendered state when onload fired
Total prerendert_prerendert_prerendertimerstprerendertimersPreRender100Time from navigation start to page becoming visible, if it went through a pre-rendered state first
Post rendert_postrendert_postrendertimerstpostrendertimersPostRender100Time from onload in a pre-rendered state to page becoming visible

Added by Server

The following are calculated on the server. There are no query string parameters.

S3AsgardDSWB BEACON_TABLE varExamplesDescription
navSt_to_boomrtimersnavsttoboomrtimersNavigationStartToBoomerang100Time from navigation start to when the Boomerang loader initiates Boomerang download
fb_to_boomrtimersfbtoboomrtimersFirstByteToBoomerang100Time from the first byte of the page to Boomerang's request initiation.
This is used by FirstByteEstimator for beacons without a valid load time.
boomr_to_endtimersboomrtoendtimersBoomerangToEnd100Time from Boomerang's request initiation to page load.
This is used by FirstByteEstimator for beacons without a valid load time.
before_dnstimersbeforednstimersBeforeDns100domainLookupStart - navigationStart from NavigationTiming
dnsdnstimertimersDns100domainLookupEnd - domainLookupStart from NavigationTiming
tcptcptimertimersTcp100connectEnd - connectStart from NavigationTiming
sslssltimertimersSsl100connectEnd - secureConnectionStart from NavigationTiming
domLoaddomloadtimertimersDomLoad100domLoading - navigationStart from NavigationTiming
domReadydomreadytimertimersDomReady100domComplete - navigationStart from NavigationTiming
renderStarttimersrenderstarttimersRenderStart100firstPaint - navigationStart from NavigationTiming
loadedtimersloadedtimersLoaded100loadEventEnd - navigationStart from NavigationTiming

Custom Timers

Custom Timers are configured via the mPulse App config dialog and may be based on a JavaScript variable, NavigationTiming, Resource Timing or User Timing.

On the beacon, these are part of t_other.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
custom[0-9]custom[0-9]customtimer[0-9]timersCustom[0-9]100Custom Timer #[n]

User Defined Timers

You may also add your own timers by calling BOOMR.plugins.RT.setTimer() from JavaScript.

Every custom timer also contains a corresponding *_st (eg: custom0_st) that specifies the timestamp (Unix epoch) when the timer started.

Custom Metrics

Custom metrics are defined in the mPulse App config dialog.

On the beacon, these metrics have the prefix cmet., followed by a sanitized version of the metric name.

In the S3 logs, these metrics are all numerically indexed. Example:

...
custom_metrics: {
    "0": 10,
    "5": 1
},
...

Metric values are always numeric.

Currency values are always in the lowest unit (eg, ¢ rather than $). If a decimal place is included, then the value is multiplied by 100 to get the actual currency value.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
cmet.[Metric0-9_Name]0custommetric[0-9]customMetric[0-9]1Custom Metric #[n]

Custom Dimensions

Custom dimensions are defined in the mPulse App config dialog.

On the beacon, these dimensions have the prefix cdim., followed by a sanitized version of the dimension name.

In the S3 logs, these dimensions are in the custom_dimensions array indexed by their dimension number. Dimensions that aren't set may be null or missing (if no dimensions follow). Example:

...
custom_dimensions: [
    {
        "Region": "us",
        "alias": "United States"
    },
    null,
    {
        "Logged In": false,
        "alias": "False"
    }
],
...

Dimension values may be numeric, string or boolean.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
cdim.[Dimension0-9_Name](index 0)customdimension[0-9]value--{ "Region": "us", "alias": "United States" }Custom Dimension #[n]

HTTP Headers

The following fields live under the headers: {} object in S3.

All headers are lower-cased.

This is not an exhaustive list of all possible HTTP headers.

SourceS3AsgardDSWB BEACON_TABLE varExamplesDescription
CDNsx-forwarded-for----1.2.3.0 1.2.3.0,2.2.2.0List of IP addresses specifying user and all proxies between CDN and the user (truncated)
CDNsx-forwarded-proto----https http spdyProtocol used by end user's browser to connect to CDN endpoint
Akamaiaka-true-ip----1.2.3.0Client IP (truncated)
Browserconnection----close keep-aliveThe HTTP Connection header specifies if the browser is willing to handle persistent TCP conections or not
BrowserhostheadershostheadersHostabc123.akstat.ioThe beacon server where the beacon was sent. This is required by the HTTP specification.
Browseraccept-encodingheadersacceptencodingheadersAcceptEncodinggzip deflate sdch brList of encoding (compression) types supported by the user's browser
Browseraccept-language----en_US fr_CHLanguage encodings supported by the user's browser. This could suggest the user's locale.
BrowseracceptheadersacceptheadersAccept*/*Content types supported by the browser, along with their priorities
Browsercontent-lengthheaderscontentlengthheadersContentLength1234This is only used for POST beacons and is the number of bytes in the POSTed beacon

Other Parameters

Navigation Timing Data

See the NavigationTiming spec for more details, and browser support can be reviewed at caniuse.com.

All timestamps are in (Unix epoch) unless otherwise specified.

The following fields live under the params: {} object in S3.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
nt_nav_typent_nav_typeparamsntnavtypentNavigationType0 1 2Navigation Type: performance.navigation.type
nt_nav_stnt_nav_stparamsntnavstntNavigationStart1512086400000Navigation Start: performance.timing.navigationStart
nt_red_stnt_red_stparamsntredstntRedirectionStart1512086400000Redirect Start: performance.timing.redirectStart
nt_red_endnt_red_endparamsntredendntRedirectionEnd1512086400000Redirect End: performance.timing.redirectEnd
nt_red_cntnt_red_cntparamsntredcntntRedirectionCnt1512086400000Redirect Count: performance.navigation.redirectCount
nt_fet_stnt_fet_stparamsntfetstntFetchStart1512086400000Fetch Start: performance.timing.fetchStart
nt_dns_stnt_dns_stparamsntdnsstntDnsStart1512086400000DNS Start: performance.timing.domainLookupStart
nt_dns_endnt_dns_endparamsntdnsendntDnsEnd1512086400000DNS End: performance.timing.domainLookupEnd
nt_con_stnt_con_stparamsntconstntTcpStart1512086400000TCP Start: performance.timing.connectStart
nt_ssl_stnt_ssl_stparamsntsslstntSslStart1512086400000SSL Start: performance.timing.secureConnectionStart
nt_con_endnt_con_endparamsntconendntTcpEnd1512086400000TCP End: performance.timing.connectEnd
nt_req_stnt_req_stparamsntreqstntRequestStart1512086400000Request Start: performance.timing.requestStart
nt_res_stnt_res_stparamsntresstntResponseStart1512086400000Response Start: performance.timing.responseStart
nt_unload_stnt_unload_stparamsntunloadstntUnloadStart1512086400000Unload Event Start: performance.timing.unloadEventStart
nt_unload_endnt_unload_endparamsntunloadendntUnloadEnd1512086400000Unload Event End: performance.timing.unloadEventEnd
nt_domloadingnt_domloadingparamsntdomloadingntDomLoading1512086400000DOM Loading: performance.timing.domLoading
nt_res_endnt_res_endparamsntresendntResponseEnd1512086400000Response End: performance.timing.responseEnd
nt_domintnt_domintparamsntdomintntDomInt1512086400000DOM Interactive: performance.timing.domInteractive
nt_domcontloaded_stnt_domcontloaded_stparamsntdomcontloadedstntDomContLoadedStart1512086400000DOMContentLoaded Start: performance.timing.domContentLoadedEventStart
nt_domcontloaded_endnt_domcontloaded_endparamsntdomcontloadedentDomContLoadedEnd1512086400000DOMContentLoaded End: performance.timing.domContentLoadedEventEnd
nt_domcompnt_domcompparamsntdomcompntDomComp1512086400000DOM Complete: performance.timing.domComplete
nt_load_stnt_load_stparamsntloadstntLoadStart1512086400000Load Event Start: performance.timing.loadEventStart
nt_load_endnt_load_endparamsntloadendntLoadEnd1512086400000Load Event End: performance.timing.loadEventEnd
nt_worker_startnt_worker_start----1512086400000Worker Start: performance.timing.workerStart
nt_firstpaintnt_firstpaintparamsntfirstpaintntFirstPaint1512086400000note that this is milliseconds on IE, but seconds.microseconds on Chrome. Not supported on other browsers.
nt_spdynt_spdyparamsntspdyntSpdy1 or 0Whether the page was served using SPDY
nt_cinfnt_cinf----http/1.1 http/2+quic/37Connection info
nt_protocolnt_protocolnextHopProtocol--http/1.1 http/2+quic/37Next Hop Protocol information (NavigationTiming2)
nt_enc_sizent_enc_size----100000Compressed size in bytes of the page (NavigationTiming2)
nt_dec_sizent_dec_size----100000Uncompressed size in bytes of the page (NavigationTiming2)
nt_trn_sizent_trn_size----100000Transfer size in bytes of the page (NavigationTiming2)
nt_badnt_bad----1If set, then the NavigationTiming data cannot be trusted due to a browser bug

Paint Timing Data

Data from the Boomerang PaintTiming plugin

See the PaintTiming spec for more details, and browser support can be reviewed at caniuse.com.

All timestamps are in DOMHighResTimeStamp rounded to the nearest millisecond.

The following fields live under the timers: {} object in S3:

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
pt.fpfpfirstPaint--100First Paint
pt.fcpfcpfirstContentfulPaint--300First Contentful Paint
pt.lcplcplargestContentfulPaint--500Largest Contentful Paint

The following fields live under the params: {} object in S3:

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
pt.hidpt.hid----1The document was loaded hidden, so paint metrics won't be added to the beacon
pt.lcp.srcpt.lcp.src----http://example.com/img.jpgSource URL of the Largest Contentful Paint element
pt.lcp.elpt.lcp.el----IMGElement tag name of the Largest Contentful Paint
pt.lcp.ept.lcp.e----body#content .hero-imageElement Pseudo-CSS selector for the Largest Contentful Paint
pt.lcp.spt.lcp.s----50000Size of the Largest Contentful Paint in device-independent pixels squared
pt.lcp.idpt.lcp.id----hero-imageElement ID of the Largest Contentful Paint
pt.lcp.srcsetpt.lcp.srcset----img.jpg 480w, img-large.jpg 800wElement srcset property of the Largest Contentful Paint
pt.lcp.sizespt.lcp.sizes----(max-width: 600px) 480px, 800pxElement sizes property of the Largest Contentful Paint

Single Page App

Information about Single Page App (SPA) navigations.

The following fields live under the params: {} object in S3.

Query String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
spa.missedspa.missed----1 or missingThe SPA hard navigation missed the SPA's route change event
spa.forcedspa.forced----1 or missingThe SPA hard navigation was forced to completion by BOOMR.plugins.spa.markNavigationComplete()
spa.waitingspa.waiting----10The number of resources that were outstanding when the SPA hard navigation was forced
spa.t_clickspa.t_click----10The delta between the last click event and the start of the route change (ms)

Debugging Info and Timestamps

Other browser timestamps and related information taken in Boomerang, mainly used for debugging.

All timestamps are in (Unix epoch) unless otherwise specified.

The following fields live under the params: {} object in S3.

Debugging Info and TimestampsQuery String ParamS3AsgardDSWB BEACON_TABLE varExamplesDescription
Boomerang VersionvvboomerangversionnameboomerangVersion1.500.0mPulse boomerang.js version
Boomerang Snippet Versionsvsv----12mPulse Loader Snippet version
Boomerang Snippet Methodsmsm----s i if pmPulse Loader Snippet method (SCRIPT, IFRAME, IFRAME fallback, or Preload)
Navigation Start Methodrt.startrt.startparamsrtstartrtStartnone navigation cookie manual csiSpecifies the method used to determine the start time
Navigation Start Timert.tstartrt.tstartparamsrttstartrtNavigationStart1512086400000Timestamp of navigation start using the method specified in rt.start
Cookie Navigation Startrt.cstartrt.cstartparamsrtcstartrtCookieStart1512086400000The value of t_start from the cookie, if different from rt.tstart
Boomerang Startrt.bstartrt.bsta