Breadcrumbs

The "Breadcrumbs" feature looks to improve real-time visibility of performance-impacting and error conditions for content delivery.

What does it do?

Breadcrumbs lets ‚ÄčAkamai‚Äč quickly do two things:

  1. Help you troubleshoot problems. To troubleshooting a delivery issue, you typically need to contact us and provide client-side and ‚ÄčAkamai‚Äč edge IP information. We consult log information, along with some amount of back-and-forth communication to diagnose the issue. Breadcrumbs gives you more real-time visibility into the ‚ÄčAkamai‚Äč network and it helps us better troubleshoot your issues. While ‚ÄčAkamai‚Äč customer support can simply look at a specific line in a specific log file to identify a problem, they have to locate this "specific" content in the logs. With Breadcrumbs, a potential problem is specifically written as a "breadcrumbs" line in the logs, making it easier to locate.

  2. Leverage Breadcrumbs data. If your delivery environment can support it, you can configure it to get real-time insight regarding the ‚ÄčAkamai‚Äč network. Breadcrumbs includes data such as network health and the location in the ‚ÄčAkamai‚Äč network ("geo") used to serve content. You can aggregate this data from the requesting client to a central location for review.

ūüďė

The actual root cause of an issue may not always be identifiable via breadcrumbs.

The bottom line

The Breadcrumbs service provides per-HTTP transaction visibility into the ‚ÄčAkamai‚Äč platform, regardless of how deep within that platform, and simplifies log review for troubleshooting.

Add Breadcrumbs

Get its behavior added to your delivery configuration ("property").

  1. Create a new property, or edit an existing one using Property Manager.

  2. In the Property Configuration Settings options, click Add Behavior.

  3. Input Breadcrumbs in Search available behaviors to filter, and select it from the list.

  4. The new behavior is added to your configuration. Set Enable Breadcrumbs to On.

Opt Mode

Use Opt Mode to conditionally Opt-IN or Opt-OUT of breadcrumbs support on a per-request basis, using a specific query string.

  • Opt-OUT. Breadcrumbs data is included in the response header for every client request. You can withdraw ("opt-out") on a per-request basis by including the ak-bc=off query string in that request.

  • Opt-IN. Breadcrumbs data is not included in responses. You can include it ("opt-in") on a per-request basis by including the ak-bc=on query string in a request.

Review this table to understand how the query strings are applied, based on your Opt Mode setting.

Query string parameterOpt-Mode settingIs Breadcrumbs data included in response?Description

ak-bc=on

Opt-IN

Yes

By default, Opt-IN blocks breadcrumbs data from all responses. By including this query string, the response for this request will contain breadcrumbs data.

Opt-OUT

Yes

The ak-bc=on parameter and "Opt-OUT" setting in your property configuration are redundant values.

ak-bc=off

Opt-IN

No

The ak-bc=off parameter and "Opt-IN" setting in your property configuration are redundant values.

Opt-OUT

No

By default, Opt-OUT includes breadcrumbs data in all responses. By including this query string, breadcrumbs data will be left out of the response to this request.

None

Opt-IN

No

If no parameter is included, the "opt-IN" method responses will not include breadcrumbs data in all responses. (You need to include the query parameter in a request to "Opt-IN" and receive the data.)

Opt-OUT

Yes

If no parameter is included, the "Opt-OUT" method includes breadcrumbs data in all responses. You need to include the query parameter in a request to "Opt-OUT" of receiving the data.

Why "opt-IN" or "opt-OUT?"

You can simply have Breadcrumbs enabled in your property configuration and set Opt Mode to Opt-OUT to have breadcrumbs applied for all requests that match your rule criteria. Breadcrumbs include a specific header in the response to a requesting player. So, this increases the number of bytes delivered. Plus, a player just may not want (or support) the breadcrumbs response header. So, you can still have Breadcrumbs enabled and include opt-IN or opt-OUT to let the individual players determine how to handle the responses on a per-request basis.

Deliver Breadcrumbs via Logs

The other options here are used to determine how Breadcrumbs data is included in response headers to a requesting client. If you're a content owner with no insight into these response headers, enable this to include all Breadcrumbs data in logs, as well. This can also be helpful if you're using DataStream 2 to retrieve log data. This way, all Breadcrumbs data is carried in the logs it uses.

ūüďė

The Opt Mode setting doesn't apply

With this enabled, Breadcrumbs data is always passed in the logs. The Opt Mode setting and its query strings only affect how Breadcrumbs data is included in the response header that's sent to a requesting client.

Caveats and known issues

If you want to add support for Breadcrumbs, consider the following:

IssueDetails

Response headers can't exceed 8 KB

If you enable Breadcrumbs, response headers delivered during a request will increase in size. If the overall size of these response headers exceeds 8 KB, an edge server will return an HTTP 502 status code. To avoid this, minimize additional data in your response headers to keep them well under this 8 KB maximum size.

XHR-based players

If you're using an XMLHttpRequest (XHR)-based player, ensure that the "Access-Control-Expose-Headers" contains the name of the Breadcrumbs response header.

  1. Click the Default CORS Policy rule.

  2. Locate the Modify Outgoing Response Header entry with its Select Header Name set to Access-Control-Expose-Headers.

  3. Ensure that Akamai-Request-BC is included in the list.

Cache Key Query Parameters

With Breadcrumbs, the player can send query parameters to control the opt-IN or opt-OUT aspects of receiving associated data in the response header. This parameter is not supported for use in the cache key. So, if you're including the Cache Key Query Parameters behavior, ensure that opt-IN and opt-OUT are left out.

The Breadcrumbs response

The Breadcrumbs response header that's sent to a player includes various sub-fields of data.

Sub-field keySub-field valueDetails

a

<Component IP>

The IP address of the node/host currently processing the request. This can be an edge server or an origin.

ūüďė

If the node or host is not either of these two types, it's excluded.

b

<Request ID>

A unique identifier that's assigned to the request. This helps ‚ÄčAkamai‚Äč support teams quickly pull the relevant log line in the request.

k

<Request End Time>

The number of milliseconds between the time the connection is accepted and the breadcrumb was gathered. This time includes the initial metadata parsing time, if metadata hadn't been loaded, as well as the metadata application time for this request, if they occurred between request acceptance and generation of the breadcrumb.

l (lower case L)

<Turn around time>

The time in milliseconds between receipt of the end of the request headers and when the breadcrumb was gathered. This includes the time it takes to fetch the object from another server in the case of a miss, or to perform (synchronous) validation of the object's freshness if the object was in cache. In both of these cases, there's a forward request corresponding to the client request. This time also may include the time to fetch the object from disk, perform ESI processing, and compute response headers.

m

<DNS LookUp Time>

The amount of time between the start of the request and the completion of the DNS lookup.

n

<Component Geo>

Geographic details for the node or host currently processing the request.

o

<Component ASN>

The autonomous system number of the current node or host.

{variable letters}

<Component Letter>

These are letter identifiers for the ‚ÄčAkamai‚Äč network component that was involved during that phase of the request. Your ‚ÄčAkamai‚Äč account representative can use this value to determine where in the request phase the breadcrumb was gathered. Component letters include the following:

  • c. This is at the cache parent with an error range of 10000-19999.
  • g. This is at the edge ghost with an error range of 20000-29999.
  • p. This is at the peer ghost with an error range of 70000-79999.
  • o. This is at the origin with an error range of 80000-89999.
  • w. This applies to ‚ÄčAkamai‚Äč's Cloud Wrapper service, if you've implemented it in your environment. Error codes can range from 90000-99999.

Example 1: When served from within ‚ÄčAkamai‚Äč

In this example, the request flow is Edge to Peer. So, the Breadcrumbs format would be in this order, with each enclosed in brackets, and separated with commas:

  1. Edge: [{component-ip},{request-id},{component-letter},{component-geo},{component-asn}]
  2. Peer: [{component-letter},{component-geo},{component-asn}]:
[a=12.34.567.89,b=12345678,c=g,n=US_CA_SANJOSE,o=20940],[c=p,n=US_CA_SANJOSE,o=55155]

Example 2: When served from an origin

In this example, the request flow is Edge to Cache to Origin. So, the Breadcrumbs format would be in this order, with each enclosed in brackets, and separated with commas:

  1. Edge: [{component-ip},{request-id},{component-letter},{component-geo},{component-asn}]
  2. Cache: [{component-letter},{component-geo},{component-asn}]
  3. Origin: [{component-ip},{component-letter}]
[a=12.34.567.89,b=12345678,c=g,n=US_CA_SANJOSE,o=20940],[c=c,n=US_CA_SANJOSE,o=55155],[a=987.654.321.09,c=o]