Stream logs to TrafficPeak

Follow these steps to send DataStream 2 logs to TrafficPeak, the packaged solution for Hydrolix services built on top of ​Akamai​'s Connected Cloud infrastructure.

You can use TrafficPeak to visualize data collected in your stream on a customized Grafana dashboard for improved insight into the traffic on your properties.

Before you begin

Before configuring TrafficPeak as a destination to send logs, contact the Akamai account team to collect the following configuration details:

  • Ingest endpoint URL in the https://<host>/ingest/event?table=<tablename>&token=<token> format, including the HTTP streaming ingest token
  • Basic HTTP authentication username and password for your custom endpoint

If you want to collect logs for 100 billion edge hits in a month or more, see Multiple streams for additional guidance.

How to

  1. Log in to Control Center using a User ID and Password with DataStream access.

  2. Click ☰ and go to > COMMON SERVICES > DataStream to open the DataStream dashboard.

  3. On the DataStream dashboard page, click + Create stream.

    A stream creation wizard opens on the Configuration tab.

Choose properties

  1. In Name, enter a human-readable name for the stream.

  2. In Group, select the relevant account control group.

  3. In Contract ID, select the relevant identifier of your contract.

  4. In Include properties, select up to 100 properties that you want the stream to collect logs for. All properties must belong to the same product, for example, AMD or DD.

  5. Click Next to continue to the Data Sets tab.

Choose data set fields

  1. In the Data Sets tab, select the parameters that you want to log in your stream. For the full list of available data set fields, see Data set parameters

📘

Best practice

For streaming logs to TrafficPeak, we recommend choosing the Include all option to log all data set fields in these categories:

  • Log information
  • Message exchange data
  • Request header data
  • Network performance data
  • Cache data
  • Geo data
  1. In Log format, select the JSON format. For details on this step, see Log format.

    Click Show sample log line to view a sample of a log line with the parameters that you selected.

  2. Click Next to continue to the Delivery tab.

Configure TrafficPeak as destination

  1. In the Delivery tab, choose Custom HTTPS as the Destination to send and store logs.

  2. In Display name, enter a human-readable name for the destination (for example, TrafficPeak).

  3. In Authentication, select Basic, enter hydrolix as the Username, and provide the authentication Password.

  4. Check the Send compressed data box.

  5. Click Additional options to add custom header details. In Content type, choose application/json.

  6. Click Validate & Save to validate the connection to the destination and save the details you provided.

  7. Optionally, change the Push frequency to receive bundled logs to your destination every 30 or 60 seconds.
    Click Next.

Activate the stream

  1. In the Summary tab, review the stream details you provided earlier, including monitored properties, data set fields to log, and the destination to send log files.
  2. Check the Activate stream upon saving box to deploy the stream and activate it on the production network up to an hour minutes after saving, or leave the box unchecked and Activate a stream later.
  3. Click Save stream to save the stream for later. The stream starts activating if you checked the Activate stream upon saving box earlier.

📘

Enable DataStream behavior in Property Manager

Activating a stream takes up to an hour, but it will start gathering and streaming data only after you add and enable the DataStream behavior to the default rule in your property configuration in Property Manager. You can set the sample percentage of data (0-100) for this property for your streams using this behavior.

For details on this step, check Enable the DataStream behavior.

Multiple streams

For properties exceeding 100 billion hits per month in traffic, reach out to the Akamai support team for best practices. If needed, you can conveniently Clone a stream with the same destination configuration.