Bot Insights

Bot Insights is a TrafficPeak solution that reveals how bots interact with your web properties, using the DataStream 2 delivery logs you already send. It works alongside enforcement tools such as Akamai Bot Manager. Those tools perform detection and blocking, while Bot Insights answers three core questions:

  • What changed in bot traffic
  • What's driving the change
  • What it's costing

Bot Insights classifies the bot traffic in your CDN delivery logs and surfaces the results in six job-specific dashboards. If you have Bot Manager and send its events through Akamai SIEM, Bot Insights adopts Bot Manager's verdicts and adds its own user agent classification and delivery metrics.

How Bot Insights works

Enabling Bot Insights adds the following resources to your TrafficPeak environment:

  • Bot classification logic that runs on your DataStream 2 logs during ingest
  • A dictionary that the classification logic references
  • Summary tables that aggregate bot activity by minute, hour, and day
  • Grafana dashboards

During ingest, each log record is checked against a dictionary to derive the bot classification columns that the dashboards surface. The summary tables aggregate these dimensions to keep dashboard queries fast at scale.

Prerequisites

  • A TrafficPeak environment that ingests DataStream 2 delivery logs
  • Breadcrumbs enabled for the properties in your DataStream 2 stream
  • If you're using the Bot Manager tier, an active Akamai Bot Manager subscription and an Akamai SIEM integration

Enable Bot Insights

Akamai provisions Bot Insights for you. You don't need to install or configure anything in your systems.

  1. Contact your Akamai account team and request Bot Insights.
  2. Your account team submits the enablement request.
  3. TrafficPeak provisions the classification logic, dictionary, summary tables, and dashboards in your environment.

Dashboards

Bot Insights includes six dashboards, each organized around a specific job. In Grafana, find them under Dashboards > TrafficPeak Certified Reference Dashboards > Security > Bot Insights.

DashboardJob
Overview and AlignmentIs the data fresh, and is the situation real?
What Changed + InvestigationWhat changed in automation, and what needs response?
SOC What ChangedHow is bot pressure impacting error rates, cache miss rates, and other key metrics?
SEO GovernanceAre good bots healthy, and are AI crawlers behaving?
Edge/OpsWhat are bots costing in infrastructure?
Multi-Domain TriageWho owns the issue, and does it need to be escalated?

Each dashboard follows the same three-layer structure.

  • A high-level orientation view that shows whether anything needs attention
  • Dimensional breakdowns that reveal what's driving a change
  • Detailed recent logs that support a response

Evidence sources and capability tiers

The value you get from Bot Insights depends on which data sources you send to TrafficPeak. Two evidence sources feed the dashboards and combine into two capability tiers.

TierDataValue
CDNDataStream 2 logs + dictionaryBot traffic mix, classification, volume trends
CDN + Bot ManagerBot Manager + DataStream 2 logs + dictionaryBot Manager verdicts and scores, with enforcement outcomes

The dashboards degrade gracefully across tiers: panels that need Bot Manager evidence populate only when that data is present.

Bot classification

Bot Insights identifies bots in different ways, depending on your tier.

  • CDN tier: Bot Insights matches the user agent string against a dictionary built from Matomo Device Detector, which identifies more than 800 individual bots across 20 categories.
  • Bot Manager tier: Bot Manager's verdict decides whether a request is a bot. User agent matching supplements it only for benign crawlers that Bot Manager allows.

Each request receives four classification dimensions. They appear as columns in your log data and as filters in the dashboards.

  • biTrafficCohort: Whether the request came from a human, an AI crawler, or another kind of bot
  • biUserAgentCategory: The kind of bot, such as AI LLM Crawler, Security Analysis, or Social Media Crawler
  • biAiCategory: The AI vendor behind an AI crawler, such as OpenAI or Anthropic
  • biBotProducer: The organization that operates the bot, such as Google Inc. or Pingdom AB

Bot Insights and enforcement tools

Bot Insights is an intelligence and visibility layer rather than an enforcement tool. It works alongside inline enforcement products such as Akamai Bot Manager.

Bot ManagerBot Insights
Enforcement: blocks or allows traffic in real timeAnalytics that reveal patterns over time
Optimized for low false positives. Lets some bot traffic through.Shows what got through and why
Reports what it blockedDetermines what should be blocked but is making it through
Action-orientedInvestigation-oriented

Bot Insights LLM skill

The open-source AI Toolkit includes a dedicated Bot Insights skill for use with many AI agents. See the toolkit installation instructions for supported agents. The skill connects to your data through the TrafficPeak MCP server, investigates bot behavior, and generates evidence-backed reports tailored to different stakeholders.

Six report types answer different questions about bots.

Report typePurpose
Executive postureHigh-level summary for executive and board alignment
Control reviewValidates whether bot management policy changes are producing intended results
SOC triageSecurity operations attribution and spoofing detection
Scorecard briefEntity scoring and reporting
Crawler governanceGood-bot health and AI crawler compliance assessment
Edge/Ops impactInfrastructure cost and cache efficiency analysis

Related information


Did this page help you?