About bots

Welcome the bots you want and mitigate those you don't.

Why manage?

More than half of web traffic is generated by bots. Some bots, like search engine crawlers are good bots. Others, like those that steal data, generate spam, or repeatedly try login credentials, are bad bots. Both good and bad bots can impact your site's performance, so Bot Manager helps you handle bots. You can:

  • Detect that bots are active on your site
  • Categorize bots based on what they do and the impact they have
  • Manage by applying different actions to different bot requests
  • View your bot traffic and analyze how effective your management strategies are

Assess your bot problem

Evaluate your website or API in terms of where bots could cause you problems and where you may want to tolerate them. For example, if you're an airline, you may want to make flight search off-limits to bots, but let bot traffic access flight schedule and status.

Airline flight example

Choose your bot solution

  • If your needs are simple, and you just want to see what bots are hitting your site, then mitigate or allow them, you can use the bot protections that come included with App & API Protector. Learn more (login required).
  • If any URLS are APIs or transactional pages that handle things like login, signup, or checkout, then you want to take it a step further and apply behavioral detection to specific endpoints you define as API resources. To combat the most sophisticated adversarial bots attacking web apps like this, check out Bot Manager Premier (login required).
  • If scrapers are hurting your site's performance or undercutting your prices, turn to Content Protector(login required), our most effective solution yet. This product can fend off the most sophisticated web scrapers.