The following table provides information about the bot categories:
Bot Category | Sub-categories | Description |
|---|---|---|
Crawler/Indexer |
| A web crawler is a robot, also called a spider, that visits your website looking for updates and new information, and uses it for their own purposes, such as a search engine. Examples include Google Bot, Apple Bot. |
Tool (Developer Tool) |
| All tools that can be programmed for any purpose, malicious or not. For example, scripts or command line tools such as Wget and lwp-request, or programming language libraries that enable web requests such as Java or Python urllib. |
Technical Partners/ Commercial Tools |
| Tools or services that send requests to a website for a positive purpose, usually by the site owner or host, such as health checkers, broken link checkers, performance measurement tools, and payment service providers. For example, Rackspace Monitoring Agent, Amazon Route 53 Health Checks. |
Social Media Agent |
| A client that fetches feeds, such as RSS feeds, or uses an API. Indexing website for internal search engines. Supplies visual support when a link is shared. For example, Yahoo Pipes, and Facebook External Hit. |
Known Violator |
| Known bad bots that are used to carry out attacks on websites. |
Browser Integrity |
| Bots not sending proper header key value pair, or sending random user-agent strings to avoid detection. |
Advanced Bots |
| Bots using the popular User-Agent Browser Automation tools, headless browsers, or web drivers. These bots are capable of executing JavaScript. |
Impersonator |
| Bots trying to mimic behavior of good bots, such as Google Bot and Bing Bot, to avoid detection. |
Advanced Persistent Bots |
| Bots that rotate IP addresses and user-agent headers to avoid detection. |
AI Bot |
| AI Bots utilize advanced machine learning and natural language processing (NLP) to generate, analyze, or enhance human-like text. These bots enable automated responses, content creation, and AI-driven assistance. However, they also pose risks such as unauthorized data scraping, intellectual property violations, and automated misinformation generation. With the rise of generative AI, unregulated bot activity raises transparency, security, and ethical concerns for online platforms. Blocking AI bots helps mitigate data theft, competitive exploitation, and compliance risks. |
Uncategorized |
| The Advanced Threat Intelligence engine cannot map the attack to one of the existing bot categories. |