It seems like your browser didn't download the required fonts. Please revise your security settings and try again.
Barracuda Campus Barracuda Campus
Barracuda WAF-as-a-Service
Overview Documentation Training Certification Materials

Bot Protection

  • Last updated on

Barracuda WAF-as-a-Service detects bots and web scrapers and protects against automated attacks carried out by malicious bots. Bots with malicious intent such as stealing data, installing viruses and malware to your website, hijacking your bandwidth, submitting junk data through online forms, etc. can have an adverse impact on the performance and reputation of a company’s website. While these bots should be blocked, it is important that known good bots such as search engine crawler bots and SEO bots are allowed to crawl your web application.

The Bot Protection component includes the following sections:

Dashboard

Use the Dashboard page to enable bot protection and view the information about unique clients, unique IP addresses, and the geographical distribution of the incoming traffic. For detailed information, see Application-Specific ABP Details.

Set Enable Advanced Bot Protection to ON to enforce the specified bot detection checks on the incoming traffic.

Bot Attacks

Set Referrer Spam to ON to protect your application against automated attacks carried out by bots.

Bot Detection

In the Bot Detection page, configure honeypots to detect a bad bot and prevent it from crawling your website.

Honeypots

Honeypot is a security mechanism that creates a virtual trap to lure bad bots crawling your website with malicious intent. When honeypots are enabled, the Barracuda WAF-as-a-Service adds deceptive elements to your application to track the behavior of bots. The tracked information is shared with the Barracuda Active Threat Intelligence (ATI) for analysis. Based on the analysis, the traffic from a bot is allowed or blocked.

Configuring Honeypots
  • Insert hidden links in response - When enabled, the Barracuda WAF-as-a-Service embeds a hidden link in the response. The embedded link does not get displayed on the browser, so a human browsing the web pages through a common browser never sees and clicks the hidden link. Therefore, any request that attempts to access the hidden link is identified as an automated bot or scraper.

  • Insert disallowed URLs in robots.txt – Typically, every website includes a “/robots.txt” file that provides access instructions such as the user agents that are allowed to access the website, and the web pages that are allowed/disallowed to be accessed by bots.

    Example    :
    User-agent: *
    Disallow: /researchtools/abc/

    Here, User-agent: Asterisk (*) is a wildcard character and indicates that this website can be accessed by all bots, and Disallow: /researchtools/abc/ indicates that the bots are not allowed to access the /researchtools/abc/ page on the website.

    When Insert disallowed URLs in robots.txt is enabled, Barracuda WAF-as-a-Service inserts an encrypted URL under Disallow in the robotos.txt file. Any request that attempts to access the URL specified under Disallow is identified as a bad bot.

  • Insert delay in robots.txt – You can slow down the requests from a bot to a web application by setting the delay time (in seconds) between subsequent requests, so that server resources are not consumed and are accessible for legitimate traffic.

    When Insert delay in robots.txt is enabled, the Barracuda WAF-as-a-Service inserts “crawl-delay” in the robots.txt file with the specified Delay Time. All good bots should honor the delay time specified in the robots.txt file while accessing the web application. If not, it is identified as a bad bot.
Client Challenges

Enable Client Challenges to validate clients connecting to your application with CAPTCHA challenges to find out if a client is a regular browser, a bot, or a crawler. If the client does not solve the challenge within the specified attempts, it is identified as a bad bot and added to the Block List.

Configuring Client Challenges
  1. Enable – Set to ON to enforce client challenges to clients attempting to access your application.
  2. Enforce CAPTCHA – Select the enforce CAPTCHA option (Suspicious Clients Only or All Clients).
    1. Suspicious Clients Only - CAPTCHA is enforced for clients that exhibit suspicious behavior.
    2. All Clients - CAPTCHA is enforced on all clients accessing the application.
  3. Max CAPTCHA Attempts – Specify the number of attempts a client can make before failing to solve the CAPTCHA.
  4. Max Unanswered CAPTCHA – Specify the number of CAPTCHA instances that can be issued to a client IP address. This prevents an attacker from executing a DoS attack on the application by rendering CAPTCHA images without submitting the CAPTCHA response.
  5. Expiration Time – Specify the number of seconds a client IP can be idle before being challenged for CAPTCHA again.

Predefined Bots

Barracuda’s bot database contains over 10,000 known bots. All bots are categorized based on their behavior and moved to a predefined category. You can block specific bots from accessing your application under Blocked Bots and add specific bots to access your application under the Allowed Bots section.

Blocked Bots

Select the check boxes next to the predefined categories to block the bots of that category from accessing your application. Any request from these bots is blocked without being challenged.

Allowed Bots

Certain bots are beneficial to your website. The Barracuda WAF-as-a-Service provides predefined search engine bots that can be helpful to your website and users. You can add a bot definition to the allow list of bots either by selecting a bot definition from the Barracuda Advanced Threat Intelligence (ATI) service or by adding a custom bot definition. To add a bot definition, perform the following configuration.

To Add a Bot from the ATI Bot Library

  1. On the WAF-as-a-Service web interface, go to the APPLICATIONS page and click on the application to which you want to add bots.
  2. On your application page, click BOT PROTECTION in the left panel and then click Predefined Bots.
  3. On the Predefined Bots page, click Add Bot under Allowed Bots.
  4. On the Add Allowed Bot window:
    1. Definition Source – Select From Barracuda Bot Dictionary.
    2. Name – Specify the bot name that you want to search, or select a bot category and bot class to list the bots associated with the selected options.
    3. Bot Category – Select the required category from the drop-down list.
    4. Bot Class – Select the required class from the drop-down list.
    5. Click Search to view the list of bots associated with the selected bot category and class.
    6. In the table, select the check boxes next to the bot names that you want to add to the allowed bots list.
    7. Click Add.

ASN-based bots can be added to the allowed bot list.

 

To Add a Custom Bot

  1. On the WAF-as-a-Service web interface, go to the APPLICATIONS page and click on the application to which you want to add bots.
  2. On your application page, click BOT PROTECTION in the left panel and then click Predefined Bots.
  3. On the Predefined Bots page, click Add Bot under Allowed Bots.
  4. On the Add Allowed Bot window:
    1. Definition Source – Select Custom.
    2. Name – Enter the bot group name.
    3. User Agent – Enter the user agent expression. This expression is used to match against the User-Agent header in the requests.
    4. Domains or IP Addresses – Enter the domain name(s) or IP address(es) that needs to be matched against the client/client IP address.
    5. Click Add.