Not All Bots Are Created Equal: How to Differentiate and Manage Bot Traffic

Oftentimes, when you think of bot traffic, you automatically think the worst. Although bots can be dangerous and cause credential leaks, data theft, and DDoS attacks, not all bot traffic is created equal. Bots come from multiple sources — from hackers or competitors to certain authorized parties and even Google and other search engines. Being able to differentiate between bots can help give you the edge you need.

How to tell a good bot from a bad bot

Distinguishing between bot traffic and regular traffic is possible but laborious. Web engineers can look at network requests and analyze the traffic for certain indicators that point to bots, such as abnormally high page views, higher bandwidth usage, poor website performance, and increased server cost. Once you know the bots are there, the challenge is in figuring out how to easily categorize them.

Remember: not all bots are detrimental to your site’s operations or business; some can be beneficial. Key questions to ask when differentiating between the two bot types include:

• Is it coming from a well-known source?
• Are they transparent about the owner?
   

Once you’ve answered both questions, you can sort the bots into several categories within the two types. Good bots include:

• Search engine bots
• Partner/vendor bots
• Aggregator bots, and
• Price comparison bots
   

Bad bots include:

• Web-scraping bots
• Credential-stuffing bots
• Spam bots
• Ad fraud bots
• DDoS bots
• Credit card-fraud bots, and
• Gift card-fraud bots

Why you need a bot management strategy

Managing bot traffic goes beyond determining which bots you want to allow through your defenses. It gives you the competitive advantage needed to provide the best digital experiences that users expect. With high bot traffic and no way to manage them, you sacrifice site performance. You also make it that much more difficult to determine how your site is actually doing in terms of page views or bounce rates and being able to conduct accurate A/B testing or optimize your conversion rates.

But it doesn’t stop there. Without a proactive line of defense, your brand’s reputation could be vulnerable to bots programmed to perform fraud or gain access to your users’ accounts. Even worse, a DDoS attack could slow down or crash your site entirely. You sacrifice both time and effort while resurfacing your site, as well as the site traffic you could have gained had your website functioned properly. Managing the traffic that hits your site safeguards you from issues caused by bad bots, affording you the advantages that beneficial bots confer.

Take control of your defense with Acquia Edge Bot Manager

Rather than paying for a team of engineers to constantly monitor your traffic — couldn't you use those funds on something else? — use a bot management tool for a superior level of control and security. Acquia’s Edge Bot Manager allows you to control the automated bots that interact with your site, be they good or bad. It strengthens your defense and empowers you to:

• Leverage bot intelligence. Access a directory of more than 1,400 known bots in 17 categories that commonly interact with other customers.
• Automate with machine learning. Automatically update the characteristics and behaviors used to identify bots.
• Conduct bot-centric reporting and analysis. Access real-time and historical reporting on bot traffic to better understand bot trends.
• Detect unknown bots. Discover traffic from unknown bots using a variety of techniques, such as user-behavior analysis, browser fingerprinting, automated browser detection, and more.
   

With Bot Manager, your visibility into the characteristics of bot traffic increases, enabling you to automatically filter malicious traffic without increasing IT overhead to do so manually. Ultimately, preventing malicious bots from reaching your site not only protects customer relationships by avoiding price and content scraping, it also also reduces the load on your web infrastructure so you don't pay for unnecessary growth caused by nonhuman site interactions.

If you’re interested in learning more, please reach out to your Acquia account manager to determine if Acquia Edge Bot Manager is right for your organization.