10 Questions to Ask a Bot-Mitigation Vendor

You suspect that you have a bot problem. Maybe you have increased infrastructure costs due to traffic surges, a high account takeover (ATO) rate, or someone’s cracking your gift cards and scraping your intellectual property. If you dig deeper, you will likely also find a fraud problem, and recent falls in brand scores and customer loyalty. You tried to handle it yourself with rate limiting, IP and geo-blocking, reputation, fingerprinting, CAPTCHA, and Multi-Factor Authentication (MFA). But it became an endless battle managing multiple solutions, trying to stay ahead of attackers, and dealing with frustrated customers who were unable to complete their purchases.

Now you’ve decided to call in professionals to help, and you’ve narrowed it down to a few vendors. Here are some questions that can help you decide if the bot mitigation vendor’s solution is the right fit for your environment.

1. How does the vendor handle attacker retooling?

If the perceived value in your customer accounts is high, attackers likely won’t give up easily, but will continuously retool and try again. This is basic attacker economics, which means this is your most important question. When a security countermeasure is put in place, persistent attackers will retool to bypass the mitigation control using various methods, tools, and even AI. Victims of credential stuffing say that fighting bots and automation by themselves is like playing whack-a-mole. You are paying a vendor to play this game for you, so ask how they would handle it.

2. Does the vendor dramatically increase customer friction?

CAPTCHA and MFA dramatically increase friction for your customers. Human failure rates range from 15% to 50% and lead to high cart-abandonment and decreased user satisfaction. And your customer may never come back, even after a single negative user experience.

So, think carefully about vendors who rely on countermeasures known to introduce friction that will frustrate users, and attackers can often get around them anyway. Fraudsters can utilise tools and human labor to solve CAPTCHAs, and leverage compromised PII to impersonate account holders in order to move phone lines to accounts under their control to complete MFA requests.

3. How does the service deal with false positives?

A false positive for a vendor is when they mark a real human as a bot. A false negative is when they mark a bot as human.

Bot mitigation will have some of both. But a vendor should be very responsive to the problem of false positives, and you should be able to contact them, complain, and have it addressed without delay.

4. When an attacker bypasses detection, how does the vendor adapt?

A bot mitigation vendor must operate as if a skilled attacker will imminently bypass all countermeasures. You may not know it has happened  until you see the side effects (account takeover, fraud, skewed business analytics, etc.). Your vendor will then need to provide a quick turnaround and work with you to remediate the problem.

5. How does the vendor handle manual (human-driven) fraud?

A determined, skilled attacker will input credentials by hand in real browsers to bypass anti-automation defenses, potentially leading to Account Takeover (ATO) and Fraud. And the vendor must be able to determine if a human is a trustworthy customer or a fraudster and take the appropriate action.

6. If one customer gets bypassed, how does the vendor protect that bypass from affecting all other customers?

In many cases, custom detection and mitigation policies should be deployed for every customer. That way, if an attacker retools enough to get around the countermeasures at one site, they can’t automatically use that playbook to get into your site. Each customer should be insulated from a retool against a different customer.

7. If an attacker bypasses a mitigation, does the service still have visibility into the attack?

The most effective bot mitigation solutions continuously collect and analyze various device, network, environment, and behavioral telemetry signals to maximize visibility and accurately identify anomalies. This, in turn, improves efficacy of closed-loop AI models while providing key insights to the vendor’s Security Operations Center (SOC).

8. How difficult is the solution to deploy and maintain?

Does the user or administrator have to install custom endpoint software, or is protection automatic? If there is no endpoint presence how does the vendor detect rooted mobile devices? How does it detect attacks using the latest security tools and data from the Dark Web? What about APIs?

9. What types of anomalies does the vendor detect?

Attackers are constantly leveraging bots, automation, and compromised credentials to assist with their efforts, with the endgame being financial gain. Basic mitigations are not enough. For example, attackers re-use IP addresses, but typically only 2.2 times on average. Often, they are only used once per day or once per week! This makes IP blocking largely ineffective.

Attackers typically invest along four vectors:

• Spoofing network traffic
• Emulating a variety of valid devices and browsers
• Using stolen credentials, PII, and synthetic identities
• Emulating and exhibiting actual human behavior

A good service will leverage a variety of signals and AI to provide actionable insights and detect anomalous behavior indicative of fraud—including copying and pasting activity, screen toggling, odd screen real estate usage, device affinity, environmental spoofing, and attempts to anonymize identity.

10. How quickly can the vendor make a change?

When the attacker retools to get around current countermeasures, how quickly will the vendor retool? Does the vendor charge extra if there is a sophisticated persistent attacker and multiple countermeasures or consultations with the SOC are needed?

Conclusion

There are other questions that are table stakes for any vendor. Things like deployment models (is there a cloud option?) and cost model (clean traffic or charge by hour?). And, of course, you should compare the service level agreement (SLA) of each vendor.

We’ve spoken to hundreds of customers. These are questions they asked, and we hope that they help you, even if you end up choosing a different bot mitigation vendor.

Byron McNaught, Senior Technical Marketing Manager at F5