The Rise of Personal Agents (and the Traffic You Can't See)

Personal AI agents are becoming the middleman between your users and your website. A customer might ask their agent to compare products, find a support article, or complete a checkout. To your servers, all of that looks like automated traffic — and until now, you had no clean way to tell whether it was a helpful customer proxy or a risk.

Blocking all agents is the easy answer, but it also blocks real customers. Cloudflare's new tooling aims to fix that blind spot. According to the official Cloudflare announcement, the goal is to give site owners visibility into which agents are visiting and control over what they can do.

Cloudflare dashboard showing personal AI agent traffic analytics and bot categories

What's New: Analytics + Policies

1. Personal Assistant Visibility in Security Analytics

The Bot Analysis chart now includes a Personal Assistants category alongside Verified Bot, Likely Human, Likely Automated, and Automated. A new Top Statistics widget lists which agents hit your site and how many requests each sends. Filter on one agent and every other widget narrows down: paths requested, status codes, and source networks.

For sites using Account Abuse Protection (early access), you can even see which user account an agent represents — useful for fraud teams investigating unusual logins.

2. Granular Access Policies

The existing AI Crawler and Agent access policies setting (renamed from AI bot policies) now supports per-agent rules:

  • Allow (do not block)
  • Block on pages with ads
  • Block

You can set an overall Agents policy and then override it per supported personal agent. Example config:

{
  "agents_policy": "block_on_ads",
  "per_agent_overrides": {
    "supported_personal_agent_x": "allow",
    "supported_personal_agent_y": "block"
  }
}

These choices are also offered during zone onboarding, tailored to whether you monetize with ads.

Network security settings page with AI crawler and agent access policies Software Concept Art

Limitations and Caveats

  • Identity is only half the story. Cloudflare can tell you which agent is calling, but not who the user is — whether they're a paying customer, what they authorized, or what protections apply. That context is the next frontier.
  • Supported agents only. Granular per-agent policies apply to supported personal agents; unsupported ones fall back to the general policy.
  • Account Abuse Protection is early access, so smaller teams may not have access to the per-user agent views yet.
  • Policy sprawl risk. With per-agent overrides, it's easy to lose track of what's allowed. Document your intent.

Next Steps for Learning

  • Audit your current bot traffic before changing policies — you may be surprised how much is already agent-driven.
  • Pair this with WAF rules on Hashed User IDs for surgical mitigation instead of blanket blocks.
  • If you run a content site, compare ad-supported pages vs. paywalled pages and set different agent policies for each.

For a deeper look at how AI-driven personalization is reshaping recommendation systems, see our analysis of Airbnb's JourneyFormer and transformer-based personalization.

AI assistant agent interacting with website requests on behalf of a user Developer Related Image

TL;DR

Personal agents are customers in disguise. Cloudflare now gives you the lens (analytics) and the lever (policies) to treat them accordingly. Review your agent traffic, decide per-agent and per-page rules, and revisit as the ecosystem matures.

If you're also evaluating the business case for data platforms powering these agent workloads, check out our breakdown of Azure Databricks' 331% ROI per Forrester TEI.

This content was drafted using AI tools based on reliable sources, and has been reviewed by our editorial team before publication. It is not intended to replace professional advice.