I asked Microsoft’s Copilot to book a dinner reservation for two at a Japanese restaurant on OpenTable. It spun up a cloud browser, searched, filled forms, and clicked through the site—getting all the way to the confirmation step before halting to demand a phone number and an SMS verification code I had to provide myself. The task eventually succeeded, but the experience laid bare the gap between the promise of autonomous AI agents and the messy reality of today’s web. Copilot Actions, Microsoft’s new consumer-facing agentic browsing feature, can genuinely perform multi-step tasks on live websites, yet it stumbles against the very barriers designed to stop bots: CAPTCHAs, multi-factor authentication, and region-locked content. It is an impressive technical demonstration—but it’s not ready to run your life.

The rise of AI agents that act on your behalf, not just answer questions, marks a pivotal shift in how we interact with the web. Microsoft, Google, OpenAI, and others are racing to build “web agents” that can navigate pages, fill forms, and complete transactions without constant human oversight. Copilot Actions, currently in limited testing, is Microsoft’s consumer answer to this trend. Under the hood, it provisions a disposable virtual machine in Azure, launches a cloud-based browser, and uses screen-capture analysis to “see” and interact with web pages just as a human would. Users watch the action unfold in a split view: a remote browser pane and a Copilot chat sidebar that explains what it’s doing. A “Take over” button lets you seize control whenever the agent gets stuck—which, in testing, happens often.

What works today is genuinely useful in narrow, low-stakes scenarios. Copilot Actions can search for products on e-commerce sites, compare listings, pre-fill public forms, and even walk through booking flows on platforms like OpenTable. In one PCMag test, the agent navigated Barnes & Noble’s website, asked clarifying questions about literary fiction, and found a 2024 bestseller—handling multiple intermediate steps before prompting for any personal input. The cloud-based architecture keeps local resource usage minimal, a boon for laptops and tablets. And the step-by-step transparency, with Copilot narrating its intentions, offers a degree of trust that fully autonomous black-box agents could not.

But the friction is pervasive. The most fundamental obstacle is the web’s own immune system. Sites deploy CAPTCHAs, SMS two-factor codes, bot-detection scripts, and login walls specifically to prevent automated access. Copilot Actions, by design, cannot bypass these legally or securely, so it must stop and request human intervention. In the dinner reservation test, the agent did everything up to the final verification, then forced the user to type in a code from a text message. For financial transactions or any flow requiring credentials the site hasn’t stored, the agent is helpless. This “last mile” problem means that, for many real-world tasks, manual completion remains faster—and often necessary.

Speed is another pain point. The cloud VM must spin up, the page must render, and the agent takes screenshots and analyzes them to decide the next click. That loop, while deliberate and cautious, adds latency that can make the automated path slower than a human using a local browser. In several side-by-side tests, performing the same task manually on a PC or phone took less time. The agent frequently pauses to ask for clarifications—“Which genre do you prefer?”—which, while helpful for ambiguous requests, erodes the hands-off advantage. Until the underlying models speed up inference and the browser automation becomes more seamless, the time savings will be marginal for all but the most complex, multi-tab workflows.

Location and context add another layer of friction. The cloud browser operates in a sandboxed environment that doesn’t inherit your local cookies, saved logins, or precise geolocation. In testing, Copilot Actions sometimes defaulted to an IP-based city that didn’t match the user’s actual location, skewing restaurant and store results. Microsoft could build secure bridges for non-sensitive context sharing, but that raises privacy trade-offs the company has yet to fully address. Right now, the agent is blissfully unaware of your bookmarks, your payment methods, and your ad-blocker settings—which can be both a blessing and a curse.

Privacy and security concerns are more than academic. Because the cloud browser captures screenshots of every page it visits to parse UI elements, sensitive data could theoretically appear in those captures. Microsoft says session data is encrypted and, in certain enterprise plans, not used for model training, but the specifics vary by subscription tier and region. A consumer using Copilot Pro may have different telemetry and retention policies than a Microsoft 365 enterprise customer with Copilot. Until these details are transparent and independently auditable, users should avoid feeding the agent sensitive personal information, credentials, or payment card numbers. The cloud VM is destroyed after each session, but what logs persist? Who can access them? Microsoft’s privacy FAQ nods to responsible handling, but the burden of proof lies with the platform.

Enterprise governance offers a clearer path, at least on paper. While Copilot Actions targets end users, Microsoft is simultaneously building developer tools for agentic workflows via Copilot Studio. The recent training module “Introduction to Tools for Declarative Agents in Copilot Studio” (the original source behind this discussion) details how organizations can create custom plug-ins that connect agents to internal data sources, apply data loss prevention (DLP) policies, and enforce admin oversight. This bifurcation—consumer actions for low-consequence tasks, enterprise actions with strict guardrails—is Microsoft’s pragmatic response to the inherent risks of autonomous browsing. For businesses, the promise is an agent that can automatically pull data from a protected SharePoint list to populate a form, but only if security policies are rigorously enforced. The documentation explicitly describes actions as “tools” that declarative agents can invoke, with connectors that respect organizational boundaries. This is the scaffolding on which a trustworthy agentic future could be built.

Competitors face the same conundrums. Google’s Project Mariner, unveiled in December 2024, takes a similar approach: cloud VMs, screen analysis, human-in-the-loop controls. Early demos show it booking flights and shopping, but also hitting CAPTCHA walls and asking for human help. OpenAI’s Operator, another closely watched project, promises to go further by learning workflows through observation, but it too must contend with the verification barriers baked into modern websites. The industry consensus is that fully autonomous web agents are technically feasible but will require a new trust framework—perhaps one where sites offer “agent-friendly” APIs or consent portals—before they can reliably complete high-security tasks without a human in the loop.

Regional availability further complicates the picture. Microsoft has withheld some Copilot features from the European Economic Area while it navigates the Digital Markets Act and other regulations. Copilot Actions may be subject to similar market-specific restrictions. If you’re in the EU, don’t assume the feature works; check Microsoft’s official rollout status first. The regulatory environment will shape how agentic behaviors are governed, especially around data residency and consent.

What should you do with Copilot Actions today? It’s a fascinating tool for low-risk experimentation. Try automating product searches, comparison shopping, or filling out non-sensitive public forms. Watch how it behaves, note the failure modes, and enjoy the novelty. For anything involving money, personal data, or credentials, keep your hands on the wheel. Enterprises should pilot the technology with strict DLP controls and audit logging, using the Copilot Studio toolchain to define explicit boundaries. The next 6 to 12 months will likely bring faster execution, better handling of MFA via secure credential vaults (perhaps integrated with Microsoft Wallet), and deeper governance. But the vision of an agent that seamlessly books your entire vacation, pays your bills, and answers CAPTCHAs on its own remains at least a year or two away—and will require not just technical breakthroughs, but a rethinking of how trust is brokered between humans, agents, and the websites they visit.

For now, Copilot Actions is a capable but cautious co-pilot. It can take the stick for a while, but it still needs the captain’s hands close by. That’s exactly the right posture for a technology with both enormous potential and genuine risk.