Two HTML tags cut a web agent's task time from 27 seconds to 2

Web agents today behave inside other people's interfaces like uninvited guests: they stare at screenshots and guess which buttons can be clicked. The smallest interface update breaks the whole logic, drives up the cost of maintaining pipelines, and user privacy suffers along the way. The authors of VOIX propose a simple but far-reaching answer: let sites themselves hand agents the actions they are permitted to take, and describe — as up-to-date text — the state that matters for choosing among them. This is the foundation of the Agentic Web: a network where people and AI cooperate under transparent rules.

What VOIX is

The framework adds just two tags to HTML:

• <tool> describes an action by name and by strictly typed parameters. When an agent invokes a tool, the site's own business logic runs, and the result is handed back to the site through a return event.
• <context> is a compact text description of the state that matters for choosing an action. It can update dynamically as the app runs. A page can also declare several contexts.

Instead of guessing which affordances are available to it, the agent reads the protocol: here is what you can do, here are the parameters, and here is what is happening on the page. The site's developer keeps control, and the agent stops wandering blindly through the user interface.

How it is built

The roles are separated cleanly and visibly.

• The site is the source of truth about its own capabilities, and declares the tools and contexts. Past that point the familiar stack does the work, from React and Vue to Laravel.
• The browser agent finds the <tool> and <context> elements on the page, shows them in a side panel, passes their text descriptions to the LLM, and dispatches calls back to the page. In the open-source reference version this is a Chrome extension.
• The inference provider, meaning any LLM. The model leans on the protocol rather than on the visual interface to pick an action and its parameters given the user's goal.

Privacy matters here. The conversation the user sends for inference is never visible to the site — it goes straight to the LLM provider. The agent sees exactly the contexts and tools the site declared to it. The user can switch off the visibility of contexts in the Chrome extension.

Tested in practice: a three-day hackathon

Sixteen developers built six applications with tool and context protocols at a three-day hackathon. Tools and contexts were usually declared locally inside a component, and tools were visible only where it made sense to use them. A user could click an object and say “rotate this 45 degrees”, and the tool's code knew which object was meant.

On the SUS (System Usability Scale) survey the average score came to 72.34 — “good”. On the TOAST scale (Trust in Open-source Autonomous Systems Test, a scale for rating trust in AI systems) users judged the system understandable and capable. The main difficulty was not technical. Participants found it hard to decide which tools to declare: you have to pick the right level of abstraction.

The UI agents Perplexity Comet and BrowserGym were also tested on two realistic user-task scenarios across all six VOIX applications. What was tracked was the latency between the user's request and visual confirmation of the result.

The authors measured the latency between a request and visual confirmation of the result for VOIX, Perplexity Comet and BrowserGym. The contrast is about as sharp as it gets. VOIX's protocol tools carried out the command to create a blue triangle in 2.32 seconds; Comet needed 27.21 seconds, and the vision-based agents failed outright on a number of tasks. In another scenario, the Fitness App, putting together a weekly HIIT plan took 14.38 seconds with VOIX, against 229.52 seconds for Comet and 1271 for BrowserGym. VOIX saves the time because the agent simply calls a tool with the right parameters and gets an immediate answer. The VOIX agent can see the user's scenarios because developers declare tools at a higher level than the buttons that already exist in the interface.

How much control does VOIX give the site's developer without costing the user autonomy? The site declares only the tools it considers safe. The user chooses which contexts to reveal to the LLM provider. The provider itself is chosen in the Chrome extension. The compute does not land on the site owner; it goes to the cloud LLM the user selected. The site's creators get no access to the user's conversation. That transparency is achieved at the level of the protocol, not of policy.

Limits and what comes next

VOIX has a price. It takes discipline to keep tools in sync with the UI, and new testing methods to catch the drift between interface and tool. The tools also have to be designed properly: too low a level of abstraction declares a pile of simple buttons, while too high a level breaks on scenarios slightly off the expected path. Guidance on how to design tools well will be needed, and possibly shared tool catalogs.

The bottom line

VOIX turns a website into a machine-readable user policy: what the user permits sites to do on their behalf, with which parameters and for which tasks. If we want an internet built for agents, protocols like VOIX are the better place to start. They make the permitted actions transparent to both sides. The “trust broker” then becomes the end user rather than the LLM provider. Without such protocols, AI agents will go on groping their way around the web blind. With them, the internet becomes an environment where AI agents work precisely, safely, and by rules a human sets.