Alpha-Service turns AI glasses into an assistant that speaks first

Unlike today's voice assistants, the AI for Service team proposes a more interactive approach. They argue that an AI should recognize on its own when a person needs help and offer it without being asked. That approach, which they call proactive assistance, was demonstrated on AI glasses streaming first-person video.

What it's for

In daily life we rarely manage to phrase a request at the right moment: thoughts flicker past, hands are busy, the situation keeps shifting. A proactive AI has to be able to:

• Identify the right moment to step in;
• Offer a short, general piece of advice when that is enough;
• Give priority to personal guidance when settled habits and long-term preferences are in play.

The researchers define proactive AI behavior through two key parameters: the moment of intervention and the method of assistance.

How it works

The proposed Alpha-Service architecture, a nod to von Neumann's classic but still relevant design, consists of five blocks that cover the whole path from taking in information to producing a response.

• Input: continuous perception of first-person video. A lightweight model catches triggers in real time; a heavier one parses the details of the scene on demand. It is a balance of speed against depth.
• CPU: the dispatcher brain, built on a fine-tuned LLM. It decides what to do: answer right away, wait for the right moment, go to memory, or call tools.
• Memory: simple but useful long-term storage — history, habits, short summaries of past interactions. Whatever is relevant gets mixed into the prompt for personalization.
• ALU: a set of tools, from web search to specialized models. They are brought in only when confidence runs short.
• Output: a short, clear answer, spoken through a local synthesizer when that is needed.

A multi-agent system splits up the roles — perception, planning, computation, memory, delivery — and keeps latency on the glasses low.

What it does in practice

1) Real-time blackjack hints. The glasses catch the moments when the total on the table changes and deliver a short piece of advice that accounts for the odds and the dealer's card. This is not magic but cold arithmetic, and it arrives before your hand moves toward the pile.

2) A museum guide that knows what you have been staring at. The system notices your interest in an exhibit, pins down its visual features, runs a targeted search if it has to, and delivers a compact explanation of where the artifact came from and what it meant.

3) Fit and style advice in a store. The glasses see you turning a dark sweater over in your hands and slip you a short cheat sheet: the fabric, the sizing, what it goes with. No lectures, no filler.

Under the hood: a word on the models

• Triggers and quick hints run on a lightweight multimodal model that reads the stream continuously and flags "points of attention".
• Deeper analysis goes to a more capable multimodal LLM, which switches on only when it is genuinely needed.
• The CPU role is played by a fine-tuned LLM: it builds the plan, decides when to stay quiet and when to speak a hint aloud, and reaches for tools.
• Web search is already wired in as a tool: results are compressed and folded into the reasoning. Memory holds anchor facts and preferences, and the output is short and friendly, spoken locally.

The point is not answers on demand but the timing of the intervention. The system learns to see temporal patterns: when a delay ruins the experience, and when it is better to say nothing at all. It combines generic hints that work for anyone with personal ones meant for you. And it does this on a device where resources are always tight.

Where this goes next

The authors see potential in deeper memory and personalization, in a wider set of tools, and above all in large-scale user studies. The goal is simple: an assistant that reads your context more precisely as the years pass and anticipates needs carefully, without taking away your autonomy. Research like this is what turns the fantastical future of the movies into something real.