i
News
News · 2026-11-01

Microsoft patent describes AI that edits speech during live video

@neuronium_ai @neuronium_ai

Microsoft has patented a system that could alter speech and a speaker’s face during a live video, letting viewers hear a version tailored to their preferences. The patent describes AI agents that detect specified words, then replace or remove them while adjusting the speaker’s mouth and face to match. It also allows edits to the audio and video around each change, so the result sounds and looks smooth. The filing covers live events and augmented reality, but a patent describes a possible system, not a product Microsoft has announced.

Cover: Microsoft patent describes AI that edits speech during live video

What the system could change

The patent lists several uses, from practical translation to edits that raise harder questions about who gets to shape what an audience hears.

Translate a live broadcast so viewers can hear and see it in their preferred language.
Remove filler words such as “um” and “uh,” along with stumbles and speech disorders.
Delete “personal words and phrases,” including invented expressions, or colloquialisms such as “y’all” and “bite the bullet.”
Remove speech that is unclear, incorrect or out of place, as well as slips, offensive language and insensitive or inappropriate expressions.

The system may use internet searches to find replacement words. The original video and audio could also be recorded and stored as a “reference recording,” according to the patent.

Examples of possible settings include lectures, presentations, award ceremonies, game shows, cooking shows and sports broadcasts.

The edit is the experience

Video and audio editing are old tools. The change here is that AI agents could apply them automatically during a live event, making the edited version feel like the event itself. Instead of turning off a broadcast, a viewer could have someone else’s speech adjusted to suit their taste.

I think the more consequential question is not whether the system can remove filler words. It is what happens when its idea of an error reaches into dialect or phrasing. A tool that cleans up speech could also make a speaker sound less like themselves, while giving each viewer a different version of the same moment.

Microsoft did not immediately respond to a request for comment. The patent says a reference recording could help address misuse and confusion, but it leaves open how viewers would know what had been changed. A live event that can be quietly tailored to each audience may still be live in timing; whether it remains a shared experience is another matter.

Daily AI news

Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.

Only what matters — every day

Follow on X