DeepL adds sarcasm and shifts in mood to speech translation

DeepL has introduced Aure, a model for its DeepL Voice live translation service. It carries over vocal qualities of the original speaker — tone, emotion, emphasis and pace — so translations sound more natural and preserve shifts in mood and sarcasm. Previously, the service generated speech in another language based on a voice sample recorded at the start of a conversation, and could not account for changes in intonation as the conversation went on. Aure continuously analyzes the translated text and the original audio.

DeepL adds sarcasm and shifts in mood to speech translation

How Aure works

During training, DeepL separates voice identity, speech melody and content. This lets each component be transferred independently to another language. Reinforcement learning helps remove accents from the generated speech.

Aure runs automatically in DeepL Voice for online meetings and in-person conversations, and through the DeepL API. At launch, the model supports English, German, Spanish, French, Italian, Japanese, Portuguese and Korean.

DeepL's other moves

DeepL first added voice preservation in September; the source does not specify the year. In May, the company cut about 250 employees and decided to reorganize around smaller teams using AI. DeepL aims to become a company built around AI from the outset.

DeepL also acquired the Mixhalo team, which specializes in audio streaming, to develop live speech translation. Block is pursuing a similar goal: the company recently introduced Dessa, a speech model that recognizes emotion and sarcasm directly from audio.

Dataist's take: Aure extends live translation to vocal qualities that can affect the meaning of what is said. At launch, however, the model covers eight languages, and the feature's availability depends on DeepL Voice and the DeepL API.