Mistral AI launches Voxtral, an open source text-to-speech model

On Thursday, French AI company Mistral AI released Voxtral TTS, a new open source text-to-speech model. The model is designed to convert text into spoken audio, allowing enterprises to build voice agents for customer support, sales, and other engagement use cases. Voxtral TTS supports nine languages and can adapt to a custom voice using an audio sample of less than five seconds. According to the company, the model is capable of capturing subtle characteristics of human speech, including accents, inflections, intonations, and flow irregularities. This capability allows the model to switch between languages easily while preserving the original voice’s unique characteristics.

The model is built on the Ministral 3B architecture and is optimized for real-time performance. It features a time-to-first-audio (TTFA)—which measures when the model starts speaking after receiving input—of 90 ms for a 10-second sample of 500 characters. Additionally, Voxtral TTS has a real-time factor (RTF)—a measure of rendering speed relative to clip length—of 6x. This means the model can render a 10-second clip in roughly 1.6 seconds. This high-speed rendering makes the model suitable for low-latency applications like dubbing or real-time translation without sounding robotic.

Pierre Stock, VP of science operations at Mistral AI, noted that the model was developed in response to customer demand for a speech model that can run on edge devices, which include hardware like smartwatches, smartphones, and laptops. “Our customers have been asking for a speech model. So we built a small-sized speech model that can fit on a smartwatch, a smartphone, a laptop, or other edge devices. The cost of it is a fraction of anything else on the market, but it offers state-of-the-art performance,” Stock said.

While the current release is a text-to-speech model, Mistral AI has broader plans for its voice technology. Stock stated that the company plans to have an end-to-end platform that can handle multimodal streams of input and output, including audio, text, and images. According to Stock, the primary benefit of this planned end-to-end agentic system is that it will allow users to obtain significantly more information by supporting audio as both an input and an output.

Why it matters

The release of Voxtral TTS positions Mistral AI to challenge established voice AI providers like ElevenLabs, Deepgram, and OpenAI. By offering an open-source alternative that can run on hardware like smartwatches, smartphones, and laptops, Mistral AI provides enterprises with a customizable option for building voice agents.