Microsoft's Universal Translator Preserves Inflection, Intonation and Accent

It’s not hard to imagine wonderful applications for automated voice translation. Embed the software in a video chat program and call China; listen as an app translates dubious menu selections read by a waiter at a foreign restaurant.

No wonder Google and Microsoft are pushing research dollars into the technology. Any company that can master universal translation will have a massive hit.

Now Microsoft is demoing a translation prototype that listens to your voice and spits out your words in real-time – in Spanish or Mandarin, all while preserving inflection, intonation, and accent. Though the technology is a big step forward, it’s certainly not an end to the race. Microsoft's prototype requires about an hour of voice input from the user to garner enough data, but the result is a digital voice, in another language, that sounds eerily like your own.

To develop its so-called universal translator, Microsoft built on several preexisting tools. The system starts with voice recognition software, uses Bing for translation, and then taps into text-to-speech technology to vocalize results. The tricky part is personalizing the voice. The recorded native language is split up into 5-millisecond chunks, and the individual sounds are remixed and used to train the program in the target language.

Microsoft showcased the translator at its 2012 TechFest, translating Chief Research Officer Rick Rashid’s opening comments, and following up with a translation of Chief Research and Strategy Officer Craig Mundie.

The system's translated voices still have a slightly inhuman, digital quality to them, but it’s faint – closer to Siri than old computer-generated monotones. Given the system's lengthy reference-building requirements, Microsoft still has its work cut out, but as – video of the demonstration shows (starting at around 12:00 in), this technology is getting closer to fruition.