AI Babel Fish turns into actuality, permitting direct speech-to-speech translations

January 15, 2025

The GIST Editors' notes

This text has been reviewed in accordance with Science X's editorial course of and insurance policies. Editors have highlighted the next attributes whereas making certain the content material's credibility:

fact-checked

peer-reviewed publication

trusted supply

proofread

AI Babel Fish turns into actuality, permitting direct speech-to-speech translations

An AI mannequin that may translate speech and textual content, together with direct speech-to-speech translations, for as much as 101 languages is described in Nature. The mannequin, named SEAMLESSM4T, fills gaps in language protection and outperforms present programs. The work might pave the way in which for speedy common translations, with assets being made publicly obtainable (for non-commercial use) to help additional analysis on inclusive speech translation applied sciences.

Readers of science fiction could be accustomed to the Babel Fish from The Hitchhiker's Information to the Galaxy, a small fish that could possibly be inserted into an ear and concurrently translate from one spoken language to a different. Such a software can be invaluable in facilitating communication in an interconnected world panorama, however most present machine studying translation programs are textual content oriented, or contain a number of steps-speech recognition, translation into textual content, and conversion of textual content to speech.

As well as, language protection for present speech-to-speech fashions falls behind that of text-to-text fashions and tends to be skewed in the direction of translating from a supply language into English, relatively than from English to a different language.

Addressing these limitations, the Seamless Communication Crew from Meta have developed a single mannequin that helps a number of modes of translation between as much as 101 languages. SEAMLESSM4T can facilitate speech-to-speech translation (recognizing 101 languages and translating to 36 languages), speech-to-text translation (101 to 96 languages), text-to-speech translation (96 to 36 languages), text-to-text translation (96 languages), and automated speech recognition (96 languages).

For speech-On the spot speech-to-speech translation, SEAMLESSM4T interprets textual content with as much as 23% extra accuracy than present programs. The AI mannequin can filter out background noise and modify to speaker variation. Though additional optimization is required, SEAMLESSM4T might symbolize a step in the direction of enhancing communication throughout language limitations, the authors conclude.

Extra data: Marta Costa-jussà, Joint speech and textual content machine translation for as much as 100 languages, Nature (2025). DOI: 10.1038/s41586-024-08359-z. www.nature.com/articles/s41586-024-08359-z

Journal data: Nature Offered by Nature Publishing Group Quotation: AI Babel Fish turns into actuality, permitting direct speech-to-speech translations (2025, January 15) retrieved 16 January 2025 from https://techxplore.com/information/2025-01-ai-babel-fish-reality-speech.html This doc is topic to copyright. Other than any honest dealing for the aim of personal examine or analysis, no half could also be reproduced with out the written permission. The content material is supplied for data functions solely.

Discover additional

Analysis might deliver automated speech recognition to 2,000 languages 7 shares