Meta AI’s Latest Announcement

There are some rather interesting developments in the world of languages and technology. In the past, many technical systems, online platforms, and programs were unavailable in less spoken and less translated languages, effectively preventing millions or even billions of people worldwide from accessing technology and therefore creating a digital divide. However, Meta AI, Facebook’s artificial intelligence laboratory, just announced a breakthrough in AI translation with their open-source No Language Left Behind (NLLB) language model—a model with many components allegedly capable of directly translating between 200 languages.

Meta AI claims their model can handle languages with fewer available textual and translated resources, such as Urdu. Meta AI’s “aim is to set a new standard of inclusion—where someday everyone can have access to virtual-world content, devices and experiences, with the ability to communicate with anyone, in any language in the metaverse. And over time, bring people together on a global scale.”
   Quote from Meta AI’s No Language Left Behind website

Meta AI’s NLLB is actually not a single model but a mixture of different models and other components!

To be able to translate between so many languages, Meta AI uses large data sets by pairing sentences from readily available language data collections. However, the developers also thought of solutions for languages with fewer available datasets. Languages with many resources due to available translations (corpus linguistics!) are, for example, English and Spanish or French and English, while bilingual text versions in Icelandic and Luganda are probably harder to find. Consequently, for those languages with little resources, they used human translations as seeding data for their model configuration.

Another part of the NLLB language model mix is self-supervised machine learning and additional human evaluation support (FLORES). FLORES’ workflows allow for feedback from translators and reviewers. NLLB also includes a sparse mixture-of-experts model, a router with shared and specialized capacities predicting what to choose or fall back on when many options are available. It is simply a regulation system that can make many decisions by itself but can also ignore irrelevant information within datasets. So, despite its ability to train on complex data samples, the system will still generalize well when new data becomes available. NLLB’s mix of models and datasets also includes lists for identifying toxic content that may have been accidentally introduced during the translation process.

Phew! This was a lot of information in a nutshell! For more information, please visit Towards AI or Meta AI.