devarena logo
Reading Time: 2 minutes


Meta is making its model, called Open Pretrained Transformer (OPT), available for non-commercial use.

Meta (formerly known as Facebook) is granting AI researchers access to a language model. The tech giant said its model was the first language model that had 175 billion words. Large language models are natural language processing systems that have been trained on a lot of text.

Large language models—powerful programs that can generate paragraphs of text and mimic human conversation—have become one of the hottest trends in AI in the last couple of years. But they have deep flaws such as parroting misinformation, prejudice, and toxic language.

Meta is making its model, called Open Pretrained Transformer (OPT), available for non-commercial use. It is also releasing its code and a logbook that documents the training process. The logbook contains daily updates from members of the team about the training data: how it was added to the model and what worked when and what didn’t. In more than 100 pages of notes, the researchers logged every bug, crash, and reboot in a three-month training process that ran nonstop from October 2021 to January 2022.

 

Weighing the risks

Margaret Mitchell, one of the AI ​​ethics researchers Google forced out in 2020, who is now at Hugging Face, sees the release of OPT as a positive move. But she thinks there are limits to transparency. Has the language model been tested with sufficient rigor? Do the foreseeable benefits outweigh the foreseeable harms—such as the generation of misinformation, or racist and misogynistic language?

Meta AI audited OPT to remove some harmful behaviors, but the point is to release a model that researchers can learn from, warts and all.

 

More Trending Stories 

The post Meta’s Strategy for Growth is ‘Criticism’! It is Giving Away its New Language AI for Free appeared first on .



Source link

Spread the Word!