Why Meta AI’s LLM Galactica Got Canceled #llm #metaai #machinelearning

Galactica was a family of LLMs developed by Meta AI for scientific research. It was released as a public demo in November of 2022 after being trained on over 48 million scientific documents, and it came in five model sizes. Galactica outperformed larger models in areas like mathematics, and also set a new state-of-the-art on PubMedQA and MedMCQA Dev. According to Ross Taylor who worked at Meta AI on Galactica, all of this was done by an 8-person team.

When Meta AI released a public demo of Galactica though, things did not go well, with many criticising the LLM's hallucinations. Michael Black called it potentially distorting and dangerous for science. Gary Marcus said LLMs inherently hallucinate and that Galactica had weaponized the ability to spread fake science. Grady Booch called it statistical nonsense at scale.

The Galactica team ended up taking the demo down three days after launch due to the backlash and then two weeks later, OpenAI released ChatGPT and everyone promptly got used to LLMs hallucinating.