With its 1.7 trillion parameters, GPT-4 is impressive. However, scientists want to develop their own trillion-parameter-sized digital brain that’s fed with scientific information only. To do this, they’ve kickstarted the Trillion Parameter Consortium (TPC).
The National Center for Supercomputing Applications (NCSA) announced that it would be joining the TPC as a founding member. This global gathering includes scientists from the world’s most prestigious research institutes, federal laboratories, academia, and industry.
The brightest brains want to get together to tackle the challenge of “building large-scale artificial intelligence (AI) systems and advancing trustworthy and reliable AI for scientific discovery.”
The open community of researchers aims to share knowledge and avoid duplication of effort in incubating, launching, and coordinating AI projects, maximizing their impact. They also foresee a global network of resources and expertise.
The name, Trillion Parameter Consortium, includes the ambition of building state-of-the-art large language models for science and engineering. The idea for collaboration goes back a few years, when exascale computing platforms started to be deployed in the US Department of Energy laboratories, such as Frontier, Aurora, and El Capitan. Training large language models (LLMs) requires a lot of machine time.
“It became clear that while the community could develop a number of smaller models independently and compete for cycles, a broader “AI for Science” community must work together if we are to create models that are at the scale of the largest private models,” the TPC website reads.
One of the most advanced private models, OpenAI’s GPT-4, already, according to some sources, has 1.7 trillion parameters, more than the scientists’ ambitious goal. However, they hope that their AI models will be trustworthy and reliable. Trillion parameter models, for them, represent “the frontier of large-scale AI.”
“At our laboratory and at a growing number of partner institutions around the world, teams are beginning to develop frontier AI models for scientific use and are preparing enormous collections of previously untapped scientific data for training,” explained Rick Stevens, Argonne associate laboratory director for computing, environment, and life sciences.
The NCSA is developing its own AI-focused advanced computing and data resource, called DeltaAI, that’s supposed to play an instrumental role in the efforts undertaken by the TPC.
“Set to come online in 2024, DeltaAI will triple NCSA’s AI-focused computing capacity and greatly expand the capacity available within the NSF-funded advanced computing ecosystem,” the press release reads.
To deploy DeltaAI, NCSA received a $10 million award from the National Science Foundation (NSF).
Other founding members are working on their own AI models. Argonne National Laboratory is creating an AI model called AuroraGPT, which could ultimately become a massive brain for scientific researchers after months of training. GPT stands for Generative Pre-trained Transformer.
“It combines all the text, codes, specific scientific results, and papers into the model that science can use to speed up research,” said Ogi Brkic, vice president and general manager for data center and HPC solutions.
Ultimately, the TPC collaboration aims to leverage global efforts. The work aims to identify and prepare high-quality training data, design and evaluate model architectures, and develop innovations in model evaluation strategies with respect to bias, trustworthiness, and goal alignment.
Your email address will not be published. Required fields are markedmarked