G42, the UAE-based technology holding company, has announced a cooperation with US-based Cerebras Systems, as well as a roadmap that each of the companies claims would drastically cut the time required to train artificial intelligence (AI) large language models (LLM). Condor Galaxy, a global cluster of nine linked cloud-based AI supercomputers with a combined capacity of 36 exaFLOPs, is currently under construction.
G42, located in Abu Dhabi, owns Condor Galaxy. Condor Galaxy is managed and operated by Cerebras Systems, which is headquartered in Sunnyvale, California. Condor Galaxy 1 (CG-1) is the first out of 9 AI supercomputers that have been installed. It has 54 million cores and 4 exaFLOPs of performance, already resulting in one of the world's largest AI supercomputers.
"G42 and Cerebras had already been discussing potential collaboration since 2021," according to Feldman. "Then earlier this year, we saw the rise of large language models. Our technology had matured, and there was interest in the UAE to do big things in AI."
CG-1 has already been used to train Jais, the world's most sophisticated large language model (LLM) for the Arabic language, lending the real proof to the concept that their technology will shorten training time. Much of the work on Jais had been done before G42 and Cerebras joined together. What was lacking was time on a supercomputer to train the model, which is where G42 and Cerebras Systems came in.
"It was the right time and the right people," Andrew Feldman explained. "So, we began a big proof of concept and things went really well. We began flying out to visit them and got to know them – not just in a business context, but also in a personal context. And their team flew out here and got to know us a little bit better. By May, we agreed to build enough compute to change the worldwide inventory. Not only did we change the inventory of computing power, but we also added new large language models to the mix."
It is going to bring the benefits of artificial intelligence to the Arab-speaking people. "One obvious use case for the new language model will be in Arabic chatbots," says Andrew Jackson, CEO of Inception, a subsidiary of G42 and the company behind Jais' debut. "As the model learns, it can be used in government to answer a whole set of questions without requiring you to wait for a person to respond. It will also be used for scientific projects and in other sectors, including healthcare and financial services. And we expect a range of other applications – including many that we haven’t even thought of."
Cerebras created a pair of technologies aimed at speeding up the data-intensive training of large language models. The first was Wafer-Scale Cluster, the world's largest and most powerful AI processor, a novel design that links up to 192 CS-2 units and enables them to work as a single logical accelerator. The Condor Galaxy AI supercomputer is powered by Cerebras' Wafer-Scale Engine, a CPU designed expressly to speed deep learning.
Weight streaming is the second technology, which uses the hardware's computation and memory features to distribute workload by streaming a model one layer at a moment in a completely data-parallel approach. "The partnership with G42 enables us to combine expertise," said Andrew Feldman.
Companies expect to have the next 2 supercomputers, CG-2 and CG-3, operational in the United States by early 2024. Then, G42 and Cerebras plan to launch the remaining 6 Condor Galaxy supercomputers later in 2024, with the goal of aiding clients not just in the Middle East, but also anywhere in the globe.
The G42-Cerebras Systems cooperation is part of the UAE's more extensive national AI program, which seeks not only to enhance the usage of AI in the country along with to create the expertise and technology necessary to accelerate the country to the forefront of the AI industry.