California-based Cerebras Systems has unveiled the Wafer Scale Engine (WSE-3), a remarkable achievement in chip design that boasts an astonishing four trillion transistors. This latest artificial intelligence (AI) chip from Cerebras delivers twice the performance of its predecessor, the Cerebras WSE-2, which previously held the record for the fastest chip.
Accelerating AI Model Training with Unparalleled Performance
The WSE-3 is designed to revolutionize AI model training by providing unprecedented computational power. Systems built using the WSE-3 will be capable of fine-tuning models with 70 billion parameters in just a single day, an unprecedented feat that showcases the chip’s immense potential in the rapidly evolving field of artificial intelligence.
Cerebras CS-3: An AI Supercomputer Powered by the WSE-3
Cerebras WSE-3 AI chip forms the core of the company’s AI supercomputer, the CS-3. Utilizing the 5nm architecture, the CS-3 delivers an astonishing 900,000 cores optimized for AI data processing. It features a massive 44GB on-chip SRAM and can store an incredible 24 trillion parameters in a single logical memory space without partitioning or refactoring, significantly simplifying the training workflow and improving developer productivity.
Scalable Memory and Unparalleled Training Capabilities
The CS-3’s external memory can be scaled from 1.5 terabytes to a staggering 1.2 petabytes, enabling the training of models ten times larger than GPT-4 or Gemini. Cerebras claims that training a one trillion parameter model on the CS-3 is as straightforward as training a one billion parameter model on GPU chips. In a four-system configuration, the CS-3 can fine-tune AI models consisting of 70 billion daily parameters, while a 2048-system configuration can train the 70 billion parameter Llama model from scratch in just a single day.
Table of Contents
Cerebras WSE-3 AI chip: Driving Innovation and Collaboration
Energy-Efficient and Code-Efficient AI Acceleration
At a time when the power consumption of GPUs doubles with every new generation, Cerebras has ensured that its latest WSE-3 chips deliver twice the performance without any increase in size or power consumption. Additionally, the AI-specific chip requires 97 percent less code to train large language models (LLMs) compared to GPUs, enabling more efficient and streamlined development.
Collaborative Deployments and Research Partnerships
Cerebras plans to deploy the WSE-3 at the facilities of its long-time collaborators, the Argonne National Laboratory and the Mayo Clinic, further advancing research capabilities at these prestigious institutions. Furthermore, in partnership with G42, Cerebras is building the Condor Galaxy-3 (CG-3), one of the largest AI supercomputers in the world, consisting of 64 CS-3 units and delivering an unprecedented eight exaFLOPS of AI computing power.
With the introduction of the Cerebras WSE-3 AI chip and the CS-3 AI supercomputer, Cerebras Systems is pushing the boundaries of what is possible in the realm of artificial intelligence. This groundbreaking technology paves the way for unprecedented advancements in AI model training, enabling researchers and developers to tackle increasingly complex challenges and unlock new frontiers in machine learning and artificial intelligence.
