Title 2024-03-28 - Cerebras for AI at Scale at MILA.pdf ✅

© 2023 Cerebras Systems Inc. All Rights Reserved Hardware, software, scaling laws and sparsity Natalia Vassilieva, VP and Field CTO, ML Joel Hestness, Principle Research Scientist Cerebras Systems
© 2023 Cerebras Systems Inc. All Rights Reserved • Cerebras overview, hardware and software • Scaling laws to train SoTA models • Cerebras sorcery: sparsity 2 Agenda
© 2023 Cerebras Systems Inc. All Rights Reserved Build the fastest AI accelerators Connect into easy to use and quick to deploy AI supercomputers Train models for the open source community and enterprise customers Provide extensive in-house ML expertise Cerebras Systems
© 2023 Cerebras Systems Inc. All Rights Reserved Cerebras Wafer-Scale Engine 2 (WSE-2) 850,000 cores optimized for sparse linear algebra 46,225 mm2 silicon 2.6 trillion transistors 40 gigabytes of on-chip memory 20 PByte/s memory bandwidth 220 Pbit/s fabric bandwidth 7nm process technology A cluster of flexible, sparse linear algebra compute resources, all on a single chip