A chip that is the largest in the world has taken the chip community by storm. Do you know the story behind the team?
By: Yaya Jun
Images: Network
Recently, the birth of a giant chip has ignited the chip community.
It has an area of 42,225 square millimeters, featuring 1.2 trillion transistors, 400,000 cores, 18GB of on-chip memory, a memory bandwidth of 19PByte/s, and a fabric bandwidth of 100Pbit/s. This is 56.7 times larger than the largest NVIDIA GPU currently available.
Image source: Cerebras Systems official website
This chip—the Cerebras Wafer Scale Engine (Cerebras WSE)—was launched by the artificial intelligence startup Cerebras Systems. One reason it attracts attention is that while the outside world continues to pursue chip miniaturization and low power consumption, this chip goes against the trend.
Image source: Cerebras Systems official website
According to an article from Venturebeat, large chips can process information faster and produce results in a shorter time. This reduces insight time or “training time,” allowing researchers to test more ideas, use more data, and tackle new problems. Google, Facebook, OpenAI, Tencent, Baidu, and many others believe that the fundamental limitation of artificial intelligence today is that training models take a long time. Therefore, shortening training time eliminates a major bottleneck for progress in the entire industry.
Interestingly, this company is extremely low-key. Even its official website has very little information, and on LinkedIn, it simply states, “Cerebras is a low-key startup. We are a group of successful entrepreneurs who dare to tackle problems that others cannot solve. Our team values unity, enthusiasm, practical problem-solving abilities, and of course, a sense of humor is essential. We look forward to more talented individuals joining us.”
On the news section of Cerebras Systems’ official website, there are only a few articles. The latest three are about the release of this giant chip.
In February of this year, CB Insights released a list of the 100 most promising AI companies in the AI industry chain, which includes Cerebras Systems.
In an interview last May with founder Andrew, he compared Cerebras to a cheetah, a specialist in the field of artificial intelligence, while Intel and NVIDIA are generalists that rely on their architecture to solve all computing problems. He stated that major innovations in artificial intelligence cannot come from chips but from a system, and what Cerebras is creating is a system.
This limited introduction adds to the mystery of the company. Today, TechSugar’s editor will dive into this company, starting with the technical leadership team.
The Mysterious and Low-Key Team of Experts
According to information from the official website, Cerebras Systems is composed of a team of top computer architects, computer scientists, and deep learning researchers. The goal is to build a new computer system that can accelerate artificial intelligence beyond the current technological scale. According to the official website, Cerebras Systems currently has 150 employees.
Andrew Feldman
Co-founder and CEO of Cerebras Systems
He holds an MBA from Stanford. Before founding Cerebras Systems, Andrew Feldman was the co-founder and CEO of SeaMicro, a pioneer in low-power server technology, which has been acquired by AMD.
Before founding SeaMicro, Andrew Feldman served as the Vice President of Marketing and Product Management at Force10 (acquired by Dell for $800 million) and as the Vice President of Corporate Marketing and Corporate Development at Riverstone Networks, from its inception until its IPO.
Gary Lauterbach
Co-founder and CTO
According to the executive information section of the official website, Gary Lauterbach is the co-founder and Chief Technology Officer of Cerebras Systems. Gary is recognized as one of the top computer architects in the industry.
Before joining Cerebras, he was the co-founder and CTO of SeaMicro. During this time, Gary was the principal investigator for a $9.3 million energy-efficient computing funding project from the U.S. Department of Energy. After the acquisition of SeaMicro, Gary served as a Corporate Fellow and CTO of the server/server CPU department.
In the early stages of his career, he served as an engineer at Sun Microsystems, where he was the chief microprocessor architect for the UltraSPARC III and UltraSPARC IV microprocessors. During his time at Sun Laboratories, he was the chief architect of the DARPA HPCS Petascale computing project. Gary holds over 50 patents.
Dhiraj Mallick
Vice President of Engineering and Business Development
Dhiraj is the Vice President of Engineering and Business Development at Cerebras Systems. Previously, Dhiraj had over 20 years of executive leadership experience at Intel, AMD, SeaMicro, and other startups, guiding large, high-performance engineering teams.
Before Cerebras, Dhiraj was the Chief Technology Officer and Vice President of Intel’s $20 billion data center business.
Prior to that, he was one of the members of the executive leadership team at SeaMicro. After SeaMicro was acquired, he continued at AMD as Vice President and General Manager of the Server Solutions Division. Additionally, Dhiraj is an advisor to several venture-backed companies and holds a master’s degree in electrical engineering from Stanford University.
Sean Lie
Co-founder and Chief Hardware Architect
Sean is the Chief Hardware Architect and co-founder of Cerebras Systems. Before Cerebras, Sean was the chief hardware architect for SeaMicro’s input/output (I/O) virtualization Fabric ASIC. After SeaMicro was acquired, Sean became a researcher and chief architect for data centers at AMD.
In the early stages of his career, he worked for five years in AMD’s advanced architecture team. He holds a bachelor’s degree in electrical engineering and a master’s degree in computer science and engineering from MIT, and has 16 patents in computer architecture.
Michael James
Co-founder and Senior Technical Designer
Before Cerebras Systems, Michael was the chief software architect responsible for distributed systems software at SeaMicro. After SeaMicro was acquired, Michael became the chief software architect at AMD.
Throughout his career, he has designed operating systems, compilers, real-time feedback control systems, and signal processing and recognition systems. Michael holds degrees in molecular neurobiology, mathematics, and computer science from the University of California, Berkeley.
J.P. Fricker
Co-founder and Chief System Architect
J.P. is the Chief System Architect at Cerebras Systems. Before co-founding Cerebras, J.P. was the senior hardware architect for DSSD (later acquired by EMC) for an innovative rack-scale flash system.
Prior to DSSD, J.P. was the chief system architect at SeaMicro, where he designed three generations of Fabric-based computer systems.
In the early stages of his career, J.P. served as the Director of Hardware Engineering at Alcatel-Lucent and as the Director of Hardware Engineering at Riverstone Networks. He holds a master’s degree in electrical engineering from the Swiss Federal Institute of Technology in Lausanne and has 18 patents.
The Glory of SeaMicro
In the resumes of these technical leaders, you will notice one company frequently mentioned: SeaMicro. Clearly, this group of people almost all come from that team. SeaMicro is also a company with a legendary status.
Founded in July 2007, SeaMicro was established by the aforementioned Andrew Feldman, Gary Lauterbach, and another individual named Anil Rao. The company secured Series A and Series B funding in 2007 and 2009, respectively.
In June 2010, SeaMicro released a heavyweight product, described by foreign media as an “atomic bomb” level product. A server equipped with 512 Intel Atom chips, this product could achieve supercomputer performance while consuming 75% less power and space than existing servers.
SeaMicro applied the concept of virtualization within its servers. The custom chips designed by founder Andrew Feldman could handle tasks beyond the capabilities of Intel microprocessors and their chipsets. The custom chips virtualized all other components to find resources when needed.
Its first-generation product SM10000 was launched in July 2010, priced at $139,000 each. In July 2011, the company released its third-generation server—the SM10000-64 HD. According to reports, the third-generation server featured 384 1.66GHz dual-core Atom N570 processors, 1.536TB DDR3 DRAM, up to 64 SATA SSDs or HDDs, and 16 10GbE or 64 1GbE uplink connections. The starting price was $237,000.
In February 2012, AMD announced the acquisition of SeaMicro for $334 million. A month before the acquisition, SeaMicro was launching a new server design, and Intel executives stated that this could capture 10% of the server market. (The new product SM10000-XE was built around Intel’s Xeon server processors and Samsung’s memory chips.)
Before the acquisition, SeaMicro had over 50 customers, including eHarmony, Skype, and Mozilla.
Image: (From left to right) AMD CEO Rory Read, SeaMicro CEO Andrew Feldman, and AMD General Manager Lisa Su.
After the acquisition by AMD, Andrew Feldman, one of the founders of SeaMicro, was tasked with overseeing AMD’s server chip business.
In 2014, AMD underwent restructuring, during which the then 44-year-old Lisa Su was promoted to AMD’s Chief Operating Officer (COO), while Andrew Feldman chose to leave, stating he needed to “take a break.” It seems that he was gathering a team to plan a new startup—Cerebras Systems.
The fate of SeaMicro did not mirror that of its founder; on April 16, 2015, AMD announced it was abandoning the SeaMicro business to focus on high-performance server CPUs.
Artificial Intelligence Requires Extreme ‘Brutality’
In 2016, Andrew Feldman led a group of former colleagues to establish Cerebras Systems. According to PitchBook data, Cerebras Systems has completed three rounds of financing, starting with a $27 million Series A in May 2016, at which point it was valued at $64.5 million. In January 2017, after a $25 million Series B funding, its valuation quickly rose to $245 million; six months later, Cerebras raised another $60 million, reaching a valuation of $860 million.
Cerebras Systems financing situation Image source: Craft
Looking at it now, this artificial intelligence company seems to have a bright future. Reviewing the founder’s experience, the concept of a giant chip is not unfamiliar; during SeaMicro, the founders were playing with the “big shot” technique.
Fieldman stated: “The Cerebras WSE is designed specifically for artificial intelligence, incorporating many foundational innovations that address decades-long technological challenges limiting chip size—such as chip yield, power delivery, and packaging—advancing the development of cutting-edge technology. Every architectural decision is made to optimize AI workloads. The result is that the Cerebras WSE provides hundreds or thousands of times the performance of existing solutions for the same workload, with minimal power consumption and space requirements.”
Creating this giant chip in just a few years is indeed astonishing.
In the white paper introducing the Cerebras WSE, there is a statement: “By accelerating artificial intelligence computation, the WSE clears the biggest obstacle to AI advancement—time. It reduces training time from months to minutes, from weeks to a few hours. It allows deep learning practitioners to validate their hypotheses faster without worrying about institutional constraints that prevent testing or pose too much risk. The WSE lowers the cost of curiosity and accelerates the arrival of new ideas and technologies in artificial intelligence.”
END
