Ai's exponential leap: moore's law is officially obsolete
The familiar rules of scaling—doubling resources leads to a predictable improvement—are obsolete when applied to artificial intelligence, according to Mustafa Suleyman, now head of AI at Microsoft. His stark assessment, published in MIT Technology Review, highlights a paradigm shift driven by a computational explosion that renders decades of technological expectation irrelevant.
The trillion-fold increase: from calculators to a collective mind
Suleyman’s perspective stems from firsthand experience. Since starting in AI in 2010, he’s witnessed a transformation bordering on the unbelievable. Early AI systems operated on roughly 1014 FLOPS (floating-point operations per second). Today's leading models boast over 1026 FLOPS—an increase of a trillionfold. “This isn’t growth; it’s an explosion,” he observes, drawing a compelling analogy: “Training AI used to be like a room full of people with calculators. Now, it’s about getting all those calculators working continuously, as a single, giant mind.”
The convergence of breakthroughs: speed, bandwidth, and scale
This shift isn't simply about faster processors. It’s a confluence of three critical advancements. First, the raw processing power of individual chips has skyrocketed. Nvidia’s chips moved from 312 teraflops in 2020 to 2,250 teraflops today, while Microsoft’s Maia 200 chip offers a 30% performance-per-dollar advantage. Second, data delivery has become dramatically faster, thanks to High Bandwidth Memory (HBM). The latest iteration, HBM3, triples the bandwidth of its predecessor, ensuring GPUs are fed with a constant stream of information and eliminating bottlenecks. Finally, these computational units are being interconnected on an unprecedented scale. Technologies like NVLink and InfiniBand are linking hundreds of thousands of GPUs into supercomputers the size of industrial buildings, effectively creating a single, unified processing entity.
The results are staggering. A task that once took 167 minutes on eight GPUs in 2020 now completes in under four minutes. That's a 50x speedup—far exceeding the modest improvements predicted by Moore's Law.
Beyond prediction: a future of exponential growth and superintelligence
Current projections suggest this momentum is far from slowing. Major AI labs are reportedly expanding their computing capacity at a rate of nearly 4x annually, a pace that has already yielded a 5x annual increase since 2020. By 2027, global AI computing power is expected to reach the equivalent of 100 million H100 chips—a tenfold jump in just three years. Companies like Microsoft and Nvidia are at the forefront of this revolution, building infrastructure that was once relegated to science fiction: clusters of 100,000 GPUs, refrigerator-sized racks consuming 120 kilowatts, and data centers drawing power equivalent to the combined peak consumption of the UK, France, Germany, and Italy.
Suleyman’s ultimate goal, shared by many in the field, remains the pursuit of Artificial Superintelligence (ASI). But, as he cautions, we’ve only witnessed roughly 10-15% of the potential impact of these advancements. The energy demands are substantial – a single AI rack can consume the equivalent power of 100 homes—yet, the plummeting costs of renewable energy—solar costs have fallen nearly 100 times in 50 years, battery costs by 97% in three decades—offer a potential pathway toward sustainability. The computational boom is the defining technological narrative of our time, and it's only just beginning.