In the southern part of Austin, Texas, engineers at Advanced Micro Devices (AMD) developed the MI300, an artificial intelligence chip that debuted a year ago. It’s projected to generate over $5 billion in sales in its first year. Meanwhile, in a high-rise in northern Austin, Amazon’s designers crafted a new and faster version of its A.I. chip, Trainium, which has been tested in everything from compact circuit boards to large, complex computers.

The Furious Contest to Unseat Nvidia as King of A.I. Chips

These efforts, taking place in Texas’ capital, highlight a significant shift in the rapidly growing A.I. chip market—arguably one of the most sought-after technologies today. For years, Nvidia dominated this sector, using its A.I. chips to become a $3 trillion giant. Competitors struggled to match Nvidia’s chips, which offered unparalleled computing power for A.I. tasks, making little headway.https://wordpress.com/

Now, chips from AMD and Amazon, as well as customer feedback, suggest that credible alternatives to Nvidia are beginning to emerge. According to Daniel Newman, an analyst at Futurum Group, Nvidia’s competitors are now delivering significantly faster speeds at much lower costs. “This is something we’ve always known was possible, and now we’re starting to see it come to life,” he explained.

The shift is being fueled by a wide range of tech companies, from giants like Amazon and AMD to smaller startups, which are customizing their chips for a critical phase of A.I. development—“inferencing.” This stage, which follows the training of A.I. models, involves executing tasks like generating responses from A.I. chatbots. Cristiano Amon, CEO of Qualcomm, pointed out that “The real commercial value lies in inference, and that’s now scaling up.”

Another strategy Nvidia’s competitors are adopting is to offer complete computing solutions, not just chips. By providing full systems, they aim to ensure customers can extract maximum performance from their A.I. chips.

The rising competition was made evident on Tuesday when Amazon unveiled its new Trainium 2 A.I. chips, alongside testimonials from prospective users like Apple. Amazon also introduced new computers powered by these chips, available in configurations of 16 or 64 units, designed to supercharge inferencing tasks with ultra-fast networking.

Amazon is even building a massive A.I. infrastructure for Anthropic, a startup it’s invested in. This “cluster” will house hundreds of thousands of Trainium chips, offering five times the power of any systems Anthropic has used before. “This will give customers more intelligence at a lower cost and faster speeds,” said Tom Brown, co-founder of Anthropic.

According to Omdia, a market research firm, spending on non-Nvidia data center computers is expected to grow by 49% this year, reaching $126 billion.

Despite this surge in competition, Nvidia is not in danger of losing its dominance just yet. Nvidia CEO Jensen Huang has pointed to the company’s strength in A.I. software and inferencing capabilities, noting that even when competitors offer chips for free, Nvidia’s total cost of ownership remains superior. “Our total cost of ownership is so good that even when the competitor’s chips are free, it’s not cheap enough,” Huang remarked during a speech at Stanford.

The changing A.I. chip landscape is also fueled by well-financed startups like SambaNova Systems, Groq, and Cerebras Systems, which have recently claimed significant speed advantages in inferencing with lower power consumption and pricing. Nvidia’s chips can cost as much as $15,000, and its new Blackwell chips are expected to be even pricier, pushing some customers to seek alternatives. For instance, Dan Stanzione, executive director of the Texas Advanced Computing Center, mentioned that while they plan to buy a Blackwell-based supercomputer next year, they’re likely to turn to SambaNova’s chips for inferencing tasks due to their better price-performance ratio.

AMD is gearing up to take on Nvidia’s Blackwell chips with its own A.I. chips next year. At AMD’s Austin labs, executives have emphasized inferencing performance as a key selling point. Meta, for example, has used Nvidia chips to train an A.I. model but relies on AMD’s MI300 chips for answering user queries.

Amazon, Google, Microsoft, and Meta are also developing their own A.I. chips to optimize specific tasks and reduce costs, while still relying on Nvidia’s chips for large computing clusters. Google, for instance, is about to roll out services based on its sixth-generation Trillium chips, which offer almost five times the speed of their predecessor.

Amazon, often seen as trailing in A.I., is now highly committed to catching up. The company has allocated $75 billion this year for A.I. chips and computing hardware. Its Austin-based Annapurna Labs, which it acquired in 2015, previously worked on networking chips and general-purpose processors. Although earlier A.I. chips like the first Trainium didn’t make a significant impact, Amazon is optimistic about the new Trainium 2 chips, which are four times faster. Additionally, Amazon plans to introduce Trainium 3, an even more powerful chip.

Eiso Kant, CTO of A.I. startup Poolside, estimates that Trainium 2 will provide a 40% improvement in price-performance compared to Nvidia hardware. Amazon also plans to deploy Trainium-powered services in data centers globally, which will benefit inferencing tasks.

As Kant summed up, “The reality is, in my business, I don’t care what silicon is underneath. I care about getting the best price-performance and delivering it to the end user.”