NVIDIA’s $20B chip could make ChatGPT look slow

Chip giant NVIDIA is preparing to unveil a powerful new artificial intelligence processor designed to speed up how chatbots and other AI tools generate responses, potentially making today’s systems like ChatGPT appear sluggish by comparison.

The new platform, expected to debut at NVIDIA’s annual GTC developer conference, is optimized for AI inference, the stage when trained models produce answers to user prompts. Unlike traditional GPUs built to handle both training and inference, the upcoming processor focuses specifically on delivering responses faster and more efficiently.

The product, if launched, will mark the first tangible result of December’s deal that brought Groq’s founders into the fold, whose company specializes in high-speed AI processing hardware.

Late last year, NVIDIA reportedly spent about $20 billion to license technology from the chip startup Groq and recruit key personnel, including its CEO. Around the same time, NVIDIA CEO Jensen Huang told employees, “We plan to integrate Groq’s low-latency processors into the NVIDIA AI factory architecture, extending the platform to serve an even broader range of AI inference and real-time workloads.”

Now, the new inference chip is expected to handle complex AI queries at high speed, with OpenAI and other leading clients likely to adopt it, according to The Wall Street Journal. Its report also showed that the new chip may handle close to 10% of OpenAI’s inference workload.

The Groq-style chip will use SRAM, sources say

During a recent earnings call, NVIDIA CEO hinted that several new products will be unveiled at the upcoming GTC event, often described as the “Super Bowl of AI.” He had remarked, “I’ve got some great ideas that I’d like to share with you at GTC.” 

Most analysts agree the Groq-style chip could be part of the lineup. They also stated that its design could shed light on how NVIDIA aims to address memory constraints in inference computing. Such platforms typically run on high-bandwidth memory (HBM). However, HBM has been difficult to source lately.

Insiders have claimed the firm plans to use SRAM in the chip rather than the dynamic RAM associated with HBM. Ideally, SRAM is more accessible and can improve the performance of AI reasoning workloads.

If the chip is unveiled, it could be a great step forward for the chip company and AI-trained models. However, speaking on its possible launch, Sid Sheth, founder and CEO of d-Matrix, cast a shadow on its development. He noted that while NVIDIA remains the clear leader in AI training, inference represents a very different landscape. He shared: “Developers can turn to competitors other than NVIDIA because running finished AI models doesn’t require the same kind of programming as training them.” 

Nevertheless, other tech giants are also advancing inference computing. Meta this week unveiled four processors tailored for inference, prompting a Silicon Valley investor to say the industry may be entering a non–“NVIDIA-dominant” phase.

However, more recently, June Paik, chief executive of FuriosaAI, a NVIDIA rival, commenting on the benefit of easily deployable inference computing, cautioned that most data centers can’t accommodate the latest liquid-cooled GPUs.

Nonetheless, despite his worries, the Bank of America analysts expect inference workloads to represent 75% of AI data center spending by 2030, when the market reaches about $1.2 trillion, up from about 50% last year. Ben Bajarin, a tech analyst at Creative Strategies, also asserted that data centers of the future won’t conform to a one-size-fits-all model, anticipating that companies will take different approaches to chip and facility development.

NVIDIA is expected to release the Vera Rubin chips later in 2026

NVIDIA has also recently rolled out its next-gen AI chips, Vera Rubin AI chips, anticipating that the rise of reasoning AI platforms such as DeepSeek will fuel even greater computing demand. It claimed the chips would help train larger AI models and provide more sophisticated outputs to a broader user base. 

According to Huang, Rubin will also hit the market in the second half of 2026, with a high-end “ultra” version coming in 2027.

He also explained that a single Rubin system would combine 576 individual GPUs into a single chip. Currently, NVIDIA’s Blackwell chip clusters 72 GPUs in its NVL72 system, meaning Rubin will feature more advanced memory.