Groq's specialized artificial intelligence (AI) chip, LPU, is becoming an overnight sensation on social media, as its publicly available benchmark tests are circulating on social media platform X.

Groq claims to offer ' the world's fastest natural language processing models,' and extensive independent testing suggests that claim may be accurate.

735783 1szzxeq groq.jpg
Groq's dedicated LPU AI chip delivers superior performance for large language models in experiments.

Groq manufactures specialized AI chips called LPUs (Language Processing Units), which are said to be faster than Nvidia's GPUs. Nvidia's GPUs are often considered the industry standard for running AI models, but initial results suggest that LPUs will surpass them, at least in tasks requiring high speed.

Essentially, Groq's LPU chip is a 'computing engine' that enables chatbots like ChatGPT and Gemini to operate with extremely fast reception and response speeds. In tests conducted, chatbots using the LPU chip outperformed eight chatbots using other AI chips on several key performance metrics, including latency versus throughput and total response time. The superior performance of the LPU was most evident when tested with Meta's Llama 2-70b model.

According to independent tests by Artificial Analysis, chatbots using LPU chips can achieve a throughput of 247 tokens per second. For comparison, OpenAI's ChatGPT currently only achieves a maximum throughput of 18 tokens per second. This means ChatGPT could be up to 13 times faster using chips developed by Groq, although this depends on many other related factors.

According to Artificial Analysis, this level of performance could unlock new possibilities for large language models (LLMs) across various fields. Chatbots using Groq's LPU chip can generate hundreds of words in an instant, allowing users to perform tasks in real time. In one experiment, Groq founder and CEO Jonathon Ross guided a CNN anchor in a live spoken conversation with an AI chatbot on the other side of the globe.

AI chatbots like ChatGPT, Gemini, and Llama could be significantly more useful if they responded faster. One of the significant limitations currently is that these chatbots cannot keep up with human speech in real time. The large delay makes the conversation sound robotic and unsuitable for many fields.

Despite the expectations surrounding Groq's LPU chip, the immediate issue is evaluating its practical applicability compared to similar products. Unlike Nvidia's GPUs or Google's TPUs, Groq's LPU is a specialized AI chip, designed to optimize for specific tasks, rather than a 'general-purpose' AI chip.

Furthermore, this is also a time when major AI developers are looking to develop their own chips to avoid dependence on Nvidia's products. OpenAI is reportedly seeking trillions of dollars in funding to develop its own chips. OpenAI CEO Sam Altman is even considering rebuilding the entire AI chip architecture from scratch.

(according to Cryptoslate)