According to Dylan Patel, principal analyst at semiconductor analysis firm SemiAnalysis , ChatGPT’s operating costs are so high because of expensive infrastructure. ChatGPT requires massive computing power to calculate responses based on user prompts. Patel bases his cost estimates on the expensive servers needed to run the chatbot.
In an interview with BI , the expert suggested that the current cost is even higher because his initial estimate was based on OpenAI's GPT-3 model. The latest model, GPT-4, will be more expensive.
While training ChatGPT’s large language model (LLM) costs tens of millions of dollars, the operating or inference costs are much higher. For years, companies using OpenAI’s language models have had to pay large sums. For example, Nick Walton, CEO of startup Latitude, revealed that the cost of running the model with Amazon Web Services servers was about $200,000 per month in 2021. So he switched to another language software provider to cut the cost in half.
To reduce the costs required to run generative AI models, Microsoft is developing an AI chip called Athena. According to The Information , the project began in 2019, a few years after the company signed a $1 billion deal with OpenAI to have the startup use only Azure cloud servers.
Microsoft is pushing for two things: It realizes it’s lagging behind Google and Amazon in the chip space; and it’s looking for a cheaper alternative to Nvidia’s chips. More than 300 Microsoft employees are working on the chip, and the chip could be available internally as early as next year.
(According to BI)
Source
Comment (0)