AI companies are looking for big profits from 'tiny language models'

Apple, Microsoft, Meta, and Google have all recently released new AI models with fewer “parameters” but still powerful capabilities, a move by tech groups to encourage financially constrained businesses to still use AI.

Illustration: FT

Generally, the higher the number of parameters, the better the performance of AI software and the more complex and nuanced its tasks. OpenAI's latest GPT-4o model and Google's Gemini 1.5 Pro, both announced this week, are estimated to have more than 1 trillion parameters. Meanwhile, Meta is training a 400 billion-parameter version of its open-source Llama model.

Concerns about data and copyright liability have also led Big Tech companies like Meta and Google to release small language models with just a few billion parameters that are cheaper, energy-efficient, customizable, require less energy to train and run, and can also prevent sensitive data from being captured.

“By getting that high quality at a lower cost, you actually get more applications for customers to access,” said Eric Boyd, corporate vice president of Microsoft’s Azure AI Platform, which sells AI models to businesses.

Google, Meta, Microsoft, and French startup Mistral have also released small language models, but still demonstrate progress and can better focus on specific tasks.

Nick Clegg, Meta's president of global affairs, said Llama 3's new 8-billion-parameter model is comparable to GPT-4. Microsoft said its small Phi-3 model, with 7 billion parameters, outperforms GPT-3.5, the previous version of OpenAI's model.

Microchips can also process tasks locally on the device rather than sending information to the cloud, which could appeal to privacy-conscious customers who want to ensure information stays within the network.

Charlotte Marshall, a partner at law firm Addleshaw Goddard, said that “one of the challenges I think many of our clients have faced” when adopting generative AI products is complying with legal requirements around data processing and transmission. She said smaller models offer “an opportunity for businesses to overcome” legal and cost concerns.

Smaller models also allow AI features to run on devices like mobile phones. Google’s “Gemini Nano” model is embedded inside the latest Pixel phones and Samsung’s latest S24 smartphone.

Apple has also revealed that it is developing AI models to run on its best-selling iPhone. Last month, the Silicon Valley giant released OpenELM, a small model designed to perform text-based tasks.

Microsoft's Boyd said the smaller models will lead to “interesting applications, all the way down to phones and laptops.”

OpenAI director Sam Altman said in November that it was also offering AI models of different sizes to customers that “serve distinct purposes.” “There are some things that smaller models will do really well. I’m excited about that,” he said.

However, Altman added that OpenAI will still focus on building larger AI models with scalability, including the ability to reason, plan and execute tasks and ultimately achieve human-level intelligence.

Hoang Hai (according to FT)

Source: https://www.congluan.vn/cac-cong-ty-ai-dang-tim-kiem-loi-nhuan-lon-tu-cac-mo-hinh-ngon-ngu-nho-post296219.html