The AI ​​expert left OpenAI — the company he was one of the founders of — earlier this year to set up his own AI lab called Safe Superintelligence Inc.

“Pre-training as we know it will cease to exist,” Sutskever said at a conference on neural information processing.

The term "pre-training" refers to the early stage in AI model development, when a large language model learns patterns from large amounts of unlabeled data, often text from the Internet, books, and other sources.

Data resource exhaustion

Sutskever said that while he believes current data can still accelerate the development of AI, the industry is running out of new resources to train models.

This, he says, will eventually lead to a change in the way AI models are trained today.

8da2491217a6b424c37773100bec59ff.jpeg
AI models will figure out how to train themselves based on limited amounts of data. Photo: Yahoo Tech

This is also the situation that has happened and is happening with fossil resources, when oil fields are a limited resource, or like the Internet only contains a finite amount of human-created content.

“We have reached the peak of data and it will not be there in the future,” Sutskever said. “We have to work with the data that is available, with a source that is the Internet today.”

Sutskever predicts that next-generation models will be “truly agent-like.” “Agent” is a buzzword in the AI ​​field, generally understood as an autonomous AI system that performs tasks, makes decisions, and interacts with software independently.

In addition to being “agent-like,” he said future AI systems will be able to reason. Unlike current AI, which mostly recognizes patterns based on what the model has learned before, future AI systems will be able to solve problems step by step in a way that is closer to thinking. “The more reasoning, the less predictable the system becomes,” Sutskever said.

AI can create its own way of training itself

The expert also compared the development of AI systems to evolutionary biology, citing research showing the relationship between brain and body size in animals.

For example, while most mammals follow a certain proportional pattern, humans have markedly different brain and body proportions.

And at some point when evolution finds new rates for the brain growth of our ancestors, AI may also find new ways to scale up, surpassing the way models are trained today.

(According to TheVerge, Yahoo Tech)

Revealing the 'sky-high' salaries of xAI and OpenAI Elon Musk accused OpenAI of paying employees too much, calling it anti-competitive. However, the billionaire's xAI startup also pays much higher than average salaries.