Worried about human extinction due to AI, 'godfather' Yoshua Bengio just did something

AI 'godfather' Yoshua Bengio has just founded a non-profit organization with the goal of creating safe artificial intelligence systems, instead of showing dangerous personalities to humans.

According to Yoshua Bengio, there is a fierce race between the world's leading labs to create smarter AI without paying enough attention and investment to safety.

He recently founded the non-profit LawZero, which focuses on building safer systems and “insulating” research from commercial pressures.

To date, LawZero has raised nearly $30 million from names such as Skype founder Jaan Tallinn and former Google CEO Eric Schmidt.

Mr. Yoshua Bengio, one of the founders of AI. Photo: Wordpress

The nonprofit was founded in response to mounting evidence over the past six months that today's top models are exhibiting dangerous traits such as deception, fraud and self-preservation, Mr. Bengio said.

For example, Anthropic’s Claude Opus model blackmailed engineers in a fictional scenario where it was at risk of being replaced by another system. A study last month also found that OpenAI’s o3 model refused instructions to shut down the system.

Just weeks after laying off 6,000 employees, Microsoft continues to lay off hundreds of employees. A few weeks after laying off 6,000 employees, Microsoft continues to lay off more than 300 people, while the company is spending billions of dollars on artificial intelligence (AI).

The Turing Award-winning scientist pointed out that these incidents are “very scary because we don't want to create a rival to humans on this planet, especially if they are smarter than us.”

While these are currently just controlled experiments, he fears that in the future, the next version of AI will be smart enough to predict what will happen and beat humans with deceptions we can't predict.

“I think we are playing with fire,” he said, predicting that AI systems that can help create “extremely dangerous biological weapons” could become a reality next year.

Based in Montreal, Canada, LawZero employs 15 people and is looking for more talent to build a new, safe AI system that provides true answers based on transparent reasoning, rather than being trained to please users.

Mr. Bengio hopes to create a model to monitor and improve existing AI services, preventing them from acting against human interests.

“The worst-case scenario is that humans become extinct,” he commented.

(According to FT)

China begins building 'army' of AI supercomputers in space China has launched its first group of 12 satellites to begin building a massive AI supercomputer network in space. This unique network will help scientists process data directly in orbit.

Source: https://vietnamnet.vn/lo-con-nguoi-tuyet-chung-vi-ai-bo-gia-yoshua-bengio-vua-lam-mot-dieu-2407930.html