AI Robot Predicts and Smiles at the Same Time as the Person Opposite

The American Robot Emo can predict smiles about 840 milliseconds before the person opposite smiles, then smiles at the same time.

Humans are getting used to robots that can communicate fluently with speech, thanks in part to advances in large language models like ChatGPT, but their nonverbal communication skills, especially facial expressions, still lag far behind. Designing a robot that can not only express a variety of facial expressions but also know how to express them at the right time is extremely difficult.

The Creative Machines Lab at Columbia University’s School of Engineering has been working on this problem for more than five years. In a new study published in the journal Science Robotics , the team has introduced Emo, an AI robot that can predict human facial expressions and perform them in real time, TechXplore reported on March 27. It predicts smiles about 840 milliseconds before the person smiles, then smiles in time.

Emo is a humanoid robot with a face equipped with 26 actuators that allow for a wide range of expressions. The robot’s head is covered in soft silicone skin with a magnetic connection system, allowing for quick adjustments and maintenance. For more immersive interactions, the team integrated high-resolution cameras into the pupils of each eye, allowing Emo to interact with her eyes, which is important in non-verbal communication.

The team developed two AI models. The first model predicted human facial expressions by analyzing subtle changes in the opposite face, while the second model generated motor commands using the corresponding expressions.

To train the robot to express emotions, the team placed Emo in front of a camera and had it make random movements. After a few hours, the robot learned the relationship between facial expressions and motor commands—similar to the way humans practice expressions by looking in a mirror. The team calls this “self-modeling”—similar to how humans imagine themselves making certain expressions.

Next, the team played videos of human facial expressions for Emo to observe frame by frame. After hours of training, Emo could predict expressions by observing the subtle changes in a person’s face as they began to smile.

"I think accurately predicting human facial expressions is a revolution in the field of human-robot interaction. Previously, robots were not designed to consider human expressions during interactions. Now robots can integrate facial expressions to respond," said Yuhang Hu, a PhD student in the Creative Machines Lab and a member of the research team.

"Having robots make expressions at the same time as humans in real time not only improves the quality of interaction but also helps build trust between humans and robots. In the future, when interacting with a robot, it will observe and interpret your facial expressions, just like a real human," Hu added.

Thu Thao (According to TechXplore )

Source link