Driven by his concern for the difficulties faced by visually and hearing impaired people in their daily lives, Phan Hai Dang, a student at Luong The Vinh High School for the Gifted ( Dong Nai province), researched and developed an application called "Multifunctional System to Support the Deaf and Blind Using Deep Learning Technology".
Phan Hai Dang, a student from Luong The Vinh High School for the Gifted (Dong Nai province), presents his application. Photo: AX
Although not from a specialized computer science program, Dang has a particular passion for programming, especially deep learning technology. Unlike many other young people who choose to develop entertainment or educational applications, Dang chose a more challenging path: building applications to help deaf and blind people communicate and integrate into the community.
The application developed by Dang is capable of converting sign language to text and speech, and transforming spoken language into corresponding text or signs, facilitating easy interaction between the two parties. The system integrates various AI technologies such as the Yolv8 algorithm, ArcFace facial recognition, and optical character recognition (OCR), etc.
Simply tap the microphone icon and click "Speak," and the app will record your speech, then display a video tutorial using appropriate sign language. Deaf users can watch this video to understand the message visually. Conversely, the camera will record the user's hand gestures, convert them into text, and play them as audio so that hearing users can understand.
"The application can also recognize emotions through eye contact and facial expressions to accurately reflect the attitude of the person communicating, thereby increasing the effectiveness of interaction and connection in society," Dang shared.
For visually impaired users, the navigation assistance system uses an integrated camera to identify objects in front, measure distances, and provide audible warnings to help users avoid collisions while driving. Additionally, facial recognition allows users to identify loved ones and make emergency calls via Google Assistant using voice commands.
In addition, the application integrates many practical support features such as traffic light recognition, currency denomination reading, book reading, location tracking, etc.
Behind Dang's journey of bringing technology closer to people with disabilities is the dedicated support of his teacher, Bui Kim Minh, who directly guided, inspired, and opened the door to practical applications for this programming-loving student.
According to Mr. Minh, from the moment Dang conceived the idea of an AI application to support communication for people with disabilities, Mr. Minh saw the creative potential of the project. Instead of just stopping at theory, he proactively connected with the Center for the Education of Children with Disabilities in Dong Nai province to gain practical experience.
"I believe that if you want technology to have real value, it must be tested in real-world situations where users actually need it. As a teacher myself, I support and connect Dang with the technology he created so he can apply it in practice," Mr. Minh shared.
Teachers from Luong The Vinh High School for the Gifted congratulate Hai Dang on winning first prize at the provincial level and third prize at the national level. Photo: AX
At the Dong Nai Provincial Center for the Education of Children with Disabilities, both teachers and students spent weeks listening, observing, experimenting, and making adjustments. Continuously updated versions were improved based on feedback from the children with disabilities and those who directly experienced the product.
After numerous trials, evaluations, and improvements, the system now achieves an accuracy rate of approximately 95% in sign language recognition. For Mr. Minh, the joy doesn't lie in awards or recognition, but in the moment of seeing his students mature in their thinking and actions.
"Teaching a student how to use technology isn't difficult. But teaching them how to use technology to serve the community—that's the ultimate goal of educators ," Mr. Minh emphasized.
With his idea of applying Deep Learning technology to convert sign language into speech and vice versa, Dang excellently won first prize at the provincial level and third prize at the national level in the Science and Technology competition for high school students.
A female student from an SOS Children's Village wins a rare scholarship and graduates from university with honors. Growing up in the SOS Children's Village in Hai Phong, Phuong Anh never felt miserable or pitiful. At the age of 18, Phuong Anh won a rare scholarship and then went on to graduate from university with honors.
Source: https://vietnamnet.vn/nam-sinh-o-dong-nai-gianh-giai-quoc-gia-voi-ung-dung-ho-tro-nguoi-khuyet-tat-2490981.html
Comment (0)