Pavithra Ramamurthy and Kathy Li both grew up surrounded by the suffering that accompanies cleft lip and palate (CLP) disorder.
The second-year master’s students in Human-Computer Interaction bonded over that shared experience when they met at the School of Informatics, Computing, and Engineering, and they’re blending their understanding of the struggles children with CLP face with technology to improve the lives of those kids. They developed Speech Buddy, an interactive robot designed to help kids enjoy speech therapy exercises outside of a clinical setting.
The project recently took home honors as the Best Graduate Project at SICE’s Fall Projects and Research Symposium.
“This is a big passion project for both me and Kathy,” Ramamurthy said. “I was born with a sibling who has cleft lip and palate disorder. Kathy’s father is a surgeon for cleft lip and palate children in China. Both of us came together on this project, and we decided to work to solve the problem.”
Children suffering from CLP typically undergo surgery when they reach their first birthday to correct the problem. As their speech develops and they begin to use more complex words and sentences, speech therapy often is required to articulate or pronounce words correctly. The therapy can be effective, but it’s mostly conducted in a clinical setting, and homework given by the therapist is difficult to track and can lead to limited long-term success. Parents also usually aren’t as effective as they need to be to help their children properly develop their speech.
Buddy is designed to be an at-home support for the speech therapists to make the therapy more engaging and fun for the children.
“The kids get to take Buddy home, and during playtime, Buddy can provide speech therapy in the form of storytelling and games,” Ramamurthy said. “Children won’t realize that it’s speech therapy. It’s more of a game they’re playing with a robot, one that can be fun for children with CLP, their unaffected siblings and friends, and their parents.”
The prototype for Buddy uses a smartphone with an animated face that interacts with children, and the use of voice-recognition software can provide real-time feedback.
“For instance, a child can bring Buddy home from the clinic,” Li said. “After dinner, his parents bring out Buddy for playtime. If the child was working on pronouncing the ‘S’ sound in a clinical setting, and his homework was to keep practicing that sound, he could talk to Buddy, and Buddy would have a conversation with him. Buddy could say, ‘It’s story time! I would like to take you on an adventure. Remember these three magic words: Sweet. Sheet. Slurp.’ Then, they could repeat the words as the story progresses, and the feedback will both correct the kids when needed and provide information to tell the speech therapist both how much work is being done and what areas still require improvement.”
Buddy is still in the early stages of development, and Li and Ramamurthy are still overcoming some of the challenges, such as coding and physically building the prototype. But the process has been instructive to the duo, and they’re excited about the potential of the project.
“Both of us are designers, and we had never built a physical thing before,” Ramamurthy said. “Right now, the body of the robot is built simply to hold the circuits. In the future, it will be soft. You’ll be able to squish it. We also have to develop the coding, because right now, Buddy is not processing anything and applying it. I’m doing a ‘Wizard of Oz’ control in the background, and I control his responses. In the future, that won’t be the case.”
Li and Ramamurthy have collaborated with speech therapists both at IU and in the general public at-large, and they’ve talked to people in person and online who have suffered from CLP disorder to get their feedback. They’re working to integrate that information into Buddy’s design.
“They’re the experts in their domain, and we’re just trying to create a tool that can help children with the process,” Li said. “We’re including as much background expertise as we can, and we’re learning a lot.”
The pair hope to create a commercial product in the future, and they’re applying to an HCI conference in Chicago in the spring with the hopes of finding funding to continue their work.
“People need a solution,” Ramamurthy said. “Knowing that we can put this out there and make it more affordable for children is important to us. The target audience is worldwide, and we hope we can help with this issue to improve lives.”