Hackathon Brings Innovative Ideas to Communication Technologies

Did you know that listening to each other is about much more than hearing?

Technology is currently in use or being developed that can track how, where and when we engage with our surroundings. We often think about using this technology while engaging with our visual world. Yet, access to our hearing world is a necessity when communicating through spoken language.

Ilse Wambacq, associate professor in Communication Sciences and Disorders, turned her Fall semester Cognition and Hearing Science course into a challenge-based learning experience, allowing her students to explore how they can make use of technology to make listening accessible.

The result was the “Access to Listening” Hackathon, where scientists, developers, individuals experiencing problems with speech perception, students and the community were invited to join together to come up with the next generation of approaches to enhance communication for all.

Challenge-based learning opportunities are an exciting way to help students become hands-on, collaborative problem solvers. “This Hackathon and class are a great example of the way we’re trying to transform education in the College and get our students ready for successful lives as professional and engaged citizens,” says Peter Kingstone, Dean of the College of Humanities and Social Sciences.

In the day-long Hackathon, students and guests were partnered into small groups and challenged to think outside of the box to brainstorm and develop innovative concepts and non-stigmatizing approaches to hearing loss and other communication problems. Wambacq hopes the unique learning approach showed students that being proactive and innovative is not only reserved for those at the top, but can be practiced by anyone.

“Learning in the classroom is no longer about obtaining an A on an exam, but it encompasses being part of a global world with the well-being of all in mind. This experience will shape students in their future as audiologists, knowing they can make a difference in their patients’ lives beyond being a clinician.”

The best idea(s) from the Hackathon will be presented to the Behavioral Management and Social Sciences labs of the University of Twente for further exploration and development.

Below are brief summaries of each group proposal from the Hackathon.

Team 1: Let Your Brain Tell Us More

We all have experienced it – we are sitting in a noisy restaurant, trying to make sense of the conversation at the table but fail to do so. Many patients visiting the audiology clinic have normal hearing capabilities yet have problems with speech perception in difficult listening conditions. Current testing protocols in the audiology clinic are not sensitive enough to provide the patient with a thorough understanding of their perceptive listening problems, which leaves a large number of patients under-served. Our solution tackles this problem by using current neuro-technological and behavioral advances.

Team 2: Classy Chat App

Our idea is based on the notion that speech processing is a vital aspect of being a part of a larger community. During spontaneous communication, the brain acts as a multi-step system involving implicit and explicit processing to interpret degraded signals. Current assistive listening technologies are designed to amplify speech signals but do not include crucial multi-modal communication strategies. The development of the Classy Chat app provides users with a way to turn their personal phones into live captions thus decreasing cognitive effort. Using current AI talk to text technology will personalize this app to learn the user’s individual voice affectations. Features like quality-of-life updates and social networking will provide users with opportunities to immerse themselves and play an active role in all conversations.

Team 3: Zoom Out – Technology Accessible to All

Modern college campuses have large lecture halls that can potentially seat hundreds of people. In this type of setting where there are brick walls, heaters, construction outside, cellphones, and other distractions it seems almost impossible for every seat to be a good one. This can put students at a disadvantage simply based on where they are placed in the classroom. When you add personal difficulties with attention, hearing, vision, lack of typing/writing speed these problems only get worse. Zoom Out hopes to work towards a solution to some of these problems. Regardless of a person’s disabilities, learning difficulty or seat choice, Zoom Out can make every 300-person lecture hall feel like a classroom built for 20. Zoom Out uses a combination of high-quality audio and video recording equipment (our learning lens) along with a students own personal device to help give them a front row experience

Team 4: Democratizing the Town Hall Meeting

How can we encourage participation in democracy and ensure inclusion and community while reducing stigma in public gathering places? Town hall meetings are a prime example of a place where due to their setting people with sensory losses and other differences are easily stigmatized and excluded from participation. Some challenges of town hall meetings include high time pressure (due to the preset agenda of the meeting) and heavy reliance upon communication. Often the physical spaces in which these town hall meetings take place are not ideal for individuals who experience attention difficulties or perception difficulties, to name a few. It is our goal to engage public officials in establishing a stigma-free and communication-conducive zone for all.

• The teams proposal, “Breaking down communication barriers on the way to a more inclusive democratic future” was submitted and accepted to present at the Futures Conference in Turku, Finland in June 2023. They will be presenting their ideas, as well as the outcomes of the CHSS co-creation event, Making Democracy Work for All. The co-creation event is a follow up event, with the goals of developing the concept further and identifying strategic solutions.