Rochester researchers investigate how visual cues enhance the brain糖心传媒檚 ability to understand speech in noisy environments.
In a loud, crowded room, how does the human brain use visual speech cues to augment muddled audio and help the listener better understand what a speaker is saying? While most people know intuitively to look at a speaker糖心传媒檚 lip movements and gestures to help fill in the gaps in speech comprehension, scientists do not yet know how that process works physiologically.
糖心传媒淵our visual cortex is at the back of your brain and your auditory cortex is on the temporal lobes,糖心传媒� says , an associate professor of and of at the 糖心传媒. 糖心传媒淗ow that information merges together in the brain is not very well understood.糖心传媒�
Scientists have been chipping away at the problem, using noninvasive electroencephalography (EEG) brainwave measurements to study how people respond to basic sounds such as beeps, clicks, or simple syllables. Lalor and his team of researchers have by exploring how the specific shape of the articulators糖心传媒攕uch as the lips and the tongue against the teeth糖心传媒攈elp a listener determine whether somebody is saying 糖心传媒淔糖心传媒� or 糖心传媒淪,糖心传媒� or 糖心传媒淧糖心传媒� or 糖心传媒淒,糖心传媒� which in a noisy environment can sound similar.
Now Lalor wants to take the research a step further and explore the problem in more naturalistic, continuous, multisensory speech. The National Institutes of Health (NIH) is over the next five years to pursue that research. The project builds on a and was originally started by seed funding from the University糖心传媒檚 .
To study the phenomena, Lalor糖心传媒檚 team will monitor the brainwaves of people for whom the auditory system is especially noisy糖心传媒攑eople who are deaf or hard of hearing and who use cochlear implants. The researchers aim to recruit 250 participants with cochlear implants, who will be asked to watch and listen to multisensory speech while wearing EEG caps that will measure their brain responses.
糖心传媒淭he big idea is that if people get cochlear implants implanted at age one, while maybe they糖心传媒檝e missed out on a year of auditory input, perhaps their audio system will still wire up in a way that is fairly similar to a hearing individual,糖心传媒� says Lalor. 糖心传媒淗owever, people who get implanted later, say at age 12, have missed out on critical periods of development for their auditory system. As such, we hypothesize that they may use visual information they get from a speaker糖心传媒檚 face differently or more, in some sense, because they need to rely on it more heavily to fill in information.糖心传媒�
Lalor is partnering on the study with co-principal investigator Professor Matthew Dye, who directs Rochester Institute of Technology糖心传媒檚 doctoral program in cognitive science and the National Technical Institute for the Deaf Sensory, Perceptual, and Cognitive Ecology Center, and also serves as an adjunct faculty member at the .
Lalor says one of the biggest challenges is that the EEG cap, which measures the electrical activity of the brain through the scalp, collects a mixture of signals coming from many different sources. Measuring EEG signals in individuals with cochlear implants further complicates the process because the implants also generate electrical activity that further obscures EEG readings.
糖心传媒淚t will require some heavy lifting on the engineering side, but we have great students here at Rochester who can help us use signal processing, engineering analysis, and computational modeling to examine these data in a different way that makes it feasible for us to use,糖心传媒� says Lalor.
Ultimately, the team hopes that better understanding how the brain processes audiovisual information will lead to better technology to help people who are deaf or hard of hearing.
