Months before their first words, babies’ brains rehearse speech mechanics

Infants can tell the difference between sounds of all languages until about 8 months of age when their brains start to focus only on the sounds they hear around them. It’s been unclear how this transition occurs, but social interactions and caregivers’ use of exaggerated “parentese” style of speech seem to help.

University of Washington research in 7- and 11-month-old infants shows that speech sounds stimulate areas of the brain that coordinate and plan motor movements for speech.

The study, published July 14 in the Proceedings of the National Academy of Sciences, suggests that baby brains start laying down the groundwork of how to form words long before they actually begin to speak, and this may affect the developmental transition.

“Most babies babble by 7 months, but don’t utter their first words until after their first birthdays,” said lead author Patricia Kuhl, who is the co-director of the UW’s Institute for Learning and Brain Sciences. “Finding activation in motor areas of the brain when infants are simply listening is significant, because it means the baby brain is engaged in trying to talk back right from the start and suggests that 7-month-olds’ brains are already trying to figure out how to make the right movements that will produce words.”

Kuhl and her research team believe this practice at motor planning contributes to the transition when infants become more sensitive to their native language.

The results emphasize the importance of talking to kids during social interactions even if they aren’t talking back yet.

“Hearing us talk exercises the action areas of infants’ brains, going beyond what we thought happens when we talk to them,” Kuhl said. “Infants’ brains are preparing them to act on the world by practicing how to speak before they actually say a word.”

In the experiment, infants sat in a brain scanner that measures brain activation through a noninvasive technique called magnetoencephalography. Nicknamed MEG, the brain scanner resembles an egg-shaped vintage hair dryer and is completely safe for infants. The Institute for Learning and Brain Sciences was the first in the world to use such a tool to study babies while they engaged in a task.

The babies, 57 7- and 11- or 12-month-olds, each listened to a series of native and foreign language syllables such as “da” and “ta” as researchers recorded brain responses. They listened to sounds in English and in Spanish.

The researchers observed brain activity in an auditory area of the brain called the superior temporal gyrus, as well as in Broca’s area and the cerebellum, cortical regions responsible for planning the motor movements required for producing speech.

This pattern of brain activation occurred for sounds in the 7-month-olds’ native language (English) as well as in a non-native language (Spanish), showing that at this early age infants are responding to all speech sounds, whether or not they have heard the sounds before.

In the older infants, brain activation was different. By 11-12 months, infants’ brains increase motor activation to the non-native speech sounds relative to native speech, which the researchers interpret as showing that it takes more effort for the baby brain to predict which movements create non-native speech. This reflects an effect of experience between 7 and 11 months, and suggests that activation in motor brain areas is contributing to the transition in early speech perception.

The study has social implications, suggesting that the slow and exaggerated parentese speech – “Hiiiii! How are youuuuu?” – may actually prompt infants to try to synthesize utterances themselves and imitate what they heard, uttering something like “Ahhh bah bah baaah.”

“Parentese is very exaggerated, and when infants hear it, their brains may find it easier to model the motor movements necessary to speak,” Kuhl said.

Gestures that speak

When you gesticulate you don’t just add a “note of colour” that makes your speech more pleasant: you convey information on sentence structure and make your meanings clearer. A study carried out at SISSA in Trieste demonstrates that gestures and “prosody” (the intonation and rhythm of spoken language) form  a  single “communication system” at the cognitive level, and that we speak using our “whole  body” and not only our vocal tract.

Have you ever found yourself gesticulating and felt a bit stupid for it while talking on the phone?

You’re not alone: it happens very often that people accompany their speech with hand gestures, sometimes even when no one can see them. Why can’t we keep still while speaking? “Because gestures and words very probably form a single “communication system”, which ultimately serves to enhance expression intended as the ability to make oneself understood”, explains Marina Nespor, a neuroscientist at the International School for Advanced Studies (SISSA) of Trieste. Nespor, together with Alan Langus, a SISSA research fellow, and Bahia Guellai from the Université Paris Ouest Nanterre La Défence, who conducted the investigation at SISSA, has just published a study in Frontiers in Psychology which demonstrates the role of gestures in speech “prosody”.

Linguists define prosody as the intonation and rhythm of spoken language, features that help to highlight sentence structure and therefore make the message easier to understand. For example, without prosody, nothing would distinguish the declarative statement “this is an apple” from the surprise question “this is an apple?” (in this case the difference lies in the intonation).

According to Nespor and colleagues, even hand gestures are part of prosody: “the prosody that accompanies speech is not ‘modality specific” explains Langus. “Prosodic information, for the person receiving the message, is a combination of auditory and visual cues. The ‘superior’ aspects (at the cognitive processing level) of spoken language are mapped to the motor‐programs responsible for the production of both speech sounds and accompanying hand gestures”.

Nespor, Langus and Guellai had 20 Italian speakers listen to a series of “ambiguous” utterances, which could be said with different prosodies corresponding to two different meanings. Examples of utterances were “come sicuramente hai visto la vecchia sbarra la porta” where, depending on meaning, “vecchia” can be the subject of the main verb (sbarrare, to block) or an adjective qualifying the subject (sbarra, bar) (‘As you for sure have seen the old lady blocks the door’ versus ‘As you for sure have seen the old bar carries it’). The utterances could be simply listened to (“audio only” modality) or be presented in a video, where the participants could both listen to the sentences and see the accompanying gestures. In the “video” stimuli, the condition could be “matched” (gestures corresponding to the meaning conveyed by speech prosody) or “mismatched” (gestures matching the alternative meaning).

“In the matched conditions there was no improvement ascribable to gestures: the  participants’ performance was very good both in the video and in the “audio only” sessions. It’s in the mismatched condition that the effect of hand gestures became apparent”, explains Langus. “With these stimuli the subjects were much more likely to make the wrong choice (that is, they’d choose the meaning indicated in the gestures rather than in the speech) compared to matched or audio only conditions. This means that gestures affect how meaning is interpreted, and we believe this points to the existence of a common cognitive system for gestures, intonation and rhythm of spoken language”.

“In human communication, voice is not sufficient: even the torso and in particular hand movements are involved, as are facial expressions”, concludes Nespor.

Infants Benefit from Implants with More Frequency Sounds

A new study from a UT Dallas researcher demonstrates the importance of considering developmental differences when creating programs for cochlear implants in infants.

Dr. Andrea Warner-Czyz, assistant professor in the School of Behavioral and Brain Sciences, recently published the research in the Journal of the Acoustical Society of America.

“This is the first study to show that infants process degraded speech that simulates a cochlear implant differently than older children and adults, which begs for new signal processing strategies to optimize the sound delivered to the cochlear implant for these young infants,” Warner-Czyz said.

Cochlear implants, which are surgically placed in the inner ear, provide the ability to hear for some people with severe to profound hearing loss. Because of technological and biological limitations, people with cochlear implants hear differently than those with normal hearing.

Think of a piano, which typically has 88 keys with each representing a note. The technology in a cochlear implant can’t play every key, but instead breaks them into groups, or channels. For example, a cochlear implant with 22 channels would put four notes into each group. If any keys within a group are played, all four notes are activated. Although the general frequency can be heard, the fine detail of the individual notes is lost.

Two of the major components necessary for understanding speech are the rhythm and the frequencies of the sound. Timing remains fairly accurate in cochlear implants, but some frequencies disappear as they are grouped.

More than eight or nine channels do not necessarily improve the hearing of speech in adults. This study is one of the first to examine how this signal degradation affects hearing speech in infants.

Infants pay greater attention to new sounds, so researchers compared how long a group of 6-month-olds focused on a speech sound they were familiarized with —“tea”’ — to a new speech sound, “ta.”

The infants spent more time paying attention to “ta,” demonstrating they could hear the difference between the two. Researchers repeated the experiment with speech sounds that were altered to sound as if they had been processed by a 16- or 32-channel cochlear implant.

The infants responded to the sounds that imitated a 32-channel implant the same as when they heard the normal sounds. But the infants did not show a difference with the sounds that imitated a 16-channel implant.

“These results suggest that 6-month-old infants need less distortion and more frequency information than older children and adults to discriminate speech,” Warner-Czyz said. “Infants are not just little versions of children or adults. They do not have the experience with listening or language to fill in the gaps, so they need more complete speech information to maximize their communication outcomes.”

Clinicians need to consider these developmental differences when working with very young cochlear implant recipients, Warner-Czyz said.

Language Structure… You’re Born with It

Humans are unique in their ability to acquire language. But how? A new study published in the Proceeding of the National Academy of Sciences shows that we are in fact born with the basic fundamental knowledge of language, thus shedding light on the age-old linguistic “nature vs. nurture” debate.

THE STUDY

While languages differ from each other in many ways, certain aspects appear to be shared across languages. These aspects might stem from linguistic principles that are active in all human brains. A natural question then arises: are infants born with knowledge of how the human words might sound like? Are infants biased to consider certain sound sequences as more word-like than others? “The results of this new study suggest that, the sound patterns of human languages are the product of an inborn biological instinct, very much like birdsong,” said Prof. Iris Berent of Northeastern University in Boston, who co-authored the study with a research team from the International School of Advanced Studies in Italy, headed by Dr. Jacques Mehler. The study’s first author is Dr. David Gómez.

BLA, ShBA, LBA

Consider, for instance, the sound-combinations that occur at the beginning of words. While many languages have words that begin by bl (e.g., blando in Italian, blink in English, and blusa in Spanish), few languages have words that begin with lb. Russian is such a language (e.g., lbu, a word related to lob, “forehead”), but even in Russian such words are extremely rare and outnumbered by words starting with bl. Linguists have suggested that such patterns occur because human brains are biased to favor syllables such as bla over lba. In line with this possibility, past experimental research from Dr. Berent’s lab has shown that adult speakers display such preferences, even if their native language has no words resembling either bla or lba. But where does this knowledge stem from? Is it due to some universal linguistic principle, or to adults’ lifelong experience with listening and producing their native language?

THE EXPERIMENT

These questions motivated our team to look carefully at how young babies perceive different types of words. We used near-infrared spectroscopy, a silent and non-invasive technique that tells us how the oxygenation of the brain cortex (those very first centimeters of gray matter just below the scalp) changes in time, to look at the brain reactions of Italian newborn babies when listening to good and bad word candidates as described above (e.g., blif, lbif).

Working with Italian newborn infants and their families, we observed that newborns react differently to good and bad word candidates, similar to what adults do. Young infants have not learned any words yet, they do not even babble yet, and still they share with us a sense of how words should sound. This finding shows that we are born with the basic, foundational knowledge about the sound pattern of human languages.

It is hard to imagine how differently languages would sound if humans did not share such type of knowledge. We are fortunate that we do, and so our babies can come to the world with the certainty that they will readily recognize the sound patterns of words–no matter the language they will grow up with.

Some innate preferences shape the sound of words from birth

Languages are learned, it’s true, but are there also innate bases in the structure of language that precede experience? Linguists have noticed that, despite the huge variability of human languages, here are some preferences in the sound of words that can be found across languages. So they wonder whether this reflects the existence of a universal, innate biological basis of language. A SISSA study provides evidence to support this hypothesis, demonstrating that certain preferences in the sound of words are already active in newborn infants.

Take the sound “bl”: how many words starting with that sound can you think of? Blouse, blue, bland… Now try with “lb”: how many can you find? None in English and Italian, and even in other languages such words either don’t exist or are extremely rare. Human languages offer several examples of this kind, and this indicates that in forming words we tend to prefer certain sound combinations to others, irrespective of which language we speak. The fact that this occurs across languages has prompted linguists to hypothesize the existence of biological bases of language (in born and universal) which precede language learning in humans. Finding evidence to support his hypothesis is, however, far from easy and the debate between the proponents of this view and those who believe that language is merely the result of learning is still open. But proof supporting the “universalist” hypothesis has now been provided by a new study conducted by a research team of the International School for Advanced Studies (SISSA) in Trieste and just published in the journal PNAS.

David Gomez, a SISSA research scientist working under the supervision of Jacques Mehler and first author of the paper, and his coworkers decided to observe the brain activity of newborns. “In fact, if it is possible to demonstrate that these preferences are already present within days from birth, when the newborn baby is still unable to speak and presumably has very limited language knowledge, then we can infer that there is an inborn bias that prefers certain words to others”, comments Gomez.

“To monitor the newborns’ brain activity we used a non-invasive technique, i.e., functional near-infrared spectroscopy”, explains Marina Nespor, a SISSA neuroscientist who participated in the study. During the experiments the newborns would listen to words starting with normally “preferred” sounds (like “bl”) and others with  uncommon sounds (“lb”). “What we found was that the newborns’ brains reacted in a significantly different manner to the two types of sound” continues Nespor.

“The brain regions that are activated while the newborns are listening react differently in the two cases”, comments Gomez, “and reflect the preferences observed across languages, as well as the behavioural responses recorded in similar experiments carried out in adults”. “It’s difficult to imagine what languages would sound like if humans didn’t share a common knowledge base”, concludes Gomez. “We are lucky that this common base exists. This way, our children are born with an ability to distinguish words from “non-words” ever since birth, regardless of which language they will then go on to learn”.

Decoding ‘noisy’ language in daily life

Suppose you hear someone say, “The man gave the ice cream the child.” Does that sentence seem plausible? Or do you assume it is missing a word? Such as: “The man gave the ice cream to the child.”

A new study by MIT researchers indicates that when we process language, we often make these kinds of mental edits. Moreover, it suggests that we seem to use specific strategies for making sense of confusing information — the “noise” interfering with the signal conveyed in language, as researchers think of it.

“Even at the sentence level of language, there is a potential loss of information over a noisy channel,” says Edward Gibson, a professor in MIT’s Department of Brain and Cognitive Sciences (BCS) and Department of Linguistics and Philosophy.

Gibson and two co-authors detail the strategies at work in a new paper, “Rational integration of noisy evidence and prior semantic expectations in sentence interpretation,” published today in the Proceedings of the National Academy of Sciences.

“As people are perceiving language in everyday life, they’re proofreading, or proof-hearing, what they’re getting,” says Leon Bergen, a PhD student in BCS and a co-author of the study. “What we’re getting is quantitative evidence about how exactly people are doing this proofreading. It’s a well-calibrated process.”

Asymmetrical strategies

The paper is based on a series of experiments the researchers conducted, using the Amazon Mechanical Turk survey system, in which subjects were presented with a series of sentences — some evidently sensible, and others less so — and asked to judge what those sentences meant.

A key finding is that given a sentence with only one apparent problem, people are more likely to think something is amiss than when presented with a sentence where two edits may be needed. In the latter case, people seem to assume instead that the sentence is not more thoroughly flawed, but has an alternate meaning entirely.

“The more deletions and the more insertions you make, the less likely it will be you infer that they meant something else,” Gibson says. When readers have to make one such change to a sentence, as in the ice cream example above, they think the original version was correct about 50 percent of the time. But when people have to make two changes, they think the sentence is correct even more often, about 97 percent of the time.

Thus the sentence, “Onto the cat jumped a table,” which might seem to make no sense, can be made plausible with two changes — one deletion and one insertion — so that it reads, “The cat jumped onto a table.” And yet, almost all the time, people will not infer that those changes are needed, and assume the literal, surreal meaning is the one intended.

This finding interacts with another one from the study, that there is a systematic asymmetry between insertions and deletions on the part of listeners.

“People are much more likely to infer an alternative meaning based on a possible deletion than on a possible insertion,” Gibson says.

Suppose you hear or read a sentence that says, “The businessman benefitted the tax law.” Most people, it seems, will assume that sentence has a word missing from it — “from,” in this case — and fix the sentence so that it now reads, “The businessman benefitted from the tax law.” But people will less often think sentences containing an extra word, such as “The tax law benefitted from the businessman,” are incorrect, implausible as they may seem.

Another strategy people use, the researchers found, is that when presented with an increasing proportion of seemingly nonsensical sentences, they actually infer lower amounts of “noise” in the language. That means people adapt when processing language: If every sentence in a longer sequence seems silly, people are reluctant to think all the statements must be wrong, and hunt for a meaning in those sentences. By contrast, they perceive greater amounts of noise when only the occasional sentence seems obviously wrong, because the mistakes so clearly stand out.

“People seem to be taking into account statistical information about the input that they’re receiving to figure out what kinds of mistakes are most likely in different environments,” Bergen says.

Reverse-engineering the message

Other scholars say the work helps illuminate the strategies people may use when they interpret language.

“I’m excited about the paper,” says Roger Levy, a professor of linguistics at the University of California at San Diego who has done his own studies in the area of noise and language.

According to Levy, the paper posits “an elegant set of principles” explaining how humans edit the language they receive. “People are trying to reverse-engineer what the message is, to make sense of what they’ve heard or read,” Levy says.

“Our sentence-comprehension mechanism is always involved in error correction, and most of the time we don’t even notice it,” he adds. “Otherwise, we wouldn’t be able to operate effectively in the world. We’d get messed up every time anybody makes a mistake.”

A Confederacy of Senses

Research on multisensory speech perception in recent years has helped revolutionize our understanding of how the brain organizes the information it receives from our many different senses, UC Riverside psychology professor Lawrence D. Rosenblum writes in the January 2013 issue of Scientific American.

“Neuroscientists and psychologists have largely abandoned early ideas of the brain as a Swiss Army knife, in which many distinct regions are dedicated to different senses,” he says. “Instead scientists now think that the brain has evolved to encourage as much cross talk as possible between the senses — that the brain’s sensory regions are physically intertwined.”

The article, “A Confederacy of Senses,” explains how research in the past 15 years has demonstrated that no sense works alone. An abstract of the article can be read here.

“The multisensory revolution is also suggesting new ways to improve devices for the blind and deaf, such as cochlear implants,” Rosenblum writes. This research also has improved speech-recognition software, he says.

Researchers have discovered that the brain “does not channel visual information from the eyes into one neural container and auditory information from the ears into another, discrete, container as though it were sorting coins,” Rosenblum writes. “Rather our brains derive meaning from the world in as many ways as possible by blending the diverse forms of sensory perception.”

Rosenblum is the author of “See What I’m Saying: The Extraordinary Powers of Our Five Senses” (Norton, 2010), and has spent two decades studying multisensory perception, lipreading and hearing. His research has been supported by the National Science Foundation and the National Institutes of Health. He is known internationally for his research on risks the inaudibility of hybrid cars pose for blind and other pedestrians.