Strategic or Random? How the Brain Chooses

Many of the choices we make are informed by experiences we’ve had in the past. But occasionally we’re better off abandoning those lessons and exploring a new situation unfettered by past experiences. Scientists at the Howard Hughes Medical Institute’s Janelia Research Campus have shown that the brain can temporarily disconnect information about past experience from decision-making circuits, thereby triggering random behavior.

In the study, rats playing a game for a food reward usually acted strategically, but switched to random behavior when they confronted a particularly unpredictable and hard-to-beat competitor. The animals sometimes got stuck in a random-behavior mode, but the researchers, led by Janelia lab head Alla Karpova and postdoctoral fellow Gowan Tervo, found that they could restore normal behavior by manipulating activity in a specific region of the brain. Because the behavior of animals stuck in this random mode bears some resemblance to that of patients affected by a psychological condition called learned helplessness, the findings may help explain that condition and suggest strategies for treating it. Karpova, Tervo and their colleagues published their findings in the September 25, 2012, issue of the journal Cell.

The brain excels at integrating information from past experiences to guide decision-making in new situations. But in certain circumstances, random behavior may be preferable. An animal might have the best chance of avoiding a predator if it moves unpredictably, for example. And in a new environment, unrestricted exploration might make more sense than relying on an internal model developed elsewhere. So scientists have long speculated that the brain may have a way to switch off the influence of past experiences so that behavior can proceed randomly, Karpova says. But others disagreed. “They argue that it’s inefficient, and that it would be at odds with what some people call one of the most central operating principles of the brain – to use our past experience and knowledge to optimize behavioral choices,” she notes.

Karpova and her colleagues wanted to see if they could create a situation that would force animals to switch into this random mode of behavior. “We tried to create a setting that would push the need to create behavioral variability and unpredictability to its extreme,” she says. They did this by placing rats in a competitive setting in which a computer-simulated competitor determined which of two holes in a wall would provide a sugary reward. The virtual competitor, whose sophistication was varied by the experimenters, analyzed the rats’ behavior to predict their future choices.

“We thought if we came up with very sophisticated competitors, then the animals would eventually be unable to figure out how to outcompete them, and be forced to either give up or switch into this [random] mode, if such a mode exists,” Karpova says. And that’s exactly what happened: When faced with a weak competitor, the animals made strategic choices based on the outcomes of previous trials. But when a sophisticated competitor made strong predictions, the rats ignored past experience and made random selections in search of a reward.

Now that they had evidence that the brain could generate both strategic and random behavior, Karpova and her colleagues wanted to know how it switched between modes. Since that switch determines whether or not an animal’s internal model of the world influences its behavior, the scientists suspected it might involve a brain region called the anterior cingulate cortex, where that internal model is likely encoded.

They found that they could cause animals to switch between random and strategic behavior by manipulating the level of a stress hormone called norepinephrine in the anterior cingulate cortex. Increasing norepinephrine in the region activated random behavior and suppressed the strategic mode.  Inhibiting release of the hormone had the opposite effect.

Karpova’s team observed that animals in their experiments sometimes continued to behave randomly, even when such behavior was no longer advantageous. “If all they’ve experienced is this really sophisticated competitor for several sessions that thwarts their attempts at strategic, model-based counter-prediction, they go into this [random mode], and they can get stuck in it for quite some time after that competitor is gone,” she says. This, she says, resembles the condition of learned helplessness, in which strategic decision-making is impaired following an experience in which a person finds they are unable to control their environment.

The scientists could release the animals from this “stuck” state by suppressing the release of norepinephrine in the anterior cingulate cortex. “Just by manipulating a single neuromodulatory input into one brain area, you can dramatically enhance the strategic mode. The effect is strong enough to rescue animals out of the random mode and successfully transform them into strategic decision makers,” Karpova says. “We think this might shed light on what has gone wrong in conditions such as learned helplessness, and possibly how we can help alleviate them.” 

Karpova says that now that her team has uncovered a mechanism that switches the brain between random and strategic behavior, she would like to understand how those behaviors are controlled in more natural settings. “We normally try to use all of our knowledge to think strategically, but sometimes we still need to explore,” she says. In most cases, that probably means brief bouts of random behavior during times when we are uncertain that past experience is relevant, followed by a return to more strategic behavior – a more subtle balance that Karpova intends to investigate at the level of changes in activity in individual neural circuits.

Brain training works, but just for the practiced task

Search for “brain training” on the Web. You’ll find online exercises, games, software, even apps, all designed to prepare your brain to do better on any number of tasks. Do they work? University of Oregon psychologists say, yes, but “there’s a catch.”

The catch, according to Elliot T. Berkman, a professor in the Department of Psychology and lead author on a study published in the Jan. 1 issue of the Journal of Neuroscience, is that training for a particular task does heighten performance, but that advantage doesn’t necessarily carry over to a new challenge.

The training provided in the study caused a proactive shift in inhibitory control. However, it is not clear if the improvement attained extends to other kinds of executive function such as working memory, because the team’s sole focus was on inhibitory control, said Berkman, who directs the psychology department’s Social and Affective Neuroscience Lab.

"With training, the brain activity became linked to specific cues that predicted when inhibitory control might be needed," he said. "This result is important because it explains how brain training improves performance on a given task — and also why the performance boost doesn’t generalize beyond that task."

Sixty participants (27 male, 33 females and ranging from 18 to 30 years old) took part in a three-phase study. Change in their brain activity was monitored with functional magnetic resonance imaging (fMRI).

Half of the subjects were in the experimental group that was trained with a task that models inhibitory control — one kind of self-control — as a race between a “go” process and a “stop” process. A faster stop process indicates more efficient inhibitory control.

In each of a series of trials, participants were given a “go” signal — an arrow pointing left or right. Subjects pressed a key corresponding to the direction of the arrow as quickly as possible, launching the go process. However, on 25 percent of the trials, a beep sounded after the arrow appeared, signaling participants to withhold their button press, launching the stop process.

Participants practiced either the stop-signal task or a control task that didn’t affect inhibitory control every other day for three weeks. Performance improved more in the training group than in the control group.

Neural activity was monitored using functional magnetic resonance imaging (fMRI), which captures changes in blood oxygen levels, during a stop-signal task. MRI work was done in the UO’s Robert and Beverly Lewis Center for Neuroimaging. Activity in the inferior frontal gyrus and anterior cingulate cortex — brain regions that regulate inhibitory control — decreased during inhibitory control but increased immediately before it in the training group more than in the control group.

The fMRI results identified three regions of the brain of the trained subjects that showed changes during the task, prompting the researchers to theorize that emotional regulation may have been improved by reducing distress and frustration during the trials. Overall, the size of the training effect is small. A challenge for future research, they concluded, will be to identify protocols that might generate greater positive and lasting effects.”Researchers at the University of Oregon are using tools and technologies to shed new light on important mechanisms of cognitive functioning such as executive control,” said Kimberly Andrews Espy, vice president for research and innovation and dean of the UO Graduate School. “This revealing study on brain training by Dr. Berkman and his team furthers our understanding of inhibitory control and may lead to the design of better prevention tools to promote mental health.”

Anxious? Activate Your Anterior Cingulate Cortex With a Little Meditation

Scientists, like Buddhist monks and Zen masters, have known for years that meditation can reduce anxiety, but not how. Scientists at Wake Forest Baptist Medical Center, however, have succeeded in identifying the brain functions involved.

“Although we’ve known that meditation can reduce anxiety, we hadn’t identified the specific brain mechanisms involved in relieving anxiety in healthy individuals,” said Fadel Zeidan, Ph.D., postdoctoral research fellow in neurobiology and anatomy at Wake Forest Baptist and lead author of the study. “In this study, we were able to see which areas of the brain were activated and which were deactivated during meditation-related anxiety relief.”

The study is published in the current edition of the journal Social Cognitive and Affective Neuroscience.

For the study, 15 healthy volunteers with normal levels of everyday anxiety were recruited for the study. These individuals had no previous meditation experience or anxiety disorders. All subjects participated in four 20-minute classes to learn a technique known as mindfulness meditation. In this form of meditation, people are taught to focus on breath and body sensations and to non-judgmentally evaluate distracting thoughts and emotions.

Both before and after meditation training, the study participants’ brain activity was examined using a special type of imaging – arterial spin labeling magnetic resonance imaging – that is very effective at imaging brain processes, such as meditation. In addition, anxiety reports were measured before and after brain scanning.

The majority of study participants reported decreases in anxiety. Researchers found that meditation reduced anxiety ratings by as much as 39 percent.

“This showed that just a few minutes of mindfulness meditation can help reduce normal everyday anxiety,” Zeidan said.

The study revealed that meditation-related anxiety relief is associated with activation of the anterior cingulate cortex and ventromedial prefrontal cortex, areas of the brain involved with executive-level function. During meditation, there was more activity in the ventromedial prefrontal cortex, the area of the brain that controls worrying. In addition, when activity increased in the anterior cingulate cortex – the area that governs thinking and emotion – anxiety decreased.

“Mindfulness is premised on sustaining attention in the present moment and controlling the way we react to daily thoughts and feelings,” Zeidan said. “Interestingly, the present findings reveal that the brain regions associated with meditation-related anxiety relief are remarkably consistent with the principles of being mindful.”

Research at other institutions has shown that meditation can significantly reduce anxiety in patients with generalized anxiety and depression disorders. The results of this neuroimaging experiment complement that body of knowledge by showing the brain mechanisms associated with meditation-related anxiety relief in healthy people, he said.

‘Should I stay or should I go?’ CSHL scientists link brain cell types to behavior

You are sitting on your couch flipping through TV channels trying to decide whether to stay put or get up for a snack. Such everyday decisions about whether to “stay” or to “go” are supported by a brain region called the anterior cingulate cortex (ACC), which is part of the prefrontal cortex. Neuroscientists from Cold Spring Harbor Laboratory (CSHL) have now identified key circuit elements that contribute to such decisions in the ACC.

CSHL Associate Professor Adam Kepecs and his team publish results that, for the first time, link specific brain cell types to a particular behavior pattern in mice – a “stay or go” pattern called foraging behavior. The paper, published online in Nature, shows that the firing of two distinct types of inhibitory neurons, known as somatostatin (SOM) and parvalbumin (PV) neurons, has a strong correlation with the start and end of a period of foraging behavior.

Linking specific neuronal types to well-defined behaviors has proved extremely difficult. “There’s a big gap in our knowledge between our understanding of neuron types in terms of their physical location and their place in any given neural circuit, and what these neurons actually do during behavior,” says Kepecs.

Part of the problem is the technical challenge of doing these studies in live, freely behaving mice. Key to solving that problem is a mouse model developed in the laboratory of CSHL Professor Z. Josh Huang. The mouse has a genetic modification that allows investigators to target a specific population of neurons with any protein of interest.

Kepecs’ group, led by postdocs Duda Kvitsiani and Sachin Ranade, used this mouse to label specific neuron types in the ACC with a light-activated protein – a technique known as optogenetic tagging. Whenever they shone light onto the brains of the mice they were recording from, only the tagged PV and SOM neurons responded promptly with a ‘spike’ in their activity, enabling the researchers to pick them out from the vast diversity of cellular responses seen at any given moment.

The team recorded neural activity in the ACC of these mice while they engaged in foraging behavior. They discovered that the PV and SOM inhibitory neurons responded around the time of the foraging decisions — in other words whether to stay and drink or go and explore elsewhere. Specifically, when the mice entered an area where they could collect a water reward, SOM inhibitory neurons shut down and entered a period of low-level activity, thereby opening a ‘gate’ for information to flow in to ACC. When the mice decided to leave that area and look elsewhere, PV inhibitory neurons fired and abruptly reset cell activity.

“The brain is complex and continuously active, so it makes sense that these two types of inhibitory interneurons define the boundaries of a behavior such as foraging, opening and then closing the ‘gate’ within a particular neural circuit through changes in their activity,” says Kepecs.

This is an important advance, addressing a problem in behavioral neuroscience that scientists call “the cortical response zoo.” When researchers record neural activity in cortex during behavior, and they don’t know which type of neurons they are recording from, a bewildering array of responses is seen. This greatly complicates the task of interpretation. Hence the significance of the Kepecs team’s results, for the first time showing that specific cortical neuron types can be linked to specific aspects of behavior.

“We think about the brain and behavior in terms of levels; what the cell types are and the circuits or networks they form; which regions of the brain they are in; and what behavior is modulated by them,” explains Kepecs. “By observing that the activity of specific cell types in the prefrontal cortex is correlated with a behavioral period, we have identified a link between these levels.”

Brain scans predict which criminals are more likely to reoffend

In a twist that evokes the dystopian science fiction of writer Philip K. Dick, neuroscientists have found a way to predict whether convicted felons are likely to commit crimes again from looking at their brain scans. Convicts showing low activity in a brain region associated with decision-making and action are more likely to be arrested again, and sooner.

Kent Kiehl, a neuroscientist at the non-profit Mind Research Network in Albuquerque, New Mexico, and his collaborators studied a group of 96 male prisoners just before their release. The researchers used functional magnetic resonance imaging (fMRI) to scan the prisoners’ brains during computer tasks in which subjects had to make quick decisions and inhibit impulsive reactions.

The scans focused on activity in a section of the anterior cingulate cortex (ACC), a small region in the front of the brain involved in motor control and executive functioning. The researchers then followed the ex-convicts for four years to see how they fared.

Among the subjects of the study, men who had lower ACC activity during the quick-decision tasks were more likely to be arrested again after getting out of prison, even after the researchers accounted for other risk factors such as age, drug and alcohol abuse and psychopathic traits. Men who were in the lower half of the ACC activity ranking had a 2.6-fold higher rate of rearrest for all crimes and a 4.3-fold higher rate for nonviolent crimes. The results are published in the Proceedings of the National Academy of Sciences.

There is growing interest in using neuroimaging to predict specific behaviour, says Tor Wager, a neuroscientist at the University of Colorado in Boulder. He says that studies such as this one, which tie brain imaging to concrete clinical outcomes, “provide a new and so far very promising way” to find patterns of brain activity that have broader implications for society.

But the authors themselves stress that much more work is needed to prove that the technique is reliable and consistent, and that it is likely to flag only the truly high-risk felons and leave the low-risk ones alone. “This isn’t ready for prime time,” says Kiehl.

Wager adds that the part of the ACC examined in this study “is one of the most frequently activated areas in the human brain across all kinds of tasks and psychological states”. Low ACC activity could have a variety of causes — impulsivity, caffeine use, vascular health, low motivation or better neural efficiency — and not all of these are necessarily related to criminal behaviour.

Crime prediction was the subject of Dick’s 1956 short story “The Minority Report” (adapted for the silver screen by Steven Spielberg in 2002), which highlighted the thorny ethics of arresting people for crimes they had yet to commit.

Brain scans are of course a far cry from the clairvoyants featured in that science-fiction story. But even if the science turns out to be reliable, the legal and social implications remain to be explored, the authors warn. Perhaps the most appropriate use for neurobiological markers would be for helping to make low-stakes decisions, such as which rehabilitation treatment to assign a prisoner, rather than high-stakes ones such as sentencing or releasing on parole.

“A treatment of [these clinical neuroimaging studies] that is either too glibly enthusiastic or over-critical,” Wager says, “will be damaging for this emerging science in the long run.”