Strategic or Random? How the Brain Chooses

Many of the choices we make are informed by experiences we’ve had in the past. But occasionally we’re better off abandoning those lessons and exploring a new situation unfettered by past experiences. Scientists at the Howard Hughes Medical Institute’s Janelia Research Campus have shown that the brain can temporarily disconnect information about past experience from decision-making circuits, thereby triggering random behavior.

In the study, rats playing a game for a food reward usually acted strategically, but switched to random behavior when they confronted a particularly unpredictable and hard-to-beat competitor. The animals sometimes got stuck in a random-behavior mode, but the researchers, led by Janelia lab head Alla Karpova and postdoctoral fellow Gowan Tervo, found that they could restore normal behavior by manipulating activity in a specific region of the brain. Because the behavior of animals stuck in this random mode bears some resemblance to that of patients affected by a psychological condition called learned helplessness, the findings may help explain that condition and suggest strategies for treating it. Karpova, Tervo and their colleagues published their findings in the September 25, 2012, issue of the journal Cell.

The brain excels at integrating information from past experiences to guide decision-making in new situations. But in certain circumstances, random behavior may be preferable. An animal might have the best chance of avoiding a predator if it moves unpredictably, for example. And in a new environment, unrestricted exploration might make more sense than relying on an internal model developed elsewhere. So scientists have long speculated that the brain may have a way to switch off the influence of past experiences so that behavior can proceed randomly, Karpova says. But others disagreed. “They argue that it’s inefficient, and that it would be at odds with what some people call one of the most central operating principles of the brain – to use our past experience and knowledge to optimize behavioral choices,” she notes.

Karpova and her colleagues wanted to see if they could create a situation that would force animals to switch into this random mode of behavior. “We tried to create a setting that would push the need to create behavioral variability and unpredictability to its extreme,” she says. They did this by placing rats in a competitive setting in which a computer-simulated competitor determined which of two holes in a wall would provide a sugary reward. The virtual competitor, whose sophistication was varied by the experimenters, analyzed the rats’ behavior to predict their future choices.

“We thought if we came up with very sophisticated competitors, then the animals would eventually be unable to figure out how to outcompete them, and be forced to either give up or switch into this [random] mode, if such a mode exists,” Karpova says. And that’s exactly what happened: When faced with a weak competitor, the animals made strategic choices based on the outcomes of previous trials. But when a sophisticated competitor made strong predictions, the rats ignored past experience and made random selections in search of a reward.

Now that they had evidence that the brain could generate both strategic and random behavior, Karpova and her colleagues wanted to know how it switched between modes. Since that switch determines whether or not an animal’s internal model of the world influences its behavior, the scientists suspected it might involve a brain region called the anterior cingulate cortex, where that internal model is likely encoded.

They found that they could cause animals to switch between random and strategic behavior by manipulating the level of a stress hormone called norepinephrine in the anterior cingulate cortex. Increasing norepinephrine in the region activated random behavior and suppressed the strategic mode.  Inhibiting release of the hormone had the opposite effect.

Karpova’s team observed that animals in their experiments sometimes continued to behave randomly, even when such behavior was no longer advantageous. “If all they’ve experienced is this really sophisticated competitor for several sessions that thwarts their attempts at strategic, model-based counter-prediction, they go into this [random mode], and they can get stuck in it for quite some time after that competitor is gone,” she says. This, she says, resembles the condition of learned helplessness, in which strategic decision-making is impaired following an experience in which a person finds they are unable to control their environment.

The scientists could release the animals from this “stuck” state by suppressing the release of norepinephrine in the anterior cingulate cortex. “Just by manipulating a single neuromodulatory input into one brain area, you can dramatically enhance the strategic mode. The effect is strong enough to rescue animals out of the random mode and successfully transform them into strategic decision makers,” Karpova says. “We think this might shed light on what has gone wrong in conditions such as learned helplessness, and possibly how we can help alleviate them.” 

Karpova says that now that her team has uncovered a mechanism that switches the brain between random and strategic behavior, she would like to understand how those behaviors are controlled in more natural settings. “We normally try to use all of our knowledge to think strategically, but sometimes we still need to explore,” she says. In most cases, that probably means brief bouts of random behavior during times when we are uncertain that past experience is relevant, followed by a return to more strategic behavior – a more subtle balance that Karpova intends to investigate at the level of changes in activity in individual neural circuits.

(Image caption: Researchers at Cold Spring Harbor Laboratory have identified the neurons in the brain that determine if a mouse will learn to cope with stress or become depressed. These neurons, located in a region of the brain known as the medial prefrontal cortex (green, left image), become hyperactive in depressed mice (right panel is close-up of left, yellow indicates activation). The team showed that this enhanced activity in fact causes depression.)

Dealing with stress – to cope or to quit?

We all deal with stress differently. For many of us, stress is a great motivator, spurring a renewed sense of vigor to solve life’s problems. But for others, stress triggers depression. We become overwhelmed, paralyzed by hopelessness and defeat. Up to 20% of us will struggle with depression at some point in life, and researchers are actively working to understand how and why this debilitating mental disease develops.

Today, a team of researchers at Cold Spring Harbor Laboratory (CSHL) led by Associate Professor Bo Li reveals a major insight into the neuronal basis of depression. They have identified the group of neurons in the brain that determines how a mouse responds to stress — whether with resilience or defeat.

For years, scientists have relied on brain imaging to look for neuronal changes during depression. They found that a region of the brain known as the medial prefrontal cortex (mPFC) becomes hyperactive in depressed people. This area of the brain is well known to play a role in the control of emotions and behavior, linking our feelings with our actions. But brain scans aren’t able to determine if increased activity in the mPFC causes depression, or if it is simply a byproduct of other neuronal changes. 

Dr. Li set out to identify the neuronal changes that underlie depression. In work published today in The Journal of Neuroscience,Li and his team, including Minghui Wang, Ph.D. and Zinaida Perova, Ph.D., used a mouse model for depression, known as “learned helplessness.” They combined this with a genetic trick to mark specific neurons that respond to stress. They discovered that neurons in the mPFC become highly excited in mice that are depressed. These same neurons are weakened in mice that aren’t deterred by stress – what scientists call resilient mice.

But the team still couldn’t be sure that enhanced signaling in the mPFC actually caused depression. To test this, they engineered mice to mimic the neuronal conditions they found in depressed mice. “We artificially enhanced the activity of these neurons using a powerful method known as chemical genetics,” says Li. “The results were remarkable: once-strong and resilient mice became helpless, showing all of the classic signs of depression.”

These results help explain how one promising new treatment for depression works and may lead to improvements in the treatment.

Doctors have had some success with deep brain stimulation (DBS), which suppresses the activity of neurons in a very specific portion of the brain. “We hope that our work will make DBS even more targeted and powerful,” says Li, “and we are working to develop additional strategies based upon the activity of the mPFC to treat depression.”

Next, Li is looking forward to exploring how the neurons in the mPFC become hyperactive in helpless mice. “These active neurons are surrounded by inhibitory neurons,” says Li. “Are the inhibitory neurons failing? Or are the active neurons somehow able to bypass their controls? These are some of the many open questions we are pursuing to understand the how depression develops.”

Learned helplessness in flies and the roots of depression

When faced with impossible circumstances beyond their control, animals, including humans, often hunker down as they develop sleep or eating disorders, ulcers, and other physical manifestations of depression. Now, researchers reporting in the Cell Press journal Current Biology on April 18 show that the same kind of thing happens to flies.

The study is a step toward understanding the biological basis for depression and presents a new way for testing antidepressant drugs, the researchers say. The discovery of such symptoms in an insect shows that the roots of depression are very deep indeed.

"Depressions are so devastating because they go back to such a basic property of behavior," says Martin Heisenberg of the Rudolf Virchow Center in Würzburg, Germany.

Heisenberg says that the idea for the study came out of a lengthy discussion with a colleague about how to ask whether flies can feel fear. Franco Bertolucci, a coauthor on the study, had found that flies can rapidly learn to suppress innate behaviors, a phenomenon that is part of learned helplessness.

The researchers now show that flies experiencing uncomfortable levels of heat will walk to escape it. But if the flies realize that the heat is beyond their control and can’t be avoided, they will stop responding, walking more slowly and taking longer and more frequent rests, as if they were “depressed.”

Intriguingly, female flies slow down more under those stressful circumstances than males do. It’s not clear exactly what that means, but Heisenberg explains, “if we realize that the fly trapped in a strange, dark box, unable to get rid of the dangerous heat pulses, has to find a compromise between saving energy and not missing any chance of escape, we can understand that such a compromise may come out differently for males and females, as their resources and goals in life are different.”

Heisenberg’s team now intends to explore other questions, such as: How long does the flies’ depression-like state last? How does it affect other behaviors, like courtship and aggression? What is happening in their brain? And more.

Heisenberg says that the findings are a reminder of a lesson that children’s books are often best at showing: “Animals have lots in common with us humans. They breathe the same air, share many of the same resources, actively explore space, and have distinct social roles. Their brains serve the same purpose, too: they help them to do the right thing.”