May 25, 2012 by Stuart Mason Dambrot
(Medical Xpress) — Regardless of an organism’s biological complexity, every encephalized animal continuously makes under-informed behavioral choices that can have serious consequences. Despite its ubiquity, however, there’s a long-standing question about its neurological basis – namely, whether these choices are made through probabilistic world models constructed by the brain, or by reinforcement of learned associations. Recently, however, scientists in the Department of Psychology at Rutgers University found that reinforcement cannot account for the rapidity with which mice modify their behavior when the chance of a given phenomenon changes. The researchers say this indicates that mice may have primordially-evolved neural capabilities to represent likelihood and perform calculations that optimize their resulting behavior – and therefore that such genetic mechanisms can be investigated and manipulated by genetic and other procedures.

The experimental environment. In the switch task, a trial proceeds as follows: 1: Light in the Trial-Initiation Hopper signals that the mouse may initiate a trial. 2: The mouse approaches and pokes into the trial-initiation hopper, extinguishing the light there and turning on the lights in the two feeding hoppers (trial onset). 3: The mouse goes to the short-latency hopper and pokes into it. 4: If, after 3 s have elapsed since the trial onset, poking in the short-latency hopper does not deliver a pellet, the mouse switches to the long-latency hopper, where it gets a pellet there in response to the first poke at or after 9 s since the trial onset. Lights in both feeding hoppers extinguish either at pellet delivery or when an erroneously timed poke occurs. Short trials last about 3 s and long trials about 9 s, whether reinforced or not: if the mouse is poking in the short hopper at the end of a 3-s trial, it gets a pellet and the trial ends; if it is poking in the 9-s hopper, it does not get a pellet and the trial ends at 3 s. Similarly, long trials end at 9 s: if the mouse is poking in the 9-s hopper, it gets a pellet; if in the 3-s hopper, it does not. A switch latency is the latency of the last poke in the short hopper before the mouse switches to the long hopper. Only the switch latencies from long trials are analyzed. Copyright © PNAS, doi: 10.1073/pnas.1205131109
In conducting their research, Prof. Randy Gallistel and doctoral student Aaron Kheifets had to first address a key challenge in identifying estimates of stochastic parameters versus reinforcement-driven processes as the behavior-optimizing mechanism in the laboratory mice studied (the c57bl/6j strain of Mus musculus, the common house mouse, from Jackson Labs). “Because both processes can lead to approximately optimal behavior in the long run,” Gallistel tells Medical Xpress, “one has to focus on the short run – that is, on the course of the transition in behavior. The problem in this case is that the transition is a change in the distribution of switch latencies.” A distribution of switch latencies is composed of a great many temporal discriminations on the part of the subject observed over a long sequence of trials, so this distribution can be used to prove that the process generating the distribution changed abruptly.
“Fortunately,” Gallistel continues, “it was obvious from simple inspection of the raw data that there was an abrupt change. The challenge was to develop a mathematical analysis that confirmed this. Meeting this challenge required the use of Bayesian methods, which are just now beginning to be applied to behavioral data. In addition, we had to develop analyses showing that differential reinforcement could not explain the transition.” The team therefore applied Bayesian methods of analysis to the determination of the parameters of a transition function for a 4-parameter mixture distribution.
“Also,” Gallistel adds, “a graphical means of displaying the raw data in such a way as to make the basic phenomenon visually apparent was required. To this end, we devised a figure with a huge number of bits per square centimeter – that is, it shows an enormous amount of readily graspable information in a small space.”


























