In a pair of recently published papers, researchers at Carnegie Mellon University, along with local and international collaborators, shed light on how specific circuits in the brain can simultaneously make decisions and learn from their outcomes.
Consider eating brunch at your favorite restaurant: How do you know whether the eggs benedict will be a better choice than the waffles? Usually, you accumulate evidence over time. At first, you randomly pick one, and then you try the other on your next visit. Perhaps you find one varies in quality every time you try it or that you consistently prefer the taste of the same dish over the other.
“For decades neuroscience and cognitive science have tried to understand how we make decisions and how we learn from their consequences, but they have done so independently,” said Timothy Verstynen, an associate professor in CMU's Department of Psychology and the Carnegie Mellon Neuroscience Institute. “We have learned a lot about the what — the brain systems — and the how — the computational algorithms — of adaptive decision-making, but we lacked a bridge that linked the two together.”
Connections linking the cortex, basal ganglia and thalamus (left) form a critical network in the brain of a wide array of species, including humans. Feedback from dopaminergic neurons modulates these connections during learning (upper right), leading to changes in decision making behavior (lower right). Credit: Carnegie Mellon University
In a recent paper published in the journal PLoS Computational Biology, Verstynen and colleagues Kyle Dunovan, a former CMU postdoctoral fellow; Cati Vich Llompart of Universitat de les Illes Baleras; Matthew Clapp of University of South Carolina; and Jonathan Rubin of University of Pittsburgh attempted to build such a theoretical bridge using a series of increasingly complex computational models.
“We started by modeling microscopic synapses of cells in an area of the brain called the basal ganglia,” Dunovan said. “We built models of small sets of neurons that made decisions between two actions. By modeling the dopamine response following whether an action had a good outcome or a bad, we were able to see how dopamine shapes these synapses over time as they learn.”
The researchers then took this information and built a much larger model of the cortical and subcortical brain networks that regulate both decision-making and learning called the cortico-basal ganglia-thalamic loops. Using large simulation of different brain areas, they altered a few of the critical synapses to reflect different degrees of learning and had the network make a series of selections between two targets. They took this behavior from the simulated brain and analyzed it as if it were a human participant, using a cognitive model that captures the process of how the brain accumulates information during decision-making.
This nested set of models, from individual synapses, to whole brain networks and finally to behavior, allowed the researchers to identify two novel ways that learning impacts the way that the brain makes decisions. When the network was configured in a way such that competition increased between a pathway that promotes behavior and a pathway that suppresses behavior, called the direct and indirect pathways, respectively, the rate at which the simulated brain accumulated information got slower. In contrast, when the network was configured in a way that increased the sensitivity of the suppressive pathway alone, the artificial brain relied on less overall information before making a decision.
“This was a somewhat unexpected discovery,” Dunovan said. “Going into this project, we expected that there should be one mechanism in the circuit that relates to how fast the agent accrues information. But the biological models revealed this second path that changes the criterion for how much information the system needs before making a response. This showed us how the same circuits in the brain can impact our decisions in different ways. It also suggested to us that these circuits might rely on different forms of feedback to alter the different parts of the decision process.”
To this end, Verstynen and Dunovan published a second paper in the Journal of Neuroscience in March that analyzed human behavior in the same way that they analyzed the artificial brain's behavior in the first paper. They tasked research participants with either pressing a button at a specific time or withholding their response when presented with a cue called a “stop-signal.”
Dunovan and Verstynen found that humans rely on different feedback to change different parts of the decision process. For example, hitting a button just a few milliseconds too fast or too slow led to a quick correction of the rate of information accumulation in the brain. In contrast, errors in selecting the correct response, like pressing the button when presented with the stop-signal changed the level of information that the brain needed to make a decision.
“Despite testing a number of possible alternative explanations of the behavior, the results kept coming back to the same two processes that simulated brain networks identified,” Verstynen said. “For a long time, we not only looked at decision-making and learning as two separate problems, but we also studied the neural circuits and cognitive algorithms separately. What we have shown is that the architecture of the circuits in the brain gives us specific clues as to what the algorithms of behavior should look like and, in this case, that clue is telling us not to think of decision-making without thinking of learning as well.”
Source: Carnegie Mellon University