Decisions and their neural representations in habitats with relative diminishing returns

Junior Samuel Lopez Yepez

Research output: Book/anthology/dissertation/reportPh.D. thesis

Abstract

Making appropriate decisions is crucial to maximize the amount of valuable resources (rewards) and/or minimize potential hazards in uncertain habitats. However, this is not straightforward when previous decisions influence how the environment provides resources. In this thesis, I explore how recent rewards and choices affect the behavior of different species in uncertain habitats. I analyze how mice, humans and virtual agents solve a two-option task, where each of the alternatives delivers rewards in a probabilistic manner. In this task, the longer an agent (e.g., a mouse) exploits an option, the less valuable that option becomes (i.e., the probability of reward decreases) with respect to the value of the alternative option. With this paradigm of behavior, I find that there is a common trend in the way different species integrate their recent history (i.e., past rewards and choices) to produce behavior. I developed a new reinforcement-learning model (RL), the double-trace model (DT), to capture this general trend and evaluate this strategy in similar problems under different virtual conditions. Moreover, I could show that the reward statistics of the available options affects the way in which animals integrate their recent history, suggesting a robust and adaptive strategy evolved to optimize resource utilization in different habitats. Furthermore, I explore how the mouse brain represents recent events by analyzing the activity of single cortical neurons. I discovered that recent rewards and choices are represented by neurons in relative and discrete time frames, which implies the function of cortical cells as ``temporal filters" for past behavioral events. In addition, I could show that these cortical neural representations are behaviourally relevant because their ensemble information is predictive of upcoming choices. Cortical neurons receive dense axonal projections from dopamine (DA) neurons. To test how recent events affect the DA population activity, I built a custom-made setup to collect fluorescent signals (photometer) from calcium influx indicators (e.g., when an action potential occurs) expressed in brains of transgenic mice. I noted that DA neurons simultaneously represent sensory (rewards) and motor (choices) history, such as the so-called reward prediction error (RPE), reward expectations and choice traces. In summary, the research findings contributes to understand the importance of recent rewards and choices to maximize valuable resources and to have a new perspective in how cortical and dopamine neurons represent this information.
Original languageEnglish
PublisherÅrhus Universitet
Number of pages144
Publication statusPublished - 18 Nov 2019

Keywords

  • reinforcement learning
  • neural representation
  • behaviour analysis
  • photometry
  • fiber fluorometry
  • dopamine
  • prefrontal cortex
  • ventral tegmental area
  • mouse
  • decision making
  • human
  • probabilistic task
  • rewards
  • choice
  • prediction error

Fingerprint

Dive into the research topics of 'Decisions and their neural representations in habitats with relative diminishing returns'. Together they form a unique fingerprint.

Cite this