Research

How do we plan in complex environments?

To plan effectively, we need to predict not only one step into the future but the possible long-term outcomes associated with our choices. Doing so requires the brain to construct detailed predictive models of the world.

Understanding both how we build those models, and how and when we rely on them to plan, is fundamental to understanding cognition, its development, and its disorders. Research on the two questions has been fairly disconnected, despite the fact that they are deeply intertwined: what you want a mental map to represent depends on its purpose, and your ability to plan depends on your choice of mental map.

Our work aims to link them. We draw on network science, control theory, and reinforcement learning, and combine behavior in humans and animals with human neuroimaging.

Question 01

How do we build models of the world?

Human behavior, from infants to adults, is shaped by temporal regularity in sequences such as speech, music, or motor movements. We are sensitive to the transition probabilities between items, and can use those statistics to segment a stream into its parts. But to plan, we need predictions that reach further than the next event, and chaining single-step predictions becomes increasingly costly with planning depth.

We ask how people extract that higher-level structure, starting from the mechanisms that support simple statistical regularities and asking how they can underpin multi-step temporal abstractions. Formalizing sequences as walks on a graph allows us to hold local statistics fixed while varying the global structure that generates them, and examine how people depend on such global structure. Reaction times then serve as an online readout of the internal model, allowing us to examine trial-by-trial predictive learning.

A network diagram of a modular graph, with densely connected clusters of nodes joined by a small number of edges between clusters. — Sequences are walks on a graph, so structure can be varied while pairwise statistics are held fixed.

Brain images showing regions where BOLD activity reflects the predictive structure of a learned sequence rather than the identity of individual stimuli. — Visual cortex representations differ between modular and non-modular graphs.

Where we're going

Learning rules for predictive models

Candidate learning rules behave identically once over-learned — the regime they are usually studied in — but diverge trial by trial. We use reaction times to tell them apart.
State features underlying behavior

State representations are core to planning, but how they are built from experience is unknown. With collaborators, we ask how rats segment space to support planning in navigation.

Question 02

How do we choose the right model to guide decision-making?

The brain is capable of both habitual and goal-directed behavior, typically mapped onto model-free planning, which caches the long-run values of actions, and model-based planning, which simulates outcomes through a world model. The two adaptively trade off. But planning in complex environments requires predictions over temporally extended timescales, where the value of an action may depend on outcomes many minutes away. In such environments, habits are insensitive to changes in distant outcomes, while simulation can be too expensive for longer horizons.

Predictive models such as the successor representation offer a middle path, learning where actions lead separately from what those outcomes are worth, so distant values can change without re-simulating. Substantial evidence demonstrates that people rely on such predictive models in deicison making, and we have found that such models trade off against finer-grained model-based planning. In particular, the brain adaptively chooses finer-grained models only when coarser predictive models are behaviorally insufficient. However, understanding what signals the brain relies for this arbitration remains unknown.

A screenshot of the sailing task, in which participants choose routes between islands laid out as a graph of possible transitions. — A task built to read out successor-representation use trial by trial, rather than from a handful of probes.

Where we're going

Predictive models across development

Adults adapt learning strategies more readily than children. Does the same hold for temporally abstract policies? Early results say children arbitrate much as adults do.
Arbitration between strategies

What signal drives the switch? State prediction errors and on- versus off-policy inconsistency are both plausible, and we are working to dissociate them.
Value representation and planning

How do predictive mechanisms interact with value? In ongoing collaborations we study how values of distant locations are updated and queried during reward-guided behavior.
Disruption in pathological behavior

Adapting planning to environmental statistics is central to healthy behavior. We are asking whether that adaptation differs in patient populations.

Also ongoing

Network structure and brain dynamics

The network science underpinning our work on graph learning applies equally to the brain’s own architecture. Diffusion imaging lets us treat white matter connectivity as a network, and ask what dynamics that network can support: how controllable it is, how easily activity can be steered between states, and how those properties change over development.

We are particularly interested in measures of indirect connectivity, such as communicability, since the regions that cooperate during learning are often not directly connected. This work runs alongside the questions above rather than apart from them — the same tools that characterize a learned graph also characterize the structural graph doing the learning.

A diffusion MRI reconstruction of white matter tracts, with streamlines coloured by their orientation through the brain.

Interested in this work?

The lab is recruiting graduate students, postdocs, and Arizona undergraduates.

How to join