How Your Brain Chunks Moments Into Events


Scientists say they have a new explanation for how the brain breaks experiences into “events,” or the related groups that help us mentally organize the day’s many situations.

They propose that the brain may actually work from subconscious mental categories it creates based on how it considers people, objects, and actions are related.

Specifically, these details are sorted by temporal relationship, which means that the brain recognizes that they tend to— or tend not to— pop up near one another at specific times.

Their explanation challenges the dominant concept known as prediction error that says our brain draws a line between the end of one event and the start of another when things take an unexpected turn.

This new concept of “shared temporal context” works very much like the object categories our minds use to organize objects, explains Anna Schapiro, a doctoral student in Princeton’s psychology department and lead author of the study.

“We’re providing an account of how you come to treat a sequence of experiences as a coherent, meaningful event,” Schapiro says. “Events are like object categories. We associate robins and canaries because they share many attributes: They can fly, have feathers, and so on. These associations help us build a ‘bird’ category in our minds.

Events are the same, except the attributes that help us form associations are temporal relationships.”

Coherent Chunks

Supporting this idea is brain activity the researchers captured showing that abstract symbols and patterns with no obvious similarity nonetheless excited overlapping groups of neurons when presented to study participants as a related group.

From this, the researchers constructed a computer model that can predict and outline the neural pathways through which people process situations, and can reveal if those situations are considered part of the same event.

The parallels drawn between event details are based on personal experience, Schapiro says. People need to have an existing understanding of the various factors that, when combined, correlate with a single experience.

“Everyone agrees that ‘having a meeting’ or ‘chopping vegetables’ is a coherent chunk of temporal structure, but it’s actually not so obvious why that is if you’ve never had a meeting or chopped vegetables before,” Schapiro says.

“You have to have experience with the shared temporal structure of the components of the events in order for the event to hold together in your mind,” she adds. “And the way the brain implements this is to learn to use overlapping neural populations to represent components of the same event.”

MRI Images

During a series of experiments, the researchers presented participants with sequences of abstract symbols and patterns. Without the participants’ knowledge, the symbols were grouped into three “communities” of five symbols with shapes in the same community tending to appear near one another in the sequence.

After watching these sequences for roughly half an hour, participants were asked to segment the sequences into events in a way that felt natural to them. They tended to break the sequences into events that coincided with the communities the researchers had prearranged, which shows that the brain quickly learns the temporal relationships between the symbols, Schapiro says.

The researchers then used functional magnetic resonance imaging to observe brain activity as participants viewed the symbol sequences. Images in the same community produced similar activity in neuron groups at the border of the brain’s frontal and temporal lobes, a region involved in processing meaning.

The researchers interpreted this activity as the brain associating the images with one another, and therefore as one event. At the same time, different neural groups activated when a symbol from a different community appeared, which was interpreted as a new event.

Deeper Understanding

They fashioned these data into a computational neural-network model that revealed the neural connection between what is being experienced and what has been learned.

When a simulated stimulus is entered, the model can predict the next burst of neural activity throughout the network, from first observation to processing.

“The model allows us to articulate an explicit hypothesis about what kind of learning may be going on in the brain,” Schapiro said. “It’s one thing to show a neural response and say that the brain must have changed to arrive at that state. To have a specific idea of how that change may have occurred could allow a deeper understanding of the mechanisms involved.”

Michael Frank, a Brown University associate professor of cognitive, linguistic and psychological sciences, says that the Princeton researchers uniquely apply existing concepts of “similarity structure” used in such fields as semantics and artificial intelligence to provide evidence for their account of event perception.

These concepts pertain to the ability to identify within large groups of data those subsets that share specific commonalities, says Frank, who is familiar with the research but had no role in it.

“The work capitalizes on well-grounded computational models of similarity structure and applies it to understanding how events and their boundaries are detected and represented,” Frank says. “The authors noticed that the ability to represent items within an event as similar to each other—and thus different than those in ensuing events — might rely on similar machinery as that applied to detect clustering in community structures.”

The model “naturally” lays out the process of shared temporal context in a way that is validated by work in other fields, yet distinct in relation to event perception, Frank adds.

“The same types of models have been applied to understanding language—for example, how the meaning of words in a sentence can be contextualized by earlier words or concepts,” Frank says. “Thus the model and experiments identify a common and previously unappreciated mechanism that can be applied to both language and event parsing, which are otherwise seemingly unrelated domains.”

Anna C Schapiro, Timothy T Rogers, Natalia I Cordova, Nicholas B Turk-Browne & Matthew M Botvinick
Neural representations of events arise from temporal community structure
Nature Neuroscience 16, 486–492 (2013) doi:10.1038/nn.3331