University of Southern California computer scientists and psychologists have teamed up to investigate, with the help of artificial intelligence, how music affects how you act, feel, and think.
Your heart beats faster, palms sweat, and part of your brain called the Heschl’s gyrus lights up like a Christmas tree. Chances are, you’ve never thought about what happens to your mind and body when you listen to music in such a detailed way.
But it’s a question that has puzzled scientists for decades: Why does something as abstract as music provoke such a consistent response?
Taking A Holistic View
The research team looked at heart rate, galvanic skin response (or sweat gland activity), brain activity, and subjective feelings of happiness and sadness in a group of volunteers as they listened to three pieces of unfamiliar music.
“Taking a holistic view of music perception, using all different kinds of musical predictors, gives us an unprecedented insight into how our bodies and brains respond to music,“
said the study’s lead author Tim Greer, a computer science Ph.D. student and a member of the USC Signal Analysis and Interpretation Laboratory (SAIL).
Contrast Is Crucial
Amongst their findings, the researchers noted that music powerfully influenced parts of the brain in the auditory complex called the Heschls’ gyrus and the superior temporal gyrus (plural: gyri).
The brain responded to pulse clarity, or the strength of the beat (put simply: your gyri will be looking lively when listening to Lady Gaga’s “Bad Romance”).
They also found that changing dynamics, rhythm, and timbre, or the introduction of new instruments, brings about the uptick in response. In other words, contrast is crucial. For instance, the gyri activate when there is a change in dynamics, or “loudness.”
“If a song is loud throughout, there’s not a lot of dynamic variability, and the experience will not be as powerful as if the composer uses a change in loudness,“
said Greer, himself a composer who plays sax and keyboard.
“It’s the songwriter’s job to take you on a rollercoaster of emotions in under three minutes, and dynamic variability is one of the ways this is achieved.“
So, if you’re listening to a whole album of black metal, which is consistently loud, you’re probably not going to see a response. But if you’re listening to Nirvana’s “Smells Like Teen Spirit,” which goes from a quiet verse to a loud chorus and back again, it’s a different story.
The team also discovered that galvanic skin response — basically, a measure of sweat — increased after the entrance of a new instrument or start of a musical crescendo.
“When each new instrument enters, you can see a spike in the collective response of the skin,“
Also, the most exciting moments in music were preceded by an increase in the complexity level of the song. In essence, the more instruments are in the song, the more people responded. (Think: the first section of Mike Oldfield’s “Tubular Bells,” as the song builds to a crescendo by adding more instruments.)
And the saddest note of all? That award goes to the raised 7th note of the minor scale. The study found the note F# in a song in the G minor key positively correlated with high sadness ratings.
That might be the reason the narrator’s anguish is almost palpable in The Animals’ “House of the Rising Sun,” which uses the raised 7th of the minor scale to launch into each increasingly emotional verse.
For this experiment, the team selected three emotional pieces of music that did not contain lyrics and were not highly familiar, so no element of memory was attached to the listeners’ emotional response. (Hearing a song that played in the background during a wisdom tooth extraction, for instance, might skew your perception.)
In the neuroimaging experiment, 40 volunteers listened to a series of sad or happy musical excerpts, while their brains were scanned using MRI. This was conducted at USC’s Brain and Creativity Institute by Assal Habibi, an assistant professor of psychology at USC Dornsife College of Letters, Arts, and Sciences, and her team, including Matthew Sachs, a postdoctoral scholar currently at Columbia University.
60 people listened to music on headphones, while their heart activity and skin conductance were measured. The same group also rated the intensity of emotion (happy or sad) from 1 to 10 while listening to music.
Then, the computer scientists crunched the data using AI algorithms to determine which auditory features people responded to consistently.
In the past, neuroscientists trying to understand the impact of music on the body, brain, and emotions better, have analyzed MRI brain scans over tiny segments of time — for instance, looking at the brain reacting to two seconds of music.
By contrast, in this study, using algorithms to analyze data gathered in the lab, the scientists were able to look at how people felt while listening to music over long periods, not only from brain scans but also combining data from other modes.
“Novel multimodal computing approaches help not just illuminate human affective experiences to music at the brain and body level, but in connecting them to how actually individuals feel and articulate their experiences,“
said Professor Shrikanth (Shri) Narayanan, the study co-author.
In addition to helping researchers identify songs for the perfect workout, study, or sleep playlist, the research has therapeutic applications — music has been shown to calm anxiety, ease pain and help people with disabilities or dementia.
“From a therapy perspective, music is a really good tool to induce emotion and engage a better mood,“
“Using this research, we can design musical stimuli for therapy in depression and other mood disorders. It also helps us understand how emotions are processed in the brain.“
According to the researchers, future studies could look at how different types of music can positively manipulate our emotional responses and whether the intent of the composer matches the listener’s perception of a piece of music.
Timothy Greer, et al. A Multimodal View into Music’s Effect on Human Neural, Physiological, and Emotional Experience. Proceedings of the 27th ACM International Conference on Multimedia Pages 167-175