Charles Darwin’s cousin, anthropologist Francis Galton, was interested in the most notable faculties of the human mind. In 1883, he noticed that some people visualized the speech of the person they were talking with in their internal mental space:
“Some few people see mentally in print every word that is uttered […], and they read them off usually as from a long imaginary strip of paper, such as is unwound from telegraphic instruments.”
The ticker tape no longer exists, but this peculiarity, known as tickertape synesthesia (TTS), does.
Fabien Hauw and Laurent Cohen, neurologists at the Paris Brain Institute, investigate people who exhibit this extraordinary trait. They hope to learn how we connect sounds, words, letters, and their meanings by doing so.
Floating Subtitles
Synesthesia is a neurological condition in which multiple senses are stimulated at the same time while processing sensory information. Some artists can hear colors, see sounds, and even taste music thanks to it.
Some synesthetes can read someone’s voice in the case of tickertape synesthesia. Catchy radio tunes, sensational news statements, a friend’s assurances, a dog barking… These sounds will all appear to them as if they were subtitles floating in front of their eyes.
“This phenomenon goes far beyond the anecdote. It opens a window to understand better the mechanisms at work in processing written language and their neural bases,”
explained Hauw, a doctoral student at Paris Brain Institute.
Exaggerated Phonology Spelling
B. Summary of non-speech sounds triggering TTS based on the participants’ introspection. The participants are sorted in the same order as in A. Each horizontal color line represents one trigger type. Triggers are sorted by decreasing number of concerned participants. Almost all participants reported TTS when listening to novel words, while only 5/26 (19%) when listening to music without lyrics. Percent values on the right represent the proportion of participants in whom TTS was elicited by the corresponding trigger.
Credit: Cortex (2022). DOI: 10.1016/j.cortex.2022.11.005
Hauw and Cohen, who co-leads the Neuropsychology and Functional Neuroimaging team, interviewed 26 people with TSS. Their goal? Determine under what circumstances subtitles appeared, in what form, and whether they could be an advantage or a disadvantage in daily life.
“We think that TTS occurs when the translation of phonemes into graphemes, i.e., sounds into letters, is too efficient. In these people, the connection between the mental representations of phonology and spelling is exaggerated, and the reading mechanism is somehow ‘forced’ as soon as they are exposed to vocal sounds,”
explains Laurent Cohen.
Despite the fact that this trait was described more than a century ago, it remains poorly understood.
Exciting But Invisible
According to a previous study, up to 1.4% of the population may experience involuntary subtitles when hearing a human voice, but this figure remains unknown. Tickertape synesthesia subjects are typically unaware that they are distinct from the average person, making detection difficult.
“Several participants in the study were shocked to learn that not everyone has built-in subtitles. We taught them through our recruitment ad. Like most forms of synesthesia, TTS is a subjective phenomenon. We don’t know how to measure it objectively, as we would evaluate visual acuity, for example. Nor is it associated with outstanding abilities or incapacitating cognitive impairments. This makes it an exciting but invisible quality,”
said Laurent Cohen.
To better comprehend the subjective experience of tickertape synesthesia, Hauw and Cohen asked 26 French-speaking participants to complete a questionnaire.
More than half of the participants reported having other forms of synesthesia, such as Grapheme-color, Space-time, Sound–color, and Auditive-gustatory synesthesia.
There were many unanswered questions, such as whether tickertape synesthesia could be triggered by a foreign language, new words, or sounds (sneezing, meowing, motor noises). Are the subtitles formatted differently for each individual (size, color, capitalization, special characters)?
Tickertape Synesthesia Advantages And Disadvantages
The results of the study indicate that for 73% of the participants, synesthesia emerged during childhood while they were learning to read.
This trait proved to be both an advantage and a disadvantage for nearly half of the participants; it helps them memorize words but distracts them in crowded places with multiple conversations. In fact, for 70% of the participants, TTS is an uncontrollable automatic process.
Some synesthetes report that the appearance of the captions can be affected by the context of verbalization, particularly when the speaker is emotionally charged; the words may then change color or size based on the intensity of the emotion.
“Letters may be blurred or shaking when I’m moved,”
said one participant.
Surprisingly, some subjects report that, when watching a foreign film, a second layer of subtitles — the result of their synesthesia — appears above the subtitles embedded in the video. Others have dreams and nightmares with subtitles, giving their oneiric activity a cinematic dimension.
Lastly, since one-third of the participants knew of other TTS cases in their families, the emergence of this type of synesthesia may have a genetic basis.
Perception Spectrum Mapped
Specific mental processes allow literate people to interpret and assign meaning to words, sounds, and letters. During development, these reading processes are fine-tuned for efficiency in response to genetic and environmental constraints.
Consequently, there is a wide range of reading performance among individuals, from dyslexia to synesthesia. In this regard, researchers believe that TTS is the result of an extremely atypical development of literacy.
“In a previous MRI study, we showed that when a synesthete listens to a monologue, certain areas of the left hemisphere are activated more strongly than in a control subject, notably, regions responsible for speech analysis and a specific area involved in spelling, the VFWA (Visual Word Form Area). These areas are identical to those related to reading,”
Hauw said.
The findings support the theory that tickertape synesthesia is a type of upended reading in which people automatically convert sounds into written words rather than simply translating written words into sounds. The findings must be replicated with a larger and more diverse sample of subjects.
“Thanks to this study, we can map the spectrum of perceptions that exist in TTS. Now, we want to ensure that it is related to an overdeveloped access to orthographic representations,”
said Cohen.
We may never know if ancient Egypt’s synesthetic scribes subtitled their interlocutors in hieroglyphics. However, by shedding light on the mechanisms of reading, we may be able to assist children who are having difficulty learning to read.
Reference:
Hauw F, El Soudany M, Cohen L. Subtitled speech: Phenomenology of tickertape synesthesia, Cortex (2022). DOI: 10.1016/j.cortex.2022.11.005