New research shows that light can be used to detect what is going on inside someone’s head and identify what image they are seeing.
The study1 demonstrates that high-density diffuse optical tomography (HD-DOT) - a noninvasive, wearable, light-based brain imaging technology - is sufficiently sensitive and precise to be potentially useful in applications such as augmented communication that are not well suited to other imaging methods.
MRI could be used for decoding, but it requires a scanner, and you can’t expect someone to go lie in a scanner every time they want to communicate. With this optical method, users would be able to sit in a chair, put on a cap, and potentially use this technology to communicate with people. We’re not quite there yet, but we’re making progress. What we’ve shown in this paper is that, using optical tomography, we can decode some brain signals with an accuracy above 90%, which is very promising,
said senior author Joseph P. Culver, professor of radiology at Washington University’s Mallinckrodt Institute of Radiology.
Some individuals are trapped within their own minds, able to think and feel but unable to express themselves because brain injury or disease has damaged their lines of communication with the outside world. The findings could be helpful in opening up communications for these people.
When neuron activity increases in any region of the brain, oxygenated blood rushes in to fuel the activity. HD-DOT uses light to detect the rush of blood.
Participants wear a cap fitted with dozens of fibers that relay light from tiny LEDs to the head. After the light is transmitted through the head, detectors capture dynamic changes in the colors of the brain tissue as a result of changes in blood flow.
Culver, along with first author and graduate student Kalyan Tripathy, and colleagues set out to evaluate the potential of HD-DOT for decoding brain signals. They started with the visual system because it is one of the best understood brain functions.
Neuroscientists long ago worked out a detailed map of the visual part of the brain by showing participants flashing checkerboard patterns on a screen and identifying the 3D units, known as voxels, in the brain that became active in response to each pattern. Decoding is the attempt to reverse the process: Detecting active voxels and then deducing which checkerboard pattern triggered that pattern of brain activity.
We know what the participant is seeing, so we can verify how well our decoding matches up to reality. By going to something that was well validated, we could optimize the experimental design, push harder on the statistics of the decoding, and obtain performance that is really very high,
The researchers started simple. They recruited five participants for multiple five- to 10-minute runs in which the participants were shown a checkerboard pattern on either the left or the right side of the visual field for a few seconds at a time, interspersed with breaks during which no image was shown.
Using one run as the template, the researchers analyzed the data from another run to determine when the checkerboard was on which side of the screen. They repeated this analysis using different runs as the template and the test until they had analyzed all possible pairings.
A First Step
The researchers were able to identify the correct position of the checkerboard — left, right, or not visible at all — with 75% to 98% accuracy. While decoding was more successful when the same person was used for the template run and the test run, the patterns from one person could be used to decode the brain activity of another person.
Then, the researchers made the problem more complex. They showed participants a checkerboard wedge that rotated at 10 degrees a second. Three participants sat for six seven-minute runs on two separate days. Using the same template and test run strategy, the researchers were able to pinpoint the position of the wedge within 26 degrees.
The results are a first step toward the ultimate goal of facilitating communication for people who struggle to express themselves because of cerebral palsy, stroke, or other conditions that result in locked-in syndrome.
Tripathy K, Markow ZE, Fishell AK, Sherafati A, Burns-Yocum TM, Schroeder ML, Svoboda AM, Eggebrecht AT, Anastasio MA, Schlaggar BL, Culver JP. Decoding visual information from high-density diffuse optical tomography neuroimaging data. NeuroImage. Oct. 31, 2020. DOI: 10.1016/j.neuroimage.2020.117516 ↩︎