Brain–machine interfaces (BMIs) can read brain activity and transform that activity to control an electronic device like a prosthetic arm or computer cursor. They have the potential to let people with paralysis move prosthetic devices with their thoughts.
Many BMIs necessitate invasive brain procedures for electrode implantation in order to monitor neural activity. However, in 2021, researchers from Caltech devised a method to interpret brain activity via functional ultrasound (fUS), a significantly less invasive modality.
A new study shows that fUS technology can be used to create a “online” BMI. This kind of BMI reads brain activity, figures out what it means using decoders programmed with machine learning, and then commands a computer that can accurately guess movement with very little delay time.
Functional Ultrasound Advantages
Richard Andersen, James G. Boswell Professor of Neuroscience and director and leadership chair of the T&C Chen Brain–Machine Interface Center; and Mikhail Shapiro, Max Delbrück Professor of Chemical Engineering and Medical Engineering and Howard Hughes Medical Institute Investigator, oversaw the research at Caltech. Mickael Tanter, director of physics for medicine at INSERM in Paris, France, oversaw the laboratory where this research was conducted in collaboration.
“Functional ultrasound is a completely new modality to add to the toolbox of brain–machine interfaces that can assist people with paralysis. It offers attractive options of being less invasive than brain implants and does not require constant recalibration. This technology was developed as a truly collaborative effort that could not be accomplished by one lab alone,”
said Andersen.
“In general, all tools for measuring brain activity have benefits and drawbacks,”
co-first author Sumner Norman, former senior postdoctoral scholar research associate at Caltech, said.
High-frequency Sound Pulses
Electrodes, despite their ability to measure the activity of individual neurons with extreme precision, are difficult to scale beyond a few small brain regions and require implantation within the brain. Non-invasive methods are also not without their drawbacks.
“Functional magnetic resonance imaging [fMRI] provides whole-brain access but is restricted by limited sensitivity and resolution. Portable methods, like electroencephalography [EEG] are hampered by poor signal quality and an inability to localize deep brain function,”
Sumner Norman explained.
Ultrasound imaging works by emitting pulses of high-frequency sound and measuring how those sound vibrations echo throughout a substance, such as various tissues of the human body.
Sound waves reflect at the boundaries between these tissue types due to the varying velocities at which they travel. For diagnostic imaging purposes and to capture images of a fetus in utero, this method is frequently employed.
Credit: Nat Neurosci (2023). Doi: 10.1038/s41593-023-01500-7
Transparent Window
Because the skull itself is not permeable to sound waves, using ultrasound for brain imaging requires a transparent “window” to be installed into the skull.
“Importantly, ultrasound technology does not need to be implanted into the brain itself. This significantly reduces the chance for infection and leaves the brain tissue and its protective dura perfectly intact.”
said Whitney Griggs, a co-first author of the study.
“As neurons’ activity changes, so does their use of metabolic resources like oxygen. Those resources are resupplied through the bloodstream, which is the key to functional ultrasound,”
said Norman.
Doppler Effect
In this investigation, the scholars employed ultrasound technology to quantify alterations in blood flow to distinct cerebral areas. Analogous to how the pitch of an ambulance siren varies with distance from the observer, the reflected ultrasound waves from red blood cells will have a higher pitch as they approach the source and a lower pitch as they travel away.
Measuring this Doppler-effect phenomenon allowed the researchers to record tiny changes in the brain’s blood flow down to spatial regions just 100 micrometers wide, about the width of a human hair. This enabled them to simultaneously measure the activity of tiny neural populations, some as small as just 60 neurons, widely throughout the brain.
The researchers employed functional ultrasonography to monitor brain activity in nonhuman primates’ posterior parietal cortex (PPC), a region that affects movement planning and contributes to movement execution. For decades, the Andersen lab has investigated the region using different methods.
Predictive Decoding
The animals were taught two tasks: either move their hand to guide a cursor on a screen or move their eyes to stare at a certain section of the screen. As the brain–machine interface read the planned activity in their PPC, they simply needed to think about executing the task and not move their eyes or hands.
“I remember how impressive it was when this kind of predictive decoding worked with electrodes two decades ago, and it’s amazing now to see it work with a much less invasive method like ultrasound,”
said Shapiro.
The ultrasound data was delivered in real-time to a decoder (which had previously been trained to understand the meaning of the data using machine learning), which then created control signals to move the cursor to where the animal wanted it to go. The BMI was able to accomplish this with mean errors of fewer than 40 degrees on eight radial targets.
“It’s significant that the technique does not require the BMI to be recalibrated each day, unlike other BMIs,” said Griggs. “As an analogy, imagine needing to recalibrate your computer mouse for up to 15 minutes each day before use.”
The group subsequently intends to investigate the performance of BMIs calculated using ultrasound technology in human subjects and to advance the fUS technology in order to enable more precise three-dimensional imaging.
Abstract
Brain–machine interfaces (BMIs) enable people living with chronic paralysis to control computers, robots and more with nothing but thought. Existing BMIs have trade-offs across invasiveness, performance, spatial coverage and spatiotemporal resolution. Functional ultrasound (fUS) neuroimaging is an emerging technology that balances these attributes and may complement existing BMI recording technologies. In this study, we use fUS to demonstrate a successful implementation of a closed-loop ultrasonic BMI. We streamed fUS data from the posterior parietal cortex of two rhesus macaque monkeys while they performed eye and hand movements. After training, the monkeys controlled up to eight movement directions using the BMI. We also developed a method for pretraining the BMI using data from previous sessions. This enabled immediate control on subsequent days, even those that occurred months apart, without requiring extensive recalibration. These findings establish the feasibility of ultrasonic BMIs, paving the way for a new class of less-invasive (epidural) interfaces that generalize across extended time periods and promise to restore function to people with neurological impairments.
Reference:
- Griggs, W.S., Norman, S.L., Deffieux, T. et al. Decoding motor plans using a closed-loop ultrasonic brain–machine interface. Nat Neurosci (2023). Doi: 10.1038/s41593-023-01500-7
Top Image: (c) fUS-BMI algorithm. Real-time 2-Hz functional images were streamed to a linear decoder that controlled the behavioral task. The decoder used the last three fUS images of the memory period to make its prediction. If the prediction was correct, the data from that prediction were added to the training set. The decoder was retrained after every successful trial. The training set consisted of trials from the current session and/or from a previous fUS-BMI session.
(d) Multicoder algorithm. For predicting eight movement directions, the vertical component (blue) and the horizontal component (red) were separately predicted and then combined to form each fUS-BMI prediction (purple). (e) Memory-guided BMI task. The BMI task is the same as in b except that the movement period is controlled by the brain activity (via fUS-BMI) rather than eye movements. After 100 successful eye movement trials, the fUS-BMI controlled the movement prediction (closed-loop control). During the closed-loop mode, the monkey had to maintain fixation on the center fixation cue until reward delivery. Red square, monkey’s eye position (not visible to the monkey); green square, BMI-controlled cursor (visible to the monkey). Credit: Nature Neuroscience (2023). DOI: 10.1038/s41593-023-01500-7