Opening the eyes provides an instant visual perception of the world — and it seems effortless. But the process that starts with photons hitting the retina and ends with ‘seeing’ is far from simple.
The brain’s fundamental task in ‘seeing’ is to reconstruct relevant information about the world from the light that hits the eyes. Because this process is rather complicated, nerve cells in the brain — neurons — also react to images in complex ways.
Experimental approaches to characterize their responses to images have proven challenging in part because the number of possible images is endless. In the past, crucial insights often resulted from stimuli that neurons in the brain ’liked.’
Finding them relied on the intuition of the scientists and a good portion of luck.
Virtual Neural Avatar
Researchers at Baylor College of Medicine and the University of Tübingen in Germany have now developed a novel computational approach to accelerate finding these optimal stimuli. They built deep artificial neural networks that can accurately predict the neural responses produced by a biological brain to arbitrary visual stimuli.
Think of these networks as a ‘virtual avatar’ of a population of biological neurons, which can be used to dissect the neural mechanisms of sensation. They demonstrated this by synthesizing new images that made particular neurons respond forcefully.
“We want to understand how vision works. We approached this study by developing an artificial neural network that predicts the neural activity produced when an animal looks at images. If we can build such an avatar of the visual system, we can perform essentially unlimited experiments on it. Then we can go back and test in real brains with a method we named ‘inception loops,’”
said senior author Dr. Andreas Tolias, professor and Brown Foundation Endowed Chair of Neuroscience at Baylor.
Training The Network
To make the network learn how neurons respond, the researchers first recorded a large amount of brain activity using a mesoscope, a recently developed large scale functional imaging microscope.
“First, we showed mice about 5,000 natural images and recorded the neural activity from thousands of neurons as they were seeing the images. Then, we used these images and the corresponding recordings of brain activity to train a deep artificial neural network to mimic how real neurons responded to visual stimuli,”
said first author Dr. Edgar Y. Walker, a postdoctoral scientist at University of Tübingen and Baylor.
“To test whether the network had indeed learned to predict neural responses to visual images like a living mouse brain would do, we showed the network images it had not seen during learning and saw that it predicted the biological neuronal responses with high accuracy,”
said co-first author Dr. Fabian Sinz, adjunct assistant professor of neuroscience at Baylor and group leader at the University of Tübingen.
“Experimenting with these networks revealed some aspects of vision we didn’t expect. For instance, we found that the optimal stimulus for some neurons in the early stages of processing in the neocortex were checkerboards, or sharp corners as opposed to simple edges which is what we would have expected according to the current dogma in the field,”
said Tolias, founder and director of the Center for Neuroscience and Artificial Intelligence at Baylor.
“We think that this framework of fitting highly accurate artificial neural networks, performing computational experiments on them, and verifying the resulting predictions in physiological experiments can be used to investigate how neurons represent information throughout the brain. This will eventually give us a better idea of how the complex neurophysiological processes in the brain allow us to see,”
Sinz said.
[1] Walker, E.Y., Sinz, F.H., Cobos, E. et al. Inception loops discover what excites neurons most using deep predictive models. Nat Neurosci (2019) doi:10.1038/s41593-019-0517-x
Last Updated on October 18, 2022