Scientists found that network simulations became unstable after periods of unsupervised learning. When they exposed network simulations to states similar to waves that our brains experience during sleep, stability was restored.
Will androids dream of electric sheep? No one can tell. But researchers from Los Alamos National Laboratory say they will almost certainly need time to get benefits the same as sleep provides to human beings.
Computer scientist Yijing Watkins said “We study spiking neural networks, which are systems that learn much as living brains do. We were fascinated by the prospect of training a neuromorphic processor in a manner analogous to how humans and other biological systems learn from their environment during childhood development.”
What surprised the scientists was that the network simulations became unstable after periods of unsupervised learning. However, when they exposed them to states that are similar to the waves that our brains experience during sleep, stability was restored. “It was as though we were giving the neural networks the equivalent of a good night’s rest,” said Watkins.
Los Alamos computer scientist and study coauthor Garrett Kenyon said: “The issue of how to keep learning systems from becoming unstable really only arises when attempting to utilize biologically realistic, spiking neuromorphic processors or when trying to understand biology itself.
“The vast majority of machine learning, deep learning, and AI researchers never encounter this issue,” he added, “because in the very artificial systems they study they have the luxury of performing global mathematical operations that have the effect of regulating the overall dynamical gain of the system.”
The researchers characterize the decision to expose the networks to an artificial analog of sleep as nearly a last ditch effort to stabilize them. They experimented with various types of noise, roughly comparable to the static you might encounter between stations while tuning a radio. The best results came when they used waves of so-called Gaussian noise, which includes a wide range of frequencies and amplitudes. They hypothesize that the noise mimics the input received by biological neurons during slow-wave sleep. The results suggest that slow-wave sleep may act, in part, to ensure that cortical neurons maintain their stability and do not hallucinate.
The groups’ next goal is to implement their algorithm on Intel’s Loihi neuromorphic chip. They hope allowing Loihi to sleep from time to time will enable it to stably process information from a silicon retina camera in real time. If the findings confirm the need for sleep in artificial brains, we can probably expect the same to be true of androids and other intelligent machines that may come about in the future.
Read more about: artificial intelligence, Brain, Sleep
