Researchers Develop Animal Brain-like Neuromorphic Processors Revolutionizing Autonomous Robots

At the intersection of drones and autonomous mobility comes autonomous drones or self-flying robots that can be used for a wide range of purposes.

With the addition of artificial intelligence (AI) in the mix, these devices can become more powerful and capable.

However, this combination comes with one challenging tradeoff: AI systems running deep artificial neural networks require GPUs (Graphics Processing Unit) and tremendous energy to function. This means they add big and heavy hardware and batteries to the entire set up, making it a costly affair while also imposing restrictions on applications.

Looking to tackle this challenge, a team of researchers from the Delft University of Technology researched, developed and experimented with new technology to boost the use of autonomous robots.

So, what did the research team do and how will it benefit autonomous robots? Let’s explore!
 

What Did The Researchers Announce?

 

• According to a news release published on its website, researchers from the Delft University of Technology announced they developed a drone that can fly autonomously using neuromorphic image processing.

• Neuromorphic computing is a branch of computing that is structured and functions like a human brain and uses physical artificial neurons to complete computations.

• However, for this experiment the researchers opted to use neuromorphic image processing and controls based on how animal brains work.

• Animal brains process information in a different way than neural networks running on GPUs. Here the neurons process information asynchronously, communicating through electrical pulses called spikes. To reduce energy usage caused by spiking, the brain minimizes spiking, leading to sparse processing.

• Using this technique, scientists and technology companies are creating new neuromorphic processors that run spiking neural networks.

• To boost energy efficiency, neuromorphic processors combine with neuromorphic sensors, such as neuromorphic cameras.

• These cameras don’t make images at fixed intervals but rather each pixel sends a signal when it becomes lighter or darker, allowing for quicker motion detection. These sensors can work in dark and bright environments, are more energy efficient and can feed directly into spiking neural networks.

• This helps reduce the amount of data and energy used in comparison to neural networks running on GPUs, so much so that the results displayed 64 times faster data processing and 3 times less energy usage.

• Furthermore, this technique would be highly suitable for small drones and robots, even allowing them to “become as small, agile, and smart as flying insects or birds”, especially considering how they face challenges of being able to carry limited sensing and computing resources.

• Such small drones can navigate in narrow environments, can be deployed in swarms and cover an area quickly. They can be used for applications such as monitoring crops in greenhouses or keep track of warehouse stocks.

• On May 15, 2024, the research team demonstrated the successful flight of a drone using neuromorphic vision and control, which was powered by a spiking neural network deployed on Intel’s Loihi neuromorphic research chip onboard the drone.

• “Thanks to the network, the drone can perceive and control its own motion in all directions.”

• “The vision part of the network, consisting of five layers and 28,800 neurons, maps incoming raw events to ego-motion estimates and was trained with self-supervised learning on real event data,” read an article published by the researchers in Science Robotics.

• As per researchers, the achievement is a great step in the direction of enhancing tiny autonomous robots, however, its true value will depend upon further progress made in scaling down the hardware and improving navigation.

 

What Did The Researchers Say?

 

• Jesse Hagenaars, PhD candidate and one of the authors of the article, said, “The calculations performed by spiking neural networks are much simpler than those in standard deep neural networks. Whereas digital spiking neurons only need to add integers, standard neurons have to multiply and add floating point numbers.”

• [Contd.] “This makes spiking neural networks quicker and more energy efficient. To understand why, think of how humans also find it much easier to calculate 5 + 8 than to calculate 6.25 x 3.45 + 4.05 x 3.45.”

• Federico Paredes-Vallés, a researcher of the study, said, “We faced many challenges, but the hardest one was to imagine how we could train a spiking neural network so that training would be both sufficiently fast and the trained network would function well on the real robot.”

• [Contd.] “In the end, we designed a network consisting of two modules. The first module learns to visually perceive motion from the signals of a moving neuromorphic camera. It does so completely by itself, in a self-supervised way, based only on the data from the camera. This is similar to how also animals learn to perceive the world by themselves.”

• [Contd.] “The second module learns to map the estimated motion to control commands, in a simulator. This learning relied on an artificial evolution in simulation, in which networks that were better in controlling the drone had a higher chance of producing offspring.”

• [Contd.] “Over the generations of the artificial evolution, the spiking neural networks got increasingly good at control, and were finally able to fly in any direction at different speeds. We trained both modules and developed a way with which we could merge them together. We were happy to see that the merged network immediately worked well on the real robot.”

Do you think this technology should be adopted by drone and autonomous robot manufacturers?

Let us know in the comments below!