Learning to Fly By Crashing

Researchers at Carnegie Mellon University successfully taught a drone to fly autonomously by deliberately crashing it 11,500 times into random objects.

Dhiraj Gandhi, Lerrel Pinto and Abhinav Gupta, three roboticists out of Carnegie Mellon University in Pennsylvania, USA used Artificial Intelligence based on the drone crash logs to train a drone to fly autonomously even in cluttered indoor environments.

The Parrot AR drone 2.0 they used crashed a total 11,500 times, during 40 hours of flying time and in 20 different indoor environments.

Their method and results are available in their complete paper “Learning to fly by crashing”. The paper shows that their unconventional approach delivers effective learning results while avoiding issues related to more standard methods.

One of the problems affecting the current research on autonomous flight and sense-and-avoid technologies has been the lack of large-scale real data – which prevented the learning algorithms from getting enough information to learn. The researchers explain in their paper:

“We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation”


Method Details

Practically, the approach is quite straightforward. After each collision, the images from the trajectory are split into two parts: the part where the drone was doing fine, and the part just before it crashes. Both parts are then crushed into a deep convolutional neural network that allows the drone to learn to fly.

Once the learning phase by crashing is over, the drone will split the image from its forward camera into a left image and a right image. It then compares both to what it sees straight ahead.

If one of those two side images seems less likely to cause a collision than going straight, the drone turns in that direction. Otherwise, it continues moving forward.

In the end, the drone is able to fly autonomously, even in narrow, cluttered environments, around moving obstacles such as humans, and in the midst of featureless white walls and even glass doors.

The real advantage compared to other conventional approaches is that if you allow crashing, the entire learning process can be self-supervised. Just set the drone up in a room and let it learn how to fly autonomously!

Despite the numerous collisions the researchers claim that the cost of the crashes are negligible since the hulls of the drone are cheap and easy to replace.

Why does this research matter?

Every day drones are flying further afield and beyond-line-of-sight is imminent.

In order to ensure safety for the drone and the payload but also and most importantly for the people around and under it, the drone must be able to react to unexpected obstacles and avoid them. This research is a very promising step forward in the field of autonomous sense-and-avoid technologies.