Learning High-Speed Flight in the Wild (Science Robotics, 2021)

Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. While this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. Indeed, the subtasks are executed sequentially, leading to increased processing latency and compounding of errors through the pipeline. Here we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and man-made environments at high speeds, with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines.

Reference:
A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, D. Scaramuzza,
"Learning High-Speed Flight in the Wild",
Science Robotics, October 6, 2021

For more info about our research on:

Affiliations:
E. Kaufmann, A. Loquercio and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
R. Ranftl, M. Müller and V. Koltun are with Intel Labs

[TonyB]

Imagine using these to search the woods for a lost child or hiker. You could send out a hundred of these to search and map the forest with infrared as well.

[imannabiyouni]

Is the very first part of the video, result of the study or it is just a motion graphics for marketing?
I am asking, because the first part of the video looks to good to be true?! a pro is flying the drone which is recording the video and they are passing very tight gaps. But after that, in the rest of the video, recordings are from a stationary camera and the drone passes easier obstacles?!

[sbstn]

Am I really the only one who thinks of the Star Wars chase scene in Endor and is excited that there's finally a real world solution for it?

[kingstonjames]

The background voice went from normal student lvl to Discovery Channel lvl.

[mikasteven]

It's a solid and useful work! Amazing!

[mcdiggi]

Awesome Work! May I ask what kind of simulator you're using to train the RL agents?

[MikaelMurstam]

so the network is running on the drone's hardware?

[Blooper]

Who the hell would dislike this video!

[richierescue]

Potential slaughterbot swarms just got really agile.

[Son_of_Nyango]

one step closer to an irl half-life man-hack 👌probably better we use it for finding lost kids like Tonyb said though lol.

[firstlast]

Hunting Osama ? Or …. is he hunting us?

[Gallardo]

One day they hunt for humans with these crazy inventions

[brendo]

Tomorrow's drones will flawlessly navigate complex urban environments to deliver munitions to randomly chosen civilians

[Flamenawer]

I have a couple of drones able to run this, off board or onboard.
Looking by the drone specs to reproduce it.

[Tetsujinfr]

Really cool. I am just a bit nervous this will end up with military applications. I hope I am wrong.

[homalozoa]

I'm curious about how did you track drone and record the drone video. If someone did it. I think this guy is really the best expert. hahaha

[kevin-jmqb]

nice way to send a c4 freedom care package to the target enemy

[syrusk]

Put another way, you're not going to outrun or escape. Unless you pretend to be a tree.

[patlelion]

This could be a good watchdog compagnon for female runner in park/forest

[rodneyk]

So where the hell is Brian laundrie hiding!??