PULP-DroNet -- Autonomous Artificial Intelligence-powered Nano-Drone

PULP-DroNet is a deep learning-powered visual navigation engine that enables autonomous navigation of a pocket-size quadrotor in a previously unseen environment.
Thanks to PULP-DroNet the nano-drone can explore the environment, avoiding collisions also with dynamic obstacles, in complete autonomy -- no human operator, no ad-hoc external signals, and no remote laptop!
This means that all the complex computations are done directly aboard the vehicle and very fast.

The visual navigation engine is composed of both a software and a hardware part.
The former is based on the previous DroNet [1] project developed by the RPG [2] from the University of Zürich (UZH).
DroNet is a shallow convolutional neural network (CNN) which has been used to control a standard-size quadrotor in a set of environments via remote computation.
The hardware soul of PULP-DroNet is embodied by the PULP-Shield an ultra-low power visual navigation module featuring a Parallel Ultra-Low-Power (PULP) GAP8 System-on-Chip (SoC) from GreenWaves Technologies [3], an ultra-low power camera, and off-chip Flash/DRAM memory; the shield is designed as a pluggable PCB for the Crazyflie 2.0 [4] nano-drone.

Then, we developed a general methodology for deploying state-of-the-art deep learning algorithms on top of ultra-low power embedded computation nodes, like a miniaturized drone.
Our novel methodology allowed us first to deploy DroNet on the PULP-Shield, and then demonstrating how it enables the execution the CNN on board the CrazyFlie 2.0 within only 64-284mW and with a throughput of 6-18 frame-per-second!
Finally, we field-prove our methodology presenting a closed-loop fully working demonstration of vision-driven autonomous navigation relying only on onboard resources, and within an ultra-low power budget.

We release here, as open source, all our code, hardware designs, datasets, and trained networks.

Reference:
D. Palossi, F. Conti, and L. Benini
An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs
To be published in Proceeding of DCOSS'19, 29-31 May 2019

[reyz]

What causes the drone to regularly bounce from left to right?

[huixind]

Where is the github link of the hardware, or I want to ask what the name of the camera you use.