Human Machine Interface (HMI) platform to make the world of aviation safer. SHORT ENG

PEGGASUS - Pilot Eye Gaze and Gesture tracking for Avionics Systems using Unobtrusive Solutions
Using the latest AI and visual tracking technologies, PEGGASUS integrates customized eye-gaze detection and gesture recognition into a single framework to understand pilots’ cognitive state. It helps enhance efficiency while working to lessen pilots’ workload. PEGGASUS is set to help airline companies maximize flight safety in heavily automated operations.

Discover immediately the power of PEGGASUS
- Real-time alerts
- Secure and robust
- Capable of adapting to daytime/nighttime scenarios
- Able to differentiate weather conditions, shocks, and vibrations
- User-centric design
- Tested by experts

Propelling safety in the EU avionics industry to new heights
Pilots are increasingly connected to their aircraft, through complex automated systems, displays and instruments in the cockpit. This increased automation has been designed to help reduce pilot workload. Paradoxically, however, there is the risk that when faced with an uncommon event, pilots’ overreliance on the autopilot system may limit their ability to cope with the vast amounts of data generated by those very same systems.

PEGGASUS optimizes human-machine relations. Using the intuitive and natural recognition of eye gaze and hand gestures. Thanks to computer vision (CV) and machine learning (ML) algorithms, the platform creates a responsive cockpit experience that is sensitive to the pilot’s needs, assessing their cognitive state while providing assistive to ease their workload.

Through the European Clean Sky 2 initiative, five major European entities working with, for or in the avionics industry pooled their expert knowledge to develop PEGGASUS.

Key Benefits
- User-centric design, offering real-time alerts to pilots and developed and tested in collaboration with qualified technical aviation experts and pilots.
- Optimized AI algorithms extracting facial features using multi-view information, a large field of view, and temporal data to improve the system’s robustness.
- Edge AI provides real-time eye-gaze tracking and gesture recognition for seamless interaction with the cockpit.
- Capable of adapting to varying environmental factors e.g., lighting conditions, vibrations, etc.

The future of avionics is now
Improving human-machine interaction in the cockpit, PEGGASUS can help pilots better manage their cognitive workload, enabling them to maintain a strong situational awareness even in stressful scenarios — creating safer skies for all. The future of avionics rests with PEGGASUS, and it’s hoped this platform will also be used to train the pilots of the future.

PEGGASUS received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461

The project was undertaken by five partners, including CSEM (Project Coordinator) (CH), Thales AVS (Topic Manager) (FR), Swiss International Air Lines AG (CH), Eidgenoessische Technische Hochschule Zurich (ETHZ) (CH), and SERMA Ingenierie (FR).