Advanced Eye Tracking with Python & JavaScript: Dual-Camera Setup and Real-Time Processing

In this video, I dive into the complexities of eye tracking technology with an emphasis on Python and JavaScript implementations. I discuss various aspects of eye and pupil tracking technologies, leveraging tools like MediaPipe, NumPy, TensorFlowJS, and BlazeFace for real-time eye tracking capabilities. I also delve into developing a Flask application to handle the backend of our eye tracking feature, while using JavaScript to offload much of the processing to the client-side for efficiency. Throughout the video, I explain the setup of our dual-webcam system aiming to achieve higher frame rates and the challenges of calibration and real-time processing. I also touch on my side project, BioniCloud, which integrates these technologies on a server I manage.

#EyeTracking #Python #JavaScript #MediaPipe #NumPy #TensorFlowJS #BlazeFace #WebDevelopment #Flask #BioniCloud #TechTutorial

00:00 - Introduction to Eye Tracking
00:06 - Overview of Current Technologies
00:12 - Setup Explanation: Python with MediaPipe and NumPy
00:43 - Hardware Setup: Dual Webcams and Microphones
01:14 - Integration with GPT-4 for Enhanced Capabilities
01:54 - Development of Flask Application for Eye Tracking
02:49 - Transition to JavaScript for Client-Side Processing
03:30 - Challenges with CPU and GPU Load
04:06 - Introduction to BioniCloud for Web App Hosting
04:50 - Exploring Alternatives to MediaPipe with BlazeFace
05:50 - Code Sharing and Development in Flask
06:09 - Future Enhancements and Refactoring Plans
07:00 - File Structure and Main Functionality Discussion
08:07 - Troubleshooting and Code Improvements
09:19 - Visual Overlays and UI Adjustments
10:01 - Complete Refactoring for Modular JavaScript
11:18 - Detailed Breakdown of JavaScript Functionality
12:05 - Integration Issues and Fixes
13:07 - Handling Module Types in JavaScript
14:00 - Final Adjustments and Testing
15:13 - Addressing Load Issues and Errors
16:05 - Complete Rewrite of Index HTML for Error Resolution
17:09 - Debugging and File Path Corrections
18:05 - Seeking Solutions for Code Errors
19:06 - Discussion on Script Types and Module Usage
20:18 - Overlay Issues on Video Footage
21:09 - CSS Adjustments and Functionality Checks
22:07 - Validation of Predictions and Video Overlay
23:13 - Final Review and Code Optimization
24:03 - Error Handling and Function Refinements