FingerIO: Using Active Sonar for Fine-Grained Finger Tracking

FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, Shyamnath Gollakota

CHI '16: ACM Conference on Human Factors in Computing Systems
Session: Tracking Fingers

Abstract
We present fingerIO, a novel fine-grained finger tracking solution for around-device interaction. FingerIO does not require instrumenting the finger with sensors and works even in the presence of occlusions between the finger and the device. We achieve this by transforming the device into an active sonar system that transmits inaudible sound signals and tracks the echoes of the finger at its microphones. To achieve sub-centimeter level tracking accuracies, we present an innovative approach that use a modulation technique commonly used in wireless communication called Orthogonal Frequency Division Multiplexing (OFDM). Our evaluation shows that fingerIO can achieve 2-D finger tracking with an average accuracy of 8 mm using the in-built microphones and speaker of a Samsung Galaxy S4. It also tracks subtle finger motion around the device, even when the phone is in the pocket. Finally, we prototype a smart watch form-factor fingerIO device and show that it can extend the interaction space to a 0.5_0.25 m2 region on either side of the device and work even when it is fully occluded from the finger.

Recorded at the 2016 CHI Conference on Human Factors in Computing Systems in San Jose, CA, United States, May 7-12, 2016