Human-AI Systems for Making Videos Useful – Amy Pavel

Computer Science Seminar Series
February 26, 2021

“Human-AI Systems for Making Videos Useful”
Amy Pavel, Carnegie Mellon University

Video is becoming a core medium for communicating a wide range of content, including educational lectures, vlogs, and how-to tutorials. While videos are engaging and informative, they lack the familiar and useful affordances of text for browsing, skimming, and flexibly transforming information. This severely limits who can interact with video content and how they can interact with it, makes editing a laborious process, and means that much of the information in videos is not accessible to everyone. But what are the future systems that will make videos useful for all users? In this talk, Amy Pavel will share her work creating interactive human-AI systems that leverage multiple communication media (e.g., text, video, and audio) across two main research areas: 1) helping domain experts surface content of interest through interactive video abstractions and 2) making videos non-visually accessible through interactions for video accessibility. First, she will share core challenges of seeking information in videos from interviews with domain experts. Then, she will share new interactive systems that leverage AI and evaluations that demonstrate system efficacy. She will conclude with how hybrid human-computer interaction and AI breakthroughs will make digital communication more effective and accessible in the future and how new interactions can help us to realize the full potential of recent advances in AI and machine learning.

Amy Pavel is a postdoctoral fellow at the Carnegie Mellon University Human-Computer Interaction Institute and a research scientist in Machine Learning and AI at Apple. Her research explores how interactive tools, augmented with machine learning techniques, can make digital communication more effective and accessible. She has published her work in conferences including the ACM Symposium on User Interface Software and Technology, the ACM Conference on Human Factors in Computing Systems, the ACM Special Interest Group on Accessible Computing Conference on Computers and Accessibility, and other ACM and Institute of Electrical and Electronics Engineers venues. She previously received her PhD in computer science at the University of California, Berkeley, where her work was supported by a Department of Defense National Defense Science and Engineering Graduate Fellowship.