Automatic Annotation Synchronizing with Textual Description for Visualization

Automatic Annotation Synchronizing with Textual Description for Visualization
Chufan Lai;Zhixian Lin;Ruike Jiang;Yun Han;Can Liu;Xiaoru Yuan

CHI'20: ACM CHI Conference on Human Factors in Computing Systems
Session: Talk visually to me

Abstract
In this paper, we propose a technique for automatically annotating visualizations according to the textual description. In our approach, visual elements in the target visualization, along with their visual properties, are identified and extracted with a Mask R-CNN model. Meanwhile, the description is parsed to generate visual search requests. Based on the identification results and search requests, each descriptive sentence is displayed beside the described focal areas as annotations. Different sentences are presented in various scenes of the generated animation to promote a vivid step-by-step presentation. With a user-customized style, the animation can guide the audience's attention via proper highlighting such as emphasizing specific features or isolating part of the data. We demonstrate the utility and usability of our method through a user study with use cases.

Video preview for the ACM CHI Conference on Human Factors in Computing Systems 2020