Interactive Machine Learning for Generative Models

Interactive Machine Learning for Generative Models

NeurIPS 2023 Workshop :

Abstract :
Effective control of generative media models remains a challenge for specialised generation tasks, where no suitable dataset to train a contrastive language model exists. We describe a new approach that enables users to interactively create bespoke text-to-media mappings for arbitrary media generation models, using small numbers of examples. This approach facilitates new strategies---very distinct from contrastive language pretraining approaches---for using language, e.g., high-level descriptors and modal properties, to drive media creation in creative contexts. These controls are not well served by existing methods, which commonly depend on attributes e.g., genre, style, description, to generate and steer creative outputs.

Paper Link :

Author :
Junichi Shimizu · Ireti Olowe · Terence Broad · Gabriel Vigliensoni · Prashanth Thattai Ravikumar · Rebecca Fiebrink