ICASSP 22 Tutorial, 'Neural Model Reprogramming and Prompting for Speech Modeling, ' Huck Yang

Despite the fact of achieving high standard accuracy in a variety of machine learning tasks, neural network based prediction models have recently been identified as having the issue of lacking adversarial robustness. In particular, with the recent advance of the speech and language deep learning models, new challenges (e.g., end-to-end evaluation, data privacy) and opportunities (e.g., noise-aware adaptation) will be introduced.

This tutorial will provide an overview of recent advances in the research of adversarial robustness, featuring both comprehensive research topics and technical depth. We will cover three fundamental pillars in (1) adversarial robustness: attack, defense, verification, and recent advances in (2) adversarial reprogramming. Attack refers to the efficient generation of adversarial examples for robustness assessment under different attack assumptions (e.g., white-box or black-box attacks). Defense refers to adversary detection and robust training algorithms to enhance model robustness. Verification refers to attack-agnostic metrics and certification algorithms for proper evaluation of adversarial robustness and standardization.

For each pillar, we will emphasize the tight connection between signal processing and the research in adversarial robustness, ranging from fundamental techniques such as first-order and zero-order optimization, minimax optimization, geometric analysis, model compression, data filtering, and quantization, subspace analysis, active sampling, frequency component analysis to specific applications such as computer vision, automatic speech recognition, natural language processing, and data regression.

Finally, motivated by studies in adversarial robustness, model reprogramming will be introduced as an emerging and powerful technique for data-efficient transfer learning for large “foundation” pre-trained models with limited target domain data.