filmov
tv
LTI Colloquium: Intuitive Reasoning as (Un)supervised Neural Generation -- Yejin Choi
Показать описание
Title: Intuitive Reasoning as (Un)supervised Neural Generation
Speaker: Yejin Choi @ University of Washington
Twitter: @yejinchoinka
Abstract: Neural language models, as they grow in scale, continue to surprise us with utterly nonsensical and counterintuitive errors despite their otherwise remarkable performances on leaderboards. In this talk, I will argue that it is time to break out of the currently dominant paradigm of sequence-to-sequence models with task-specific supervision built on top of large-scale pre-trained neural networks. First, I will argue for unsupervised inference-time algorithms to make better lemonade out of neural language models. As examples, I will demonstrate how unsupervised decoding algorithms can elicit advanced reasoning capabilities such as non-monotonic reasoning (e.g., counterfactual and abductive reasoning) out of off-the-shelf left-to-right language models, and how in some controlled text generation benchmarks, unsupervised decoding can match or even outperform supervised approaches. Next, I will highlight the importance of melding explicit and declarative knowledge encoded in symbolic knowledge graphs with implicit and observed knowledge encoded in neural language models. As a concrete case study, I will present Social Chemistry 101, a new conceptual formalism, a knowledge graph, and neural models to reason about social, moral, and ethical norms.
#NLProc #MachineLearning Natural Language Processing CMU LTI
Speaker: Yejin Choi @ University of Washington
Twitter: @yejinchoinka
Abstract: Neural language models, as they grow in scale, continue to surprise us with utterly nonsensical and counterintuitive errors despite their otherwise remarkable performances on leaderboards. In this talk, I will argue that it is time to break out of the currently dominant paradigm of sequence-to-sequence models with task-specific supervision built on top of large-scale pre-trained neural networks. First, I will argue for unsupervised inference-time algorithms to make better lemonade out of neural language models. As examples, I will demonstrate how unsupervised decoding algorithms can elicit advanced reasoning capabilities such as non-monotonic reasoning (e.g., counterfactual and abductive reasoning) out of off-the-shelf left-to-right language models, and how in some controlled text generation benchmarks, unsupervised decoding can match or even outperform supervised approaches. Next, I will highlight the importance of melding explicit and declarative knowledge encoded in symbolic knowledge graphs with implicit and observed knowledge encoded in neural language models. As a concrete case study, I will present Social Chemistry 101, a new conceptual formalism, a knowledge graph, and neural models to reason about social, moral, and ethical norms.
#NLProc #MachineLearning Natural Language Processing CMU LTI