'An Overview of Probabilistic Programming' by Vikash K. Mansinghka

Probabilistic inference is a widely-used, rigorous approach for processing ambiguous information based on models that are uncertain or incomplete. However, models and inference algorithms can be difficult to specify and implement, let alone design, validate, or optimize. Additionally, inference often appears to be intractable. Probabilistic programming is an emerging field that aims to address these challenges by formalizing modeling and inference using key ideas from probability theory, programming languages, and Turing-universal computation.

This talk will illustrate the common underlying principles of probabilistic programming using three research platforms:

BayesDB, a Bayesian database that enables users to directly query the probable implications of data tables without training in statistics. It provides BQL, an SQL-like language for Bayesian data analysis, and MML, a minimal language for building generative population models by combining automatic model-building techniques with qualitative constraints and custom statistical code. BayesDB has been applied to problems such as cleaning and exploring a public database of Earth satellites and assessing the evidence for microbial biomarkers of Kwashiorkor, a form of severe malnutrition.

Picture, an imperative probabilistic language for 3D scene perception. Picture uses deep neural networks and statistical learning to invert generative models based on computer graphics. 50-line Picture programs can infer 3D models of human poses, faces, and other object classes from single images.

Venture, an integrated platform that aims to be sufficiently expressive, efficient, and extensible for general-purpose use. It provides VentureScript, a language that gives users fine-grained control over both modeling and inference, and defines a common interface for integrating components written in other probabilistic languages. Recent applications include structure discovery from time-series via Gaussian processes and reflective AI techniques such as Bayesian optimization.

Vikash K. Mansinghka
MIT

[thechhavibansal]

how do you determine the type of ML algorithm to run for a particular recommend query? or given any query which asks for prediction

[Keepedia]

Bayesian simulation of data would miss any assumptions in the training data which may not hold in the necessary tests.. it's slightly scary
More (real) data could have many patterns too subtle (eg. Sampling ber(0.01) 10 vs 100 vs 1000 times) that the larger simulated data would provide wrong inferences about
Ig training with more data can only improve performance when the data is not yet representative. After it is, real data becomes the same as simulated data