Constructing Models to Deal with Missing Data | SciPy 2016 | Deborah Hanus

Most scientists carefully collect data and select data resources. In a perfect world, we would have pristine, complete datasets. Yet, we are frequently challenged by incomplete and missing data. We are often taught to "ignore" missing data. In practice, however, ignoring the wrong types of data may build biases into our datasets, invalidating our conclusions. Here, we discuss three types of missing data (data missing completely at random, missing at random, and missing not at random) and heuristics for identifying and dealing with each type. Then we delve into an example, where we impute missing data for a simulator that utilizes reinforcement learning to predict effective HIV treatments. When we finish, you will know how to identify each of the three types of missing data and how to deal with each in your own projects.

[daphnes]

Thank you, this was very helpful to understand the types of missing data and how to deal !

[tesfahun_taddege]

Very Interesting, both the presentation and the questions were Helpful !
Thank you!

[ServetEdu]

Thank you! This is awesome. Good questions too.

[xupgfwq]

Thank you for the lecture, could you kindly reccomend what to read on developing dynamic models for longitudinal data with missing values?

[BigBadBurrow]

I might be missing something, but if you have sets of data where some of it is missing, but you have other data sets where it exists / is complete, then why not train a network with the data you have where it's complete, in order to predict the data where it's not? i.e. a machine learning solution to a machine learning problem.

Hi mam, i am also a Ph.D student doing in agricultural statistics in IASRI. I need to know building models for missing data. Can you send me your mail id. Because I need some help from you mam.