ECE 695E Data Analysis, Design of Experiment, ML Lecture 4: Model Selection and Goodness of Fit

Table of Contents:
00:00 Lecture 4. Model Selection and Goodness of Fit
02:02 Copyright 2018
02:04 Course Outline
04:29 Outline
04:30 Matching moments to distributions
06:02 Problem of matching the moments
06:45 (1) Linear regression: balanced errors
09:46 Uncertainty in regression coefficients
12:53 Methods of least squares for weibull
14:19 Lecture 4. Model Selection and Goodness of Fit
15:51 Method of correlation coefficient
18:07 (2) Fisher's Maximum Likelihood Method
25:01 Example: origin of least square method
25:06 Example (continued)
25:10 Example: MLE estimator for one-parameter distribution
31:06 Example: MLE estimator for one-parameter distribution
32:02 Example: MLE estimator for Weibull
33:53 HW: MLE for Log-Normal
34:23 Outline
35:25 (1) Goodness of Fit: First check visually
36:19 (2) Goodness of Fit: Residual method
38:06 (3) Q-Q Method: An example
40:37 (3) Goodness of fit: Q-Q Method
41:48 Q-Q Method: An example
41:56 Q-Q method: an example
43:01 (4) Goodness of Fit: Cox-Oakes measure
44:50 (5) Kolmogorov-Smirnov algorithm
47:09 Example: Kolmogorov-Smirnov Test
48:16 (6) Pearson χ2 – test algorithm
49:21 A famous example: Schon story

This course will provide the conceptual foundation so that a student can use modern statistical concepts and tools to analyze data generated by experiments or numerical simulation. We will also discuss principles of design of experiments so that the data generated by experiments/simulation are statistically relevant and useful. We will conclude with a discussion of analytical tools for machine learning and principle component analysis. At the end of the course, a student will be able to use a broad range of tools embedded in MATLAB and Excel to analyze and interpret their data.

Topics Covered:
Review of Basic Statistical Concepts
Where do data come from: Big vs. Small Data
Collecting and Plotting Data: Principles of Robust Data Analysis
Physical vs. Empirical Distribution
Model Selection and Goodness of Fit
Scaling Theory of Design of Experiments
Statistical Theory of Design of Experiments
Machine Learning vs. Physics-based Machine Learning