Dealing with nonlinear data: Polynomial regression and log transformations

[toryreads]

Consider this me asking nicely (BEGGING) for the non-linear regression/Bayesian video! :D Also, arm twist! Arm twist!

[derekcaramella]

How have I not found this channel sooner! Amazing stuff, binge watching this channel

[jackelsey]

Here's another vote for a nonlinear regression analysis video. That approach made sense for my dissertation research (inverse problems with mechanistic time-series models), and I'm curious what your perspective is. It seems to me like weighted least squares can work well in many heteroscedastic contexts if you assume residuals are independent and have a constant CoV.

[AurelODJO]

Hello, i am statistician. i live in Africa and really appreciate this lessons. So Fun thanks to the Teacher.

[dominicl]

Well done! Looking forward to the GLM video. I still did not fully understand the link functions there.

[Lello]

Hey Dustin! Speaking of non-linear data, what about a video on Generalized Additive (Mixed) Models? GA(M)Ms?! I'm sure it'd be useful for many of us!!!

[galenseilis]

I look at non-linear regression and Bayesian regression as logically independent classes of models. They can both involve using more fundamental principles, rather than just grabbing a recipe off the shelf as the other extreme, which I think is a valuable skill for a statistician to have.

[galenseilis]

I have noticed the word "line" in "linear", but unfortunately the terminology is more complicated than Dustin presented. I'll give a couple of reasons:

The first is that they are not synonyms in mathematics. All lines are linear, but not all linear functions are lines. For example, the derivative operator linear on the space of analytic functions, but it is not a line per se.

The second is that statisticians were focused on the parameters when they coined the term "linear model". Conventionally "linear model" refers to a regression model which is linear in its conditional expection with respect to the unknown parameters. This makes both the example polynomial regression and log-transformed regression model in the video out to be special cases of linear models.

[StatisticsSupreme]

On the log transform, you say the estimate b is now on a log scale, yes. But that is not a problem for interpretation, when you transform it back to where it came from. Exponentiate that value and you are back and can interpret it as normal. So there is no real "cost" there. But overall nice video, as always :D

[galenseilis]

Plotting the residuals can be very beneficial for learning about the performance of a predictive model. There is a common pitfall worth mentioning though. The distribution of the residuals is not in general the likelihood distribution.

Take for example the equation

Y = X + epsilon

where

Y ~ Poisson(lambda)
X ~ Poisson(mu)

and

epsilon ~ Poisson(tau).

If you compute the residuals you will obtain a Skellam random variable rather than a Poisson random variable.

How do you interpret the coefficients of the polynomial model?

[adrianor.]

What package has the visualize() function? Great explanations, as usual!

[sarahahmedchawsheen]

Good job, thanks. But what if we have 2 or more IV's??

[galenseilis]

It has been a long while since I have really thought about semi-partial correlation coefficients. But if memory serves it does not in general equal to the conditional correlation coefficient except under certain families of distributions. A sufficient criterion for distributional assumptions to hold such that the partial correlation equals the conditional correlation is when the joint distribution is in an exponential parametric family of distributions.

[iiveup]

I would recommend Fractional Polynomial Models that identify the best transformations of the covariates, with the obvious risk of overfitting and ambiguity in the interpretation of the coefficients.

[caseypdx]

If you're looking to report the original values (not logged transformed) would you say it's best to avoid log transforming?

[igoryakovenko]

Great video! Any chance you be open to sharing a link to the dataset used so we can re create the exercise and try it ourselves? Thank you!

[omarabdelrahman]

Please do a video on Quantile Regression?

[TheMrSodo]

Thanks for your work as always, I am approaching Bayesian statistics so it would be great to see you going into bayesian regression. please please please!

[aaditya]

Why don't you use poly(var_name, n) instead, for orthogonal polynomials?