Detrending and deseasonalizing data with fourier series

This is Part 3 of a multi-part series on Pricing Weather Derivatives. In this video we take Daily Average Temperature (DAT) series from Sydney Observatory Hill starting from 1-Jan 1859 and attempt to de-trend and remove seasonal variation using fourier series. In timeseries this is also known as time series decomposition, where the terms are detrending and deseasonalizing data.

The denoised temperature time series reveals that temperatures have somewhat uniform peaks. This implies that we could use a first order fourier series model to estimate the seasonal variation in Daily Average Temperature.

In this series we take a deep dive into a type of exotic financial products weather derivatives. Weather derivatives are financial instruments that can be used to reduce risk associated with adverse weather conditions like temperature, rainfall, frost, snow, and wind speeds.

★ ★ Code Available on GitHub ★ ★

★ ★ QuantPy GitHub ★ ★

★ ★ Discord Community ★ ★

★ ★ Support our Patreon Community ★ ★
Get access to Jupyter Notebooks that can run in the browser without downloading python.

★ ★ ThetaData API ★ ★
ThetaData's API provides both realtime and historical options data for end-of-day, and intraday trades and quotes. Use coupon 'QPY1' to receive 20% off on your first month.

★ ★ Online Quant Tutorials ★ ★

★ ★ Contact Us ★ ★

Disclaimer: All ideas, opinions, recommendations and/or forecasts, expressed or implied in this content, are for informational and educational purposes only and should not be construed as financial product advice or an inducement or instruction to invest, trade, and/or speculate in the markets. Any action or refraining from action; investments, trades, and/or speculations made in light of the ideas, opinions, and/or forecasts, expressed or implied in this content, are committed at your own risk an consequence, financial or otherwise. As an affiliate of ThetaData, QuantPy Pty Ltd is compensated for any purchases made through the link provided in this description.

[mohammedboussardi]

I like watching even if i do not understand the concept yet

[alexgregory]

Great stuff! How would you approach the same problem with hourly data with diurnal and annual seasonality?

[laukittom]

it is amazing, please do more analysis

[rawsaucerobert]

Hi there, I tried to apply this to my own weather data, and while the model_fit and model_fit_general fit perfectly to the seasonality, I needed to phase shift the "model" function by 180 degrees to get the correct curve. I would like to attribute it to my data being in the northern hemisphere but why would a sine wave care about the seasons? Do you have any thoughts on this?
TLDR: Had to phase shift the "model" function by 180 degrees to get a proper fit on my data, everything else worked as expected.

[denisplotnikov]

Need videos about how to start (maybe step-by-step plan) the journey of quantitative finance until get a position of quantitative trader. Could you create such videos?

[graymars]

I’m half way through, given that I’m familiar with the math: this is awesome. Thanks for uploading

[husseinnasser]

Great stuff! Can you please make a video on Sticky Delta/Sticky Strike and how that relates to fixed strike vols and Implied/Realized Skew on the vol surface?

[gambu]

Wow, please show your analysis as well. If you can take us through these steps in R-Studio or Python it would really solidify our knowledge.

[stretch]

Adjusting omega for speed of processes is where I'm getting stuck at the moment!

[arinpacuretu]

Hello! Can you please tell me which source have you used for the simplified Fourier series? I can't find them in the bio, thanks!

[phsopher]

Interesting video. Would have loved to see more of the code.

Not sure what parsonomy means. Is it like parsimony?

[giovannipaolo]

Great video. Btw. have you thought about making a newsletter with new articles on your website?

[CTT]

2:30. You said you used a square function convoluted with the time series data and the got the plot. I got pretty confused here. First, why did you choose the square function (btw, wat exactly it is?) instead of sth else? Second, how did you calculate the convolution of a function with a discrete data, by substituting the time t into the square function and then computing the convolution of two data series?

[_AbUser]

Well i think non normal error distribution caused by non monotonous growth the chart in a real.. You trying to multiply perfect line and perfect sin() and the modeling chart will growths monotonous while the real one - not. And of course You will get some excesses thant moment the real chart will start to grow like parabolic in average.. As for me thats pretty obviously.. I think the real chart should be additionally averaged, thats makes the average error bigger but makes it close to normal... (There is why You started to talk to try ARIMA like model i think..) Or to try the rose noise, i heard someone using to modeling the stock prices..

But the season decomposition with Fourier i think should be awesome to timing the stock market..

[tsehayenegash]

I am lucy when I get you. I have 15 years temperature data upto 36km. I have some questions
1. how can I calculate the amplitude and phase of the annual and semi annual oscillation?
2. how can I determine the trend of the given temperature data?
3. How did you determine a, b, alpha and theta in your equation? please send me the code. I need to your email

[drewwilson]

Hi QuantPy, i've used your code and found the curve generated devolves into a fuzzy straight line if we remove the line omega = 2*np.pi/365.25. I've tried this because i'm interested in a series without a regular period.

Do you know why this is?