Handling Missing Value in Time Series Data using Python

#timeseries #machinelearning #missingvalue

In time series typically handling missing data is not as straight forward as traditional ML algorithm. Apart from knowledge of data and domain there are multiple factors that come into play like seasonality, trend etc

In this video we will be analyzing a dataset and seeing multiple ways data can be imputed

[jaspreet]

23:46
To the viewers who already have prior knowledge of TSA, can directly visit 23:46 to skip the fundamentals and get to the main content of imputations strategies

[arunap]

Actually instead of taking the previous year value of temperature we are taking the next year value as we apply "- pd.offsets.DateOffset(year=-1)", should be either or "-

[vaisuliafu]

great video, thank you very much. I believe the 'TEMP_ROLLING' correction should be:

aq_df_imp['TEMP_ROLLING'] = aq_df_imp['TEMP'].fillna(aq_df_imp['TEMP'].rolling(3, min_periods=1).mean())

...where the window size is 3 because it includes the NA value itself.

[vipulgaurav]

Wonderful content sir! Loved the hands-on approach taught me how to handle a problem apart from theory! Thankyou, will be following the entire playlist and waiting for more!

[satyamrastogi]

Actively following your time series' Please upload more videos and in depth algorithms of time series forecasting. Thank a lot sir.

[thechaoticneuron]

Hello Sir,

Did you consciously impute 2015 Temperature missing values with 2016 values?

You gave an offset of "-1" years and subracted it from 2015 index, resulting in 2016 values.

[randyluong]

Dear, why the 1st NAN (2015-02-21 120000) + 2nd NAN (2015-02-21 140000) + 3rd NAN of TEMP_PREVY = [1.9, 4.9, 3] 31:20; while they are [3, 3.5, 1.2] respectively in 2014 30:02 ?

[atharvnikhare]

Could you please send me this notebook ? It will be really helpful.
Thanks

[TarekFansa]

very good explanation ! and a very nice youtuber !
That was helpful !
i am Mathe Tutor on Youtube
and learning Python
much Love from Germany !

[anujsaboo]

Thank you for making such elaborate videos on such a topic where not enough content is present. This really helps.

Just the last imputation of previous year has a mistake, the correct code would be below:

aq_df_imp['TIME_PYEAR'] = aq_df_imp.apply(lambda x: + if pd.isna(x['TEMP']) else x['TEMP'], axis=1)

[jaspreet]

Lots of details and concepts discussed. Thanks a lot for sharing.
Sir if you could share your GitHub repo for this codes in your description box, it would be a lot easy for us to replicate the codes, and to spend more time on consuming concepts and less time in re-writing coding.

[rabbilbhuiyan]

Excellent and clear video for manipulating missing values in time series data. Thanks a lot for such knowledge sharing !

[meanuj]

Thnak you very much for making concepts Lucid and clear..Could you please post code repo also..?

[oscarelysee]

when I make this x: + if pd.isna(x['TEMP']) else x['TEMP'], axis=1)


i have error KeyError: Timestamp('2017-03-29 00:00:00')

[sckeshari]

Very intuitive and clear video with understanding point of view.

[anshulsingh]

Very good approach to time series data, i have questions, out of the all approach which one you think fits best

[abinavprasad]

Hey, for the last code the substituting null with previous year values, m getting a timestamp error..I checked in a lot of places but couldn't find. Can u help me with it?

[abdulhannan-gohl]

Wow! just awesome, I rarely comment on any YouTube video but this series is a masterpiece, and I am looking forward to completing it. For those who are searching for a good time series playlist, this playlist is sufficient to learn from A to Z.

[chandanapriya]

ModuleNotFoundError Traceback (most recent call last)
in <module>
----> 1 import plotly.express as px
2
3 fig = px.line(delhi, x='date', y = 'pm25', title='pm25 with Slider')
4
5 fig.update_xaxes(

ModuleNotFoundError: No module named 'plotly'

I'm getting this error even though I installed 'pip install plotly'

[sangeethaezhumalai]

For the monthly sales data. If there is no sales in a particular month. How to handle this discontinuity?? Can we impute 0 there because there is no sales happened that time?