An Intro to Markov chains with Python!

Tutorial introducing stochastic processes and Markov chains. Learn how to simulate a simple stochastic process, model a Markov chain simulation and code out the n-step transition matrix for a Markov chain.

GitHub repo containing the notebook under "Statistics with Python"

CONNECT:

|-Video Chapters-|
0:00 - Intro
0:24 - Definition of stochastic process
1:15 - Simulating a stochastic process with gambler's ruin
7:40 - Probability of gambler's ruin
11:26 - Definition of Markov chains
14:17 - Markov transition graph
15:38 - Coding a Markov chain simulation
23:15 - Memorylessness of Markov chains
24:05 - Simulating an n-step transition matrix
29:21 - Stationary distribution of a Markov chain
31:16 - 2-step transition matrix given an initial distribution
33:42 - References and additional learning

[dzuynguyen]

Really pulled me out of a dark place man. Thank you.

[spiderjerusalem]

This is one of the BEST Python tutorials I have ever seen, not just among Markov Chain tutorials, but among all tutorials. Great work!

[Raggaliamous]

Completely fantastic. Plus, I appreciate the ending references. Interested in making a basic weather simulator, I'm now searching a simple way of harvesting open weather data, to formulate the probabilities.

[TheDunningKrugerEffectisReal]

Good stuff, I'm going to save it to watch it again later.

[uniqueninja]

Thanks Adrian for this tutorial on HMM. Thanks for being so simple and practical on the concept.

[gmonkman]

Great explanation. I see the mathematical notation descriptors and just switch off, but this practical example is great. Respect for the credits as well.

[ddiq]

Thanks for including the reference for that book, definitely going to check that out!

[HonestADVexplorer]

Thank you very much! Clean Python code and clear explanations!

[breadandcheese]

Hands down the best example; Professors in universities please take note.

[entahgatau]

why after several iterations on n-step transitions, the rows become the columns and vice versa?

[carrotsofallages]

Hello, thanks for the video :). I wanted to ask what to do if you get a KeyError. Or how that error occurs in the first place

[PatrickNieto]

Is there a way to implement this if there were conditions on the transitions. I guess if it wasn't memorylessness. For example, what if I could only go to Seoul if I hadn't gone to Cairo or Paris the last 7 states?

[MuhammadRizkiAchirlita]

Thank you so much for this, really help me

But I want to ask something, How to make the transition graph with python also?

[RAFBI]

How did you show the plot of the markov chains?

[shhreyaanand]

Hi why isn't the blue graph showing?

[rodrigobotello]

Hello! when I use the code of the matrixpower, it says that matrix power is not defined, can you help me with it?

[rajeshmanjrekar]

while len(travel_sim) < 25:
city=np.random.choice(mc.iloc[mc.index.get_loc(city)].index, p=mc.iloc[mc.index.get_loc(city)])
travel_sim.append(city)

when i run this code it says p has to be one-dimensional as ERROR