AWS Kinesis Tutorial for Beginners [FULL COURSE in 65 mins]

00:09 - What the course will cover
00:54 - Set Up Work
05:43 - Kinesis Streams Theory
09:01 - SDK Vs KPL Theory
10:31 - Kinesis Data Streams Practical
12:03 - Kinesis SDK
15:54 - KPL Practical
22:26 - Lambda Consumer Theory
23:19 - Lambda Consumer Practical
31:45 - KCL Theory
32:43 - KCL Practical
37:24 - Studio Note Book Theory
38:19 - Studio Note Book Practical
44:22 - Kinesis Firehose Theory
45:17 - Kinesis Firehose Practical
53:27 - Kinesis Analytics Theory
54:03 - Kinesis Analytics Practical

In this course we take a look at AWS Kinesis in full using the console. In under 65mins we go from zero to hero. The course covers the AWS Kinesis Data Streams, AWS Kinesis Data Analytics and AWS Kinesis Firehose. Over the course of 65mins we create a fully functioning kinesis streaming application.

😎 About me
I have spent the last decade being immersed in the world of big data working as a consultant for some the globe's biggest companies.My journey into the world of data was not the most conventional. I started my career working as performance analyst in professional sport at the top level's of both rugby and football. I then transitioned into a career in data and computing. This journey culminated in the study of a Masters degree in Software development. Alongside many a professional certification in AWS and MS SQL Server.

[LeeLeeCode]

Thank you so much!! Very nice and detailed instructions. Much appreciated!

[atulkumar]

Awesome Going! Thanks for all your hard work in putting up these videos (y)

[xeroxeroxeroxeroxeroxeroxero]

Things to clean up when you're done:
- Delete the CloudFormation Template
- Empty and Delete the S3 Buckets housing your data
- Delete the lambda function created in lab 6

[shoooozzzz]

This video took me from Kinesis zero to hero! Thanks, mate. I now feel comfortable producing and consuming data from Kinesis <3

[milogodopre]

hI, I've been trying to do an exercise which consist on ingest data from an website (currencies), store them, and then show in a graphic the data collected, that's very simple to say but very difficult for me to do it, do you have any information I will really appreciate it. I have the API key from the source of data

[skviknesh]

Thanks so much! This was awesome!!!! Do more....

[akshayhs]

Hi Johnny. Your AWS Glue video was super helpful. Happy to be one of the first viewers of this video !!!

[shayankabasi]

Hi Johnny - Thanks for the nice tutorial, could you please check provide some work-through of the yml script that you are using in cloudformation section for resource creation.

[leamon]

Hi, is it possible to enable "dynamic partitioning" in kinesis firehose when integrating with cloudwatch subscription filter? I tried it, but it reports an error saying "Only UTF-8 encoded data is supported for dynamic partitioning records.". So now my workaround is Cloudwatch -> subscription filter -> lambda -> kinesis firehose -> S3, which I'm not sure if it's a best practice.

[yinwong]

Thank you for your awesome tutorial! But can you also teach us how to clean up?

[book_beats]

Hello from Ukraine, Johnny!
Thanks for your videos! At this point i`ve watched all videos on aws glue and kinesis, they are really great.

[barisdnmz]

nice tutorial, by the way how to clean up the environment that all set up?

[andrewaylward]

Love the Belfast accent! Good work Johnny!

[OttoTtt]

Guys, don't forget to shut down everything on Zeppelin (Johnny says about this in the video). I missed one query and got a $16 bill after one day. Better delete the whole Cloud Formation right after you finish this video.

[Chuukwudi]

Why did you not use Kinesis Data Generator ? Why did you write your own custom data generator? Is the data generator presently broken? I have been trying for the last 48 hours to get KDG to work but without success.

[sanyk]

This guy is awesome 😎, deserves more subscribers. Keep up your good work on data engineering. I am a full time software engineer planning to move on data engineering.

[jessemcmullen-crummey]

was going great until the maven compilation failed for me around 20:00. Got an error: " Fatal error compiling: Unable to make field private :(

[rakeshmali]

one question: do we need to mention which shard a consumer can listen/read to? or the customer automatically discovers the shard to read from?

[jeffgoodrich]

You're awesome Johnny C! I'm transitioning into data engineering in the next 6 months and your videos are great! I would like to study for the AWS solutions architect certs, and I will pay you for a custom project and learning plan. Let me know if you would be open for that, I am an engineer by training, so I do not need to be spoon fed everything, but I do want to take the most effective and efficient path. Thank you!

[erickjhormanromero]

KPL Practical is throwing the next exception Could not copy native binaries to temp directory