Apache Kafka - Installing Kafka in Cloud

Spark Programming and Azure Databricks ILT Master Class by Prashant Kumar Pandey - Fill out the google form for Course inquiry.
-------------------------------------------------------------------
Data Engineering using is one of the highest-paid jobs of today.
It is going to remain in the top IT skills forever.

Are you in database development, data warehousing, ETL tools, data analysis, SQL, PL/QL development?
I have a well-crafted success path for you.
I will help you get prepared for the data engineer and solution architect role depending on your profile and experience.
We created a course that takes you deep into core data engineering technology and masters it.

If you are a working professional:
1. Aspiring to become a data engineer.
2. Change your career to data engineering.
3. Grow your data engineering career.
4. Get Databricks Spark Certification.
5. Crack the Spark Data Engineering interviews.

ScholarNest is offering a one-stop integrated Learning Path.
The course is open for registration.

The course delivers an example-driven approach and project-based learning.
You will be practicing the skills using MCQ, Coding Exercises, and Capstone Projects.
The course comes with the following integrated services.
1. Technical support and Doubt Clarification
2. Live Project Discussion
3. Resume Building
4. Interview Preparation
5. Mock Interviews

Course Duration: 6 Months
Course Prerequisite: Programming and SQL Knowledge
Target Audience: Working Professionals
Batch start: Registration Started
Fill out the below form for more details and course inquiries.

--------------------------------------------------------------------------
Best place to learn Data engineering, Bigdata, Apache Spark, Databricks, Apache Kafka, Confluent Cloud, AWS Cloud Computing, Azure Cloud, Google Cloud - Self-paced, Instructor-led, Certification courses, and practice tests.
========================================================

SPARK COURSES
-----------------------------

KAFKA COURSES
--------------------------------

AWS CLOUD
------------------------

PYTHON
------------------

========================================
We are also available on the Udemy Platform
Check out the below link for our Courses on Udemy

=======================================
You can also find us on Oreilly Learning

=========================================
Follow us on Social Media

========================================

[ScholarNest]

Want to learn more Big Data Technology courses. You can get lifetime access to our courses on the Udemy platform. Visit the below link for Discounts and Coupon Code.

[pranavsindhavad]

Must say Prashant has an amazing knack for teaching and making complex topics more simpler and understandable to all audiences, please keep up the good work Prashant, wish you all the best and keep it coming.

[foodandsunshine]

Very good videos. Loved it. The sound quality is good as well. Thanks Learning Journal!

[AshishGupta-cbhd]

Man, you are awesome....please keep sharing the knowledge

[SahilRallySuperStar]

1. We can have a consumer facade in the consumer group which talks to Kafka . The only responsibility it has is to distribute workload among other consumers. This way we can autoscale consumers ?

2. Don't we have autoscaling mechanism for Brokers ? Seems like we need to provide number of brokers at the start of server. ???

[meirzeevi]

One of the best tutorials
Thank you for sharing

[babujayaraman]

we have to change the command to kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test if you use version older than 2.2 you should use --zookeeper option and pass connection string to zookeeper

[anurag]

I like your videos, I need Kafka version is 2.8. Please share link for same.

[sharanappagr]

When i ran the command got below error kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
Exception in thread "main" bootstrap-server is not a recognized option

[hussamcheema]

I'm getting this error on the fist command:
WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092
) could not be established. Broker may not be available.

[vijaymahendrasinghrana]

Any guidline or documentation to install kafka on openshift3.11.

[appy]

nice work sir - great work sharing information :-)

[balaji]

Sir iam a beginner in Kafka. I need to implement it in my project.

We can expect some 200k records in the mysql user table. I have to track User events like User Creation, User Address updation, User Email address updation etc....

I have to integrate Kafka producer here. The important requirement is we have to maintain the ordering of events.

I have 2 clarifications sir,
1. Is it to good to have one topic called Users with multiple partions based on the Users role. The idea of having more partitions is to make the solution more scalable.
2. How many Kafka brokers we can have in Accepatnce and Development environment to make this an optimal solution.

[Jay-ibii]

Hi sir, your videos are amazing. Could you also make a tutorial on Apache kafka integration with Apache flink and streaming it.?

[MrManishaman]

Sir i am unable to start kafka-server it says permission denies. I am using RHEL server in my local machine

[sushmashamsundar]

sir am getting timed out exception while creating topics., no solution in goole also. pls help.


Exception in thread "main" Timed out waiting for connection while in state: CONNECTING

[neeharikamandarapu]

Thank you so much sir for the Kafka tutorial. Could you please do a tutorial on integration of Kafka with Spark streaming

[rikuntri]

Hi sir, can you please make a video on how to install various big data components in Google data proc e.g nifi or flink

[veerukbr]

What is the difference between Kafka streaming and spark streaming ?


How to stream customized website data using Kafka ??

[NishantKumar-xxbq]

i am using kafka 2.1.1 on GCP but i am unable to create a producer. i am getting below error.
"zookeeper is not a recognized option". i tried with both zookeeper and bootstrap-server. it didn't work