HOW TO INSTALL AND START APACHE KAFKA IN MAC | Start Apache Kafka In Mac | InterviewDOT

Показать описание

How to install and start Apache Kafka in Mac ?

Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java.

Apache Kafka is designed for high volume publish-subscribe messages and streams, meant to be durable, fast, and scalable.

At its essence, Kafka provides a durable message store, similar in some ways to a database, run in a server cluster, that stores streams of records in categories called TOPICS.

For example, by integrating diverse kinds of data such as likes, page clicks, searches, orders, shopping carts, and inventory, Apache Kafka can help feed data in real time into a predictive analytics engine to analyze customer behavior.

1. Download and install kafka using the following link

2. Unzip the kakfa file contents
cd kafka_2.11-1.1.0

3. Start the zookeeper and kafka server

...
Now start the Kafka server:

INFO Verifying properties
...

4. Create the topic for posting the messages

5.Send some messages using the kafka producer

Kafka comes with a command line client that will take input from a file or from standard input and send it out as messages to the Kafka cluster. By default, each line will be sent as a separate message.

Run the producer and then type a few messages into the console to send to the server.

my first message
my second message

6.Start the kafka consumer to view the messages

Kafka also has a command line consumer that will dump out messages to standard output.

my first message
my second message

Kafka – Install Apache Kafka on Mac

To install Apache Kafka on Mac, Java is the only prerequisite. First we shall look into the installation steps of Java and then we shall setup Apache Kafka and run it on the Mac.

Install Java
Click on JDK, check “Accept License Agreement” and download .dmg file for installation on Mac.
Double click on the downloaded file and proceed with the installation steps.
You have successfully installed Java. You may delete the .dmg file, as memory is costly on Mac.
You may verify the installation of Java on Mac, by running the following command on a Terminal.

java -version

Using Brew
If the mac users have Brew installed, they can use it for Kafka installation.

There are following steps used to install Apache Kafka using brew:

Step1: Use 'brew install kafka' and press enter key to install Kafka. Wait a while, and Kafka will be installed on the system. To test, use any kafka command.

Step2: If the 'kafka-' command is used, it will show files with and without '.sh' extension files. It is because the bin path is set in the '.bash_profile' file. So, edit the file again and remove that line from the end. Save, and exit.

Step3: Again use the command 'kafka-', now the files without '.sh' will appear.

Step4; But, to use kafka commands, the user needs to visit the kafka_directory always.

So, this is another way to install Apache Kafka on the system.

Kafka® is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.

Apache Kafka was originally developed as a LinkedIn message queue. As a project of the Apache Software Foundation, the open source software has developed into a powerful streaming platform with a wide range of functions.

Apache Kafka can be used to store and process data streams. The system is based on a distributed architecture that is optimized for high fault tolerance and scalability. This makes Apache Kafka suitable for large amounts of data and applications in the big data environment.

Via the provided interfaces it is possible to load data streams from third-party systems or to export them to these systems. The core component of the system is a distributed commit or transaction log. Apache Kafka's advantages include low latency, high throughput and the ability to process real-time streams.