Prerequisites
Have Java 8 or greater installed on your machine. Open the command prompt and execute this to confirm it is installed:
java -version
Meanwhile I'm using Java 11
Download The Apache Kafka Binary
Go to https://kafka.apache.org/downloads, if you're using Scala on your machine then find the matched version, if not download any latest available version Kafka binary.
Let's say I got kafka_2.12-2.8.0.tgz. From here on, I will use my home directory C:\Users\dendi so you can use yours to follow along. I extracted it into C:\Users\dendi and renamed the folder from kafka_2.12-2.8.0 into kafka. So the complete path is C:\Users\dendi\kafka.
Here is the structure inside the kafka folder:
kafka
|__ bin
|__ config
|__ libs
|__ licenses
|__ logs
|__ site-docs
|__ LICENSE (file)
|__ NOTICE (file)
Provide Data Folder For Zookeeper and Kafka
In the kafka directory, let's create another folder named data and create two folders named zookeeper and kafka inside the data folder.
kafka
|__ bin
|__ config
|__ data
|__ kafka
|__ zookeeper
|__ libs
|__ licenses
|__ logs
|__ site-docs
|__ LICENSE (file)
|__ NOTICE (file)
Configure Zookeeper and Kafka Server
Find the config\zookeeper.properties file, open it with code editor and find the dataDir variable and fill it with:
...
dataDir=data/zookeeper
...
And then find the config\server.properties file, open it with code editor and find the log.dirs variable and fill it with:
...
log.dirs=data/kafka
...
If you're using app like Windows Terminal, Terminus or Hyper, it's very useful for managing consoles because we will open 4-5 consoles later.
Running Zookeeper
Now let's open our 1st command prompt, go to kafka directory, and execute this command:
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
if the result similar like this, then it's already running:
[2021-06-06 11:09:01,739] INFO Server environment:java.io.tmpdir=C:\Users\dendi\AppData\Local\Temp\ (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,739] INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,741] INFO Server environment:os.name=Windows 10 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,742] INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,743] INFO Server environment:os.version=10.0 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,743] INFO Server environment:user.name=dendi (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,743] INFO Server environment:user.home=C:\Users\dendi (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,744] INFO Server environment:user.dir=C:\Users\dendi\kafka (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,744] INFO Server environment:os.memory.free=492MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,745] INFO Server environment:os.memory.max=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,745] INFO Server environment:os.memory.total=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,747] INFO minSessionTimeout set to 6000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,748] INFO maxSessionTimeout set to 60000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,755] INFO Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 60000 datadir data\zookeeper\version-2 snapdir data\zookeeper\version-2 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-06-06 11:09:01,802] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2021-06-06 11:09:01,806] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-06-06 11:09:01,810] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-06-06 11:09:01,834] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase)
[2021-06-06 11:09:01,841] INFO Reading snapshot data\zookeeper\version-2\snapshot.0 (org.apache.zookeeper.server.persistence.FileSnap)
[2021-06-06 11:09:01,891] INFO Snapshotting: 0x8d to data\zookeeper\version-2\snapshot.8d (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-06-06 11:09:01,919] INFO PrepRequestProcessor (sid:0) started, reconfigEnabled=false (org.apache.zookeeper.server.PrepRequestProcessor)
[2021-06-06 11:09:01,931] INFO Using checkIntervalMs=60000 maxPerMinute=10000 (org.apache.zookeeper.server.ContainerManager)
Running Kafka
Let's open our 2nd command prompt, go to kafka directory and execute this command:
.\bin\windows\kafka-server-start.bat .\config\server.properties
Then your kafka server should be running, hopefully there is no error log in console result. But we will test if it actually running well or not by creating a topic in next step.
Creating Topic
Let's open our 3rd command prompt, go to kafka directory, we will create a simple topic named test with this command:
.\bin\windows\kafka-topics.bat --create --topic test --bootstrap-server localhost:9092
We could tell if it was successfuly created when we see this result:
Created topic test.
Running Consumer
We could use the same 3rd command prompt after we creating topic, now we will use the built-in kafka consumer. We already inside kafka folder, then execute this command to start the consumer for our created test topic:
.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning
After executing it, you will see absolutely nothing because the consumer is running and streaming the test topic and there is no data available at the moment. Don't close this command prompt, we will see if it's actually working if we produce a message.
Running Producer
Let's open our 4th command prompt, go to kafka directory, and let's run the built-in producer for the test topic by this command:
.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test
It will open interactive session that we can type anything in it and hit enter to produce the message to the topic. Try type anything and hit enter, see the typed message is showed in the consumer (3rd command prompt). If you see it, then the kafka is running perfectly and you're ready to use it for your project development.
version used:
java version "11.0.10" 2021-01-19 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.10+8-LTS-162)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.10+8-LTS-162, mixed mode)
kafka 2.12-2.8.0
Top comments (2)
For development I much rather use a docker setup. Easy to destroy and recreate. Or the Confluent cli, which is great when you need multiple components of the Confluent platform.
how can i use sqlite3 dll in widows 10 32 bit