Confluent is the primary sponsor of Apache Kafka and not surprisingly they provide great Docker images for Kafka (and its dependency, ZooKeeper) via their Confluent Platform.
The following docker-compose.yml file spins up a single-node Kafka "cluster", but the INTERNAL/EXTERNAL bits enable multi-node clusters to be declared with a small amount of changes.
version: "3"
services:
zk:
image: confluentinc/cp-zookeeper:${CP_VERSION:-5.5.3}
environment:
JAVA_TOOL_OPTIONS: -Xmx256m
ZOOKEEPER_CLIENT_PORT: 2181
volumes:
- zk:/var/lib/zookeeper/data
kafka:
image: confluentinc/cp-kafka:${CP_VERSION:-5.5.3}
environment:
JAVA_TOOL_OPTIONS: -Xmx512m
KAFKA_ZOOKEEPER_CONNECT: zk:2181
KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:9092,INTERNAL://kafka:9192
KAFKA_LISTENERS: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:9192
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_LOG_RETENTION_HOURS: 1
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
# some loggers are set to TRACE by default...and they're noisy
KAFKA_LOG4J_LOGGERS: kafka.controller=INFO,state.change.logger=INFO
volumes:
- kafka:/var/lib/kafka/data
ports:
- 9092:9092
depends_on:
- zk
volumes:
zk:
kafka:
Docker Compose also makes it easy to exec the usual helper scripts provided by the image, such as listing topics:
docker-compose exec kafka \
kafka-topics --bootstrap-server localhost:9092 \
--list
and consuming from a topic:
docker-compose exec kafka \
kafka-console-consumer --bootstrap-server localhost:9092 \
--topic your-topic
Confluent has extensive documentation on the images used here.
Also, make sure to update or pass in the CP_VERSION appropriate from the newest tags published.
Top comments (0)