I was doing some troubleshooting between two services recently and wanting to poke around to see what was happening in the REST calls between them. Normally I’d reach for tcpdump
to do this but imagine my horror when I saw:
root@ksqldb-server:/# tcpdump
bash: tcpdump: command not found
Of course, being Docker containers and being built with the correct philosophy of not including the kitchen sink, tcpdump
wasn’t present.
My erstwhile companion on my IT career, Google, soon pointed me to the answer courtesy of Philippe Bogaerts in his blog post How to TCPdump effectively in Docker. Here I’ll shamelessly plagiarise the salient points and apply them to my Docker situation.
First up, you can build Docker images using Here Documents which is pretty cool:
docker build -t tcpdump - <<EOF
FROM ubuntu
RUN apt-get update && apt-get install -y tcpdump
CMD tcpdump -i eth0
EOF
So that’s built me a local image with tcpdump
on:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
tcpdump latest eebe12b8051f 31 minutes ago 98.3MB
I’m interested in the communication between these two containers:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
20a7bb264c82 confluentinc/ksqldb-server:0.6.0 "/usr/bin/docker/run" 17 hours ago Up 42 minutes 0.0.0.0:8088->8088/tcp ksqldb-server
df2be147f1ef confluentinc/cp-kafka-connect:5.4.0-beta1 "bash -c 'echo \"Inst…" 17 hours ago Up 17 hours (healthy) 0.0.0.0:8083->8083/tcp, 9092/tcp kafka-connect-01
Using Docker’s ability to run a container that attaches to the network of another with the --network=container:<container_name>
option we can now run tcpdump
"piggybacked" on my container of interest:
docker run --tty --net=container:ksqldb-server tcpdump
From this we can see all the network chatter going on:
$ docker run --tty --net=container:ksqldb-server tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:11:11.547672 IP kafka.ksqldb-twitter_default.29092 > ksqldb-server.59926: Flags [P.], seq 373631892:373631900, ack 22720794, win 15552, options [nop,nop,TS val 65125468 ecr 65125418], length 8
11:11:11.547806 IP kafka.ksqldb-twitter_default.29092 > ksqldb-server.59926: Flags [P.], seq 8:22, ack 1, win 15552, options [nop,nop,TS val 65125468 ecr 65125418], length 14
11:11:11.547955 IP ksqldb-server.59926 > kafka.ksqldb-twitter_default.29092: Flags [.], ack 22, win 32044, options [nop,nop,TS val 65125468 ecr 65125468], length 0[…]
What we can see here is our container (ksqlDB server) talking to the Kafka broker kafka.ksqldb-twitter_default.29092
.
kafka
is the broker’s hostnameksqldb-twitter_default
the name of the Docker network (that in this case Docker Compose has created)29092
the Kafka broker’s listener port.
I’m not interested in this traffic, so instead of using the default runtime arguments for tcpdump
that were defined in the CMD
section when we built the Docker image above, we can override it:
docker run --tty --net=container:ksqldb-server tcpdump tcpdump -N -A 'port 8083'
The first
tcpdump
is the name of the Docker image to run-
The second
tcpdump
overrides the command to execute (bypassing theCMD
default of the image), and callstcpdump
with arguments:-
-N
- Don’t include the domain qualifications (in this case theksqldb-twitter_default
network name) -
-A
- render in ASCII -
'port 8083'
- Only show traffic on port 8083
-
Now we get to see the stuff we’re interested in, like the ksqlDB server sending a REST call to create a connector
11:15:02.394620 IP ksqldb-server.52086 > kafka-connect-01.8083: Flags [P.], seq 1:898, ack 1, win 229, options [nop,nop,TS val 65148580 ecr 65148580], length 897
E.....@.@..9.........v...^..7..............
........POST /connectors HTTP/1.1
Content-Length: 662
Content-Type: application/json; charset=UTF-8
Host: kafka-connect-01:8083
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_222)
Accept-Encoding: gzip,deflate
{"name":"SOURCE_TWITTER_01","config":{"connector.class":"com.github.jcustenborder.kafka.connect.twitter.TwitterSourceConnector","twitter.oauth.accessToken":"${file:/data/credentials.pro
[…]
My thanks to Philippe Bogaerts for his excellent blog post How to TCPdump effectively in Docker on which this one is entirely based.
Top comments (0)