DEV Community

John Ajera
John Ajera

Posted on

Datadog Agent Debugging Checklist for Docker Containers

#datadog #docker #monitoring #observability

📌 Datadog Agent Debugging Checklist for Docker Containers

If your Datadog Agent is running as a Docker container but your metrics or logs are missing from the Datadog UI, follow this guide to diagnose and resolve the issue efficiently.

1️⃣ Verify Datadog Agent Container is Running

Run the following command to check if the Datadog Agent container is active:

docker ps | grep datadog-agent
Enter fullscreen mode Exit fullscreen mode

✅ Expected Output:

CONTAINER ID   IMAGE                            COMMAND                  STATUS              NAMES
abcdef123456   gcr.io/datadoghq/agent:7         '/bin/entrypoint.sh'                  Up X minutes        datadog-agent
Enter fullscreen mode Exit fullscreen mode

🚨 If the container is not running, restart it:

docker-compose -f /opt/datadog/docker-compose.yml up -d
Enter fullscreen mode Exit fullscreen mode

2️⃣ Check Datadog Agent Logs

To review logs for potential issues:

docker logs --tail 50 datadog-agent
Enter fullscreen mode Exit fullscreen mode

✅ Look for messages related to:

  • API key authentication failures
  • Connection refused errors
  • Failure to reach Datadog servers

🚨 If errors appear, check environment variables in your docker-compose.yml.

3️⃣ Verify API Key & Connectivity

Ensure the API key is correctly set inside the container:

docker exec -it datadog-agent env | grep DD_API_KEY
Enter fullscreen mode Exit fullscreen mode

Then, test connectivity manually:

docker exec -it datadog-agent sh -c 'curl -v "https://api.datadoghq.com/api/v1/validate?api_key=$DD_API_KEY"'
Enter fullscreen mode Exit fullscreen mode

✅ Expected Response (200 OK):

{"valid":true}
Enter fullscreen mode Exit fullscreen mode

🚨 If response is 403 Forbidden or 401 Unauthorized, the API key is incorrect. Verify and update it in your docker-compose.yml.

4️⃣ Ensure Datadog Agent Can Reach Datadog Servers 

docker exec -it datadog-agent curl -v https://app.datadoghq.com
Enter fullscreen mode Exit fullscreen mode

✅ Expected Output: No Connection refused or Timeout errors.

🚨 If blocked, allow outbound traffic:

sudo ufw allow out to api.datadoghq.com
Enter fullscreen mode Exit fullscreen mode

5️⃣ Verify Datadog Agent is Sending Data 

docker exec -it datadog-agent agent status | grep -A10 "Forwarder"
Enter fullscreen mode Exit fullscreen mode

✅ Look for:

Transactions flushed: successfully: X, errors: 0
Enter fullscreen mode Exit fullscreen mode

🚨 If errors: X appears, the agent is not sending data. Restart it:

docker-compose -f /opt/datadog/docker-compose.yml restart datadog-agent
Enter fullscreen mode Exit fullscreen mode

6️⃣ Manually Send a Test Metric

If the agent is running but not visible, manually send a test metric:

docker exec -it datadog-agent agent telemetry send test.metric 1
Enter fullscreen mode Exit fullscreen mode

✅ Check in Datadog UI → Metrics Explorer and search for test.metric.

📌 Summary of Debugging Steps

Check Command Expected Result
Container is running `docker ps grep datadog-agent` Container is running
Check logs for errors docker logs --tail 50 datadog-agent No connection/auth errors
Verify API key `docker exec -it datadog-agent env grep DD_API_KEY` Correct API key is set
Test Datadog connectivity docker exec -it datadog-agent sh -c 'curl -v "https://api.datadoghq.com/api/v1/validate?api_key=$DD_API_KEY"' No network errors
Check agent network reachability docker exec -it datadog-agent curl -v https://app.datadoghq.com Successful connection
Ensure metrics are sent `docker exec -it datadog-agent agent status grep -A10 "Forwarder"` No errors in transactions
Manually send a test metric docker exec -it datadog-agent agent telemetry send test.metric 1 Appears in Datadog UI

🚀 Follow these steps to ensure your Docker-based Datadog Agent is properly configured! Need more help? Drop a comment or reach out! 🚀

Top comments (0)

profile
Datadog
 Promoted

Image of Datadog

The Future of AI, LLMs, and Observability on Google Cloud

Datadog sat down with Google’s Director of AI to discuss the current and future states of AI, ML, and LLMs on Google Cloud. Discover 7 key insights for technical leaders, covering everything from upskilling teams to observability best practices

Learn More