Step-by-Step Guide: Setting Up Elasticsearch and Kibana with Docker

Setting up Elasticsearch and Kibana using Docker on a Mac is a straightforward process. Follow this step-by-step guide to integrate and visualize your developer metrics data.

Prerequisites: Ensure you have Docker installed on your Mac. You can download it from the Docker website.

Step 1: Create a docker-compose.yml File

Create a directory for your project and within that directory, create a docker-compose.yml file with the following content:

version: '7.16.3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
    ports:
      - "9200:9200"
    volumes:
      - esdata:/usr/share/elasticsearch/data

  kibana:
    image: docker.elastic.co/kibana/kibana:7.16.3
    container_name: kibana
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

volumes:
  esdata:
    driver: local

This configuration sets up both Elasticsearch and Kibana, with Elasticsearch running on port 9200 and Kibana on port 5601.

Step 2: Start the Docker Containers

Open your terminal, navigate to the directory containing your docker-compose.yml file, and run:

docker-compose up -d

This command will start both Elasticsearch and Kibana in the background.

Step 3: Index the JSON Data into Elasticsearch

Consider this is only an example JSON just to explain the integration.

Save the following JSON data into a file named developer_metrics.json:

{
  "developer": "John Doe",
  "metrics": {
    "time_spent_coding": 120,
    "review_time": 35,
    "planned_overhead_time": 20,
    "untracked_time": 25,
    "effective_work": 15,
    "other_metrics": {
      "opened_pull_requests": 18,
      "merged_pull_requests": 15,
      "closed_pull_requests": 3,
      "created_feature_branches": 10,
      "lines_of_code": 2000,
      "inactive_days": 2
    }
  },
  "dates": {
    "start_date": "2024-05-01",
    "end_date": "2024-05-28"
  }
}

Use curl to index this data into Elasticsearch:

curl -X POST "localhost:9200/developer_metrics/_doc/1?pretty" -H 'Content-Type: application/json' -d @developer_metrics.json

Step 4: Create a Mapping for the Index

To ensure Elasticsearch correctly interprets our data fields, we need to define a mapping for our index.

Run the following curl command to create the mapping:

curl -X PUT "localhost:9200/developer_metrics?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "developer": { "type": "keyword" },
      "metrics": {
        "properties": {
          "time_spent_coding": { "type": "integer" },
          "review_time": { "type": "integer" },
          "planned_overhead_time": { "type": "integer" },
          "untracked_time": { "type": "integer" },
          "effective_work": { "type": "integer" },
          "other_metrics": {
            "properties": {
              "opened_pull_requests": { "type": "integer" },
              "merged_pull_requests": { "type": "integer" },
              "closed_pull_requests": { "type": "integer" },
              "created_feature_branches": { "type": "integer" },
              "lines_of_code": { "type": "integer" },
              "inactive_days": { "type": "integer" }
            }
          }
        }
      },
      "dates": {
        "properties": {
          "start_date": { "type": "date" },
          "end_date": { "type": "date" }
        }
      }
    }
  }
}'

Step 5: Visualize the Data in Kibana

With our data indexed, it’s time to create visualizations in Kibana.

Creating an Index Pattern

1. Open Kibana in your browser by navigating to http://localhost:5601.
2. Navigate to Stack Management -> Index Patterns.
3. Click Create index pattern and specify developer_metrics as the index pattern.
4. Follow the prompts to complete the setup.

Creating Visualizations

1. Go to the Visualize Library.
2. Click Create visualization and choose a visualization type (e.g., bar chart, line chart).
3. Select the developer_metrics index pattern.
4. Configure your visualization by selecting fields like time_spent_coding, opened_pull_requests, etc.

Building a Dashboard

1. Navigate to Dashboard.
2. Click Create new dashboard.
3. Add your visualizations to the dashboard.

Conclusion

Using Docker to set up Elasticsearch and Kibana on a Mac simplifies the process and ensures a consistent environment. By following these steps, you can easily visualize and analyze developer metrics data, helping you make informed decisions and track performance effectively.

This setup can be customized and expanded to suit your specific needs. Start leveraging these powerful tools today to enhance your project’s efficiency and transparency.

References

https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html

https://www.elastic.co/guide/en/kibana/current/introduction.html