DEV Community

kay-adamof
kay-adamof

Posted on

Letting two Docker containers talk to each other

Suppose we have two Docker containers, A and B. After processing the data in the container A, we send the data to the container B. The container B performs another processing and returns the data to the container A. I will explain how to achieve this sequence of operations with an actual example.

Before explaining, we will identify some important points.

  • The Address for accessing the container B must be specified correctly. It's vital.

    • When two containers are launched in the docker-compose.yml file, they are assigned the same network.
    • Containers in the same network can communicate with each other using each other's service name and port number in the network.
  • container A is started after container B is started.Be aware of the order of running containers

    • The container B, as a server, receives data from the container A, so it must be started first.
    • Specify the order of startup with depends_on in the docker-compose.yml file.
  • Wait a certain amount of time for a value to be returned from the container B.

    • If you don't set the wait for a certain period of time, the container will finish running before the value is returned.

Now let's look at the implementation flow of each container.

Creating the container A

The role of the container A is to obtain a transcription of a YouTube video.

  • Specify the URL of the YouTube video

  • Obtain the id from the URL.

  • Based on the id, use a Python library to obtain a transcription of the video.

  • Send the text data of the transcription to the container B.

  • Receive the text data processed separately in the container B.

  • Save the received text to a file.

The script that implements the above flow is as follows

import urllib.parse
from youtube_transcript_api import YouTubeTranscriptApi
import requests

url = "https://www.youtube.com/watch?v=CJjSOzb0IYs"

parsed_url = urllib.parse.urlparse(url)
query_params = urllib.parse.parse_qs(parsed_url.query)
video_id = query_params.get("v") # returns an array including the value next to "v=..."

if video_id:.
    video_id = video_id[0].
    transcript_list = YouTubeTranscriptApi.get_transcript(video_id)
    transcription = ""
    for t in transcript_list:.
        transcription += " "+ t["text"]

    with open("transcription.txt", "w") as file: file.write(transcription)
        file.write(transcription)

    # Send the request and wait for the response until 5 sec
    try: # Send the request and wait for the response until 5 sec
        posted = requests.post("http://punctuate_text:5000/punctuate", data={"text": transcription}, timeout=5)
        posted.raise_for_status() # Raise an exception for non-2xx responses

        if posted.status_code == 200: with open("output.txt")
            with open("output.txt", "w") as file: file.write(posted.text)
                file.write(posted.text)
    except requests.exceptions.RequestException as e:
        print("Error occurred:", e)
else: print("No video ID found.
    print("No video ID found.")
Enter fullscreen mode Exit fullscreen mode

⭐️ The most noteworthy line is the following one.

posted = requests.post("http://punctuate_text:5000/punctuate", data={"text": transcription}, timeout=5)
Enter fullscreen mode Exit fullscreen mode
  • punctuate_text : The service name of the container B specified in the docker-compose file that will appear later.

  • 5000 : port number in the network, also specified in the docker-compose file.

  • /punctuate : The root address specified in the Flask server of the container B.

The Dockerfile for container creation is as follows.

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python3", "app.py"].
Enter fullscreen mode Exit fullscreen mode

Creating container B

The role of the container B is to punctuate the transcription sent from the container A.

  • Create a server using Pytyon's Flask.

  • Create a route for creating the punctuation.

  • Receive text data POSTed to the route.

  • Punctuate the text using SpaCy, Python's natural language processing library.

  • Returns the punctuated text.

  • Specify the server port number 5000.

The script that implements the above flow is as follows

from flask import Flask, request
import spacy

app = Flask(__name__)
nlp = spacy.load("en_core_web_sm")


@app.route('/punctuate', methods=['POST'])
def punctuate_text():.
    text = request.form.get('text')
    doc = nlp(text)
    punctuated_text = ""
    for sentence in doc.sentences:.
        punctuated_text += sentence.text + ". "
    return punctuated_text

if __name__ == '__main__':.
    app.run(host='0.0.0.0', port=5000)
Enter fullscreen mode Exit fullscreen mode

The Dockerfile for container creation is as follows.

FROM python:3.9-slim
WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -U pip setuptools wheel
RUN pip install --no-cache-dir -r requirements.txt
RUN python -m spacy download en_core_web_sm

COPY . .

CMD ["python3", "app.py"].
Enter fullscreen mode Exit fullscreen mode

Creating the Docker Compose file

The entire file should look like this

# docker-compose.dev.yml

version: "3.8"

services: # Docker-Compose.dev.yml version: "3.8
  # Container A
  services: # Container A
    build: .
      context: . /python
    container_name: transcription
    depends_on: - depends_on: .
      -. /python container_name: .
    volumes: .
      - . /python:/app

  # Container B
  punctuate_text: - volumes: .
    build: .
      context: . /punctuate
    container_name: punctuate
    ports: 8000:5000
      - 8000:5000

Enter fullscreen mode Exit fullscreen mode

A Important part of the container description.

depends_on:: ``docker
- punctuate_text
Enter fullscreen mode Exit fullscreen mode
  • By specifying the service name of the container B, you can start the container A after the container B.

Important parts of the container B description.

ports:
  - 8000:5000
Enter fullscreen mode Exit fullscreen mode
  • The 5000 number is the number in the network, the same as the port number of the Flask server in the container B.

Run docker-compose command

Start the two containers.

## docker-compose -f docker-compose.dev.yml up -d
Enter fullscreen mode Exit fullscreen mode

Summary

  • The address to access the containers is determined by the service name and port number in the docker-compose file.
  • Pay attention to the order in which containers are started.
  • Set the time to wait for a response from the server.

Conclusion

It took me a while to solve the problem because I didn't know how Docker was resolving the address dynamically.

You can see the source code on GitHub

Top comments (0)