Recently, we needed to download a file from a Docker container. Since the file was not inside a bind mount, we couldn't directly access the remote server to get the file on the file system.
So we quickly found the docker cp utility to copy the file on our host and voila, we had the file on our computer. ☺️
Unfortunately, we didn't know the exact name of the container, as containers created by docker compose are, by default, suffixed with a randomly generated hexadecimal string, such as .eq6l28g5mwsenh0gvag7zcg0j. 😶🌫️
So, to obtain the full name of the container, we used the docker ps command, which lists the containers, along with its --format option to get only its full name.
If we combine these commands, we can have only one command to run to download our file.
It worked fine on our local computer, but the real file we were interested in was on a remote server... And this is where it all gets complicated for us. 🤮
Constraints
We wanted an automated solution where we could just provide some information and we would get the file. 🤖
The container was hosted on a random server from a pool of Docker machines. 🌈
We had to be privileged users to run certain Docker commands. 👑
After a lot of guesswork and mistakes, we finally managed to get a working solution under these constraints, but it took longer than expected. Let's dive into some of the issues we encountered. 😌
Issues
How to run sudo commands on a remote server
In my opinion, this step is the most complicated part of our final script. Please be patient with me as I explain some of the things we have done below. 🤢
To run multiple commands via SSH, we use a here document structure that lists all the commands we want to run on the remote host.
Since the variables were not replaced inside this structure, we encompassed it with the envsubst command.
To make the shell variables accessible to envsubst, we had to export them.
Since we wanted to capture the container name in a variable, we ended up splitting the commands because our one-line command didn't work well with sudo.
Since my user was not able to run Docker commands on this server, we had to gain root privileges. We have tried many (many) attempts and we managed to get a working solution by outputting our password to the first sudo -S command. The following commands don't need to have the password again. 🥺
For some reason still unknown to me, the value of the CONTAINER_NAME variable is an array... So, to access it, we had no choice but to use CONTAINER_NAME[0]... 🤷
As you may have noticed, we didn't need to pass our password when running the ssh command, because we are using SSH public keys. If you want to install SSH keys on your servers, check out my article on how to automate ssh-copy-id! 😉
How to transfer the file from the remote server to our host
This part was easy to find, using sshpass and scp commands.
Since we had previously installed an SSH public key on the server and we are using the same username on our host and the server, we can rewrite the previous command as follows.
Simpler, right? 😉
How to find the server where the application container runs
To browse over a cluster of servers, we have provided the list as an array, along with a simple switch case. It's always useful when we need it to have a switch case example in Bash , so I'm putting it here for future reference. 😋
Unfortunately, since we are not able to know on which server we need to transfer the file from, we have to run all the previous commands on each server... 😵💫
So, we get a lot of scp: /home/ranb2002/[...]: No such file or directory error because only one server has the wanted file... 😒
How to ensure the arguments are provided
Since our script could fail if some arguments were not supplied or were incorrect, we added some conditions to verify these cases.
Note : if the password is incorrect or if the user is not a privileged user, the script will still fail because we haven't taken some time to figure out how to prevent this.
Bonus : if for some reason you need to transform an uppercase value to its lowercase equivalent, here is the Bash command you can use: LOWER_ENVIRONMENT=$(echo "${ENVIRONMENT}" | awk '{print tolower($0)}'). 😜
Final solution
If we put together all the pieces together, our final script looks like below. 🧩
Hope this post is useful to you! Stay safe! 🍻
Top comments (0)
Subscribe
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)