I'm curious about how you guys are using containers (be it Docker, Kubernetes or any other platform) and tools (docker-compose etc...) on a daily basis.
What are the cool things you like to do with containers when developing? And on the opposite side, what are your current frustrations when working with containers?
Top comments (6)
In our development organization we have 5 in-house clusters. Our newest product consists of ~15 helm v3 charts corresponding to about 22 pods. Some of the charts have multiple deployments.) Our container and chart repo is an in-house harbor repo.
Our -ui apps were using nginx until recently. One day, before the switchover, I realized I could co-opt one of the containers of a ui pod by getting on the clusters worker node that it was running on and locating the container using docker commands, stop the currently running container (and just let kubernetes restart it)....but then doing a docker commit on the container that I stopped.
I then tagged it for the library project on harbor and pushed it there. By running that one I started finding that I could exec into it, install whatever I wanted, and re-commit the new modified container.
Eventually I installed pwsh, and kubectl and wrote a small script to query the status of all 5 of our clusters and all of the pods running on them, and use powershell's convertto-html to put that where nginx could serve it up.
Added a few features so that if any node went NotReady it would send an email, and a feature to retain its 5 minute index.html updates to an archive.
(This is my first post here, btw)... :)
It's great if you don't want to install things like composer and node on your machine too. Switching versions is incredibly easy and requires littler extra storage. Simple set up aliases and boom. You've got your fav tools anywhere, regardless of operating system or project. You can even bind things like node and composer versions to a project, making onboarding ten times easier if you have a policy in place!
This right here. Needed a backup solution to snapshot a running DB that literally took hours to import as a MySQL dump. Grabbed an rsync image, spin down the db image, mount the data volume to the rsync image and pulled the whole db out in 8 minutes.
Restored the snapshot in 5 minutes. So much nicer.
I use FreeBSD Jails for my containerization platform.
Instead of doing any dev locally at all, this is how I've done things since before I can remember...
Host Server OS has NFS/Samba file share for the working directory. I mount that remotely on whatever workstation I happen to be on at the time (this allows me to jump workstations without missing a beat!)
That share is then nullfs mounted into a Jail to run the code.
That container is setup 100% identical to how production runs, except the FS is nullfs instead of a local ZFS dataset.
There is NEVER any worry about "works on my system" syndrome, because the exact versions of FreeBSD, Nginx, PHP, MariaDB, and all related utilities are literally 100% identical, even down to running on the same hardware.
ZFS and Jails also enable some wickedly cool things, too. I can easily snapshot a jail, and then replicate that exact jail state on any other host. There are no custom or proprietary tools. ZFS does all of this natively, and has a very well supported ecosystem.
This also means that changes can be tested and rolled back effortlessly, not just in code changes, but INFRASTRUCTURE changes! Want to upgrade a database to a new version? But then IT BREAKS? Just roll back to the previous ZFS snapshot. This is done in literally seconds.
There are tons of prototyping advantages to this, its just endless possibilities with very easy to use tooling compared to what's available on other OSes. :)
In our company we run everything in containers (except our monolith which relies on Microsoft IIS, which we're currently migrating)
Otherwise we have like 180 different containers with our microservices.
(We created an in house tool to facilitate these for the developers)
In my personal business I run a kubernetes cluster which I feel simplifies my developer experience.
There i collaborate with a couple of developers which makes it easier for the front ender to spin up everything on their specific machine.
We have a couple of use cases.
First we build everything in containers to ensure all devs use the same versions of build dependencies for all projects, be it node, go, java or python. The Dockerfile for the build environment is checked in with the code, and also used in our build servers.
Secondly we package our tools in docker containers to get them available in lots of different teams, no matter what base technology they are using, and also no matter what os they are using.
For example tools for working with our secrets manager or deployment tools which has prepackaged kubectl and helm and gets configs injected either from host on devs computers or as env on build servers.
We also like to develop selenium e2e tests towards browsers running in containers, before expanding to run on our device farm.
And lastly we host quite a few kubernetes clusters onprem that are being used both for dev, test, stage, and prod for applications from different teams.
We are still not full-fledged devops yet, but working our way there. Our current setup help alot in the journey as ops help during development and dev starts to worry about how things are hosted and monitored with help of ops. The deployment tool, which is build on helm with some extra metadata and custom extentions, is used to deploy everywhere. So the devs gets freedom of using what ever tech they want on their side as long as its packaged in docker containers and can be deployed with the ”extended” helm chart using our deploy tool. Then our ops just need to handle everything in the same way once its running in our clusters.
…oh yeah… almost forgot! We also have migrated all our custom tool servers to docker and deployed in our dev-kubernetes clusters. Lots of different stuff, from onboarding tools to deamonsets helping to keep logs to max 29days.