In this episode we will create workers pool, to run workloads on "our" servers. This process requires a few steps, let's see how quickly and easily we can add new worker to the app.
During this episode I will heavily use Spacelift documentation.
Create private key
First step is to create the key which wil be used to authenticate worker with application.
I'll use WSL2 for it.
openssl req -new -newkey rsa:4096 -nodes -keyout spacelift.key -out spacelift.csr
These files we need to setup our pool.
Create workers pool
Navigate to Worker pools
in the Spacelift app. You should see something similar like on the screen below.
Click New worker pool
.
Add the csr
file generated in previous step. Also, I changed space to root
. Yes, you can have pools set by space.
Please remember, that one worker is able to run one execution. If this is good or bad, I leave it to you, however it doesn't sound very appealing if I will use virtual machines, am I right? But! We can use this VM as containers platform and it starts to look a little bit better.
Ok, we can create our pool.
My pool is created and the file with token is downloaded automatically. Well, I am not fully sure here. I almost missed this download, my browser didn't show anything. I knew there will be a token, so I was careful, but I think it will be nice if Spacelift improve the UX here.
Create worker
I decided to go easy way and I run the EC2 on AWS using Spacelift image.
I've changed the default volume only. The type I selected is gp3 and I increased the size to 20G.
The provided AMI is not completed. I mean, it contains everything what we need to download and configure the launcher. Let's do it then!
First, we need to encode the key we generated earlier
cat spacelift.key | base64 -w 0
Now we can login to the newly created machine and do
sudo -i
wget https://downloads.spacelift.io/spacelift-launcher
install spacelift-launcher /usr/bin/spacelift-launcher
chmod +x /usr/bin/spacelift-launcher
export SPACELIFT_TOKEN="<token_from_file>"
export SPACELIFT_POOL_PRIVATE_KEY="key_from_encode_operation>"
/usr/bin/spacelift-launcher
What I do not like - to make the activity quickly I had to use root
account.
What we've just done is very unstable and should be treated as "let me check it" way only. When we do it properly, all these should be automated and run without any access from human side. In fact... Spacelift provides a repository with IaC to provision the infrastructure!
Let's see our workers pool
Yep, we are done!
The 'hmmm' moment
I need to dig deeper into the workers pool. I saw couple of things which made me think.
- What will happen if I stop my worker node?
- Logs for spacelift-launcher are set to
debug
by default (at least, this is what I see after quick check). - In logs I saw information about S3. And this is something what I have to explore. What S3? Why? Who owns it? I have some ideas, what is behind, but I have to confirm it.
Takeways
Spacelift allows us to create private pools of workers which can be used for different workloads. Creation is simple and management is quite effective.
Top comments (0)