At Prenetics, our mission is to decentralise healthcare by offering affordable and accessible healthcare services across the pillars of Prevention, Diagnostic and Personalised Care to the public.
Engineering at Prenetics focuses on delivering robust, secure and scalable capabilities to our digital products and services. We deploy our API as microservices onto K8s (AWS EKS) and have implemented an EFK stack for centralised logging, monitoring and alerting. We have implemented CI/CD pipelines using CodePipeline and have been practicing Continuous Delivery.
Our core engineering principle is "You build it; you run it" and in order for our engineers to fully embrace this DevOps model, they must be able to easily build and test locally, thereby reducing the build and test feedback loop to the minimum. To this end we have developed the Dynamic Feature Environment.
Dynamic Feature Environment allows engineers to spin up/down a fully functional independent K8s cluster that runs our full set of microservices.
Precisely, a dynamic feature environment consists of an EC2 instance running Minikube and connecting to a small RDS instance that is created from a snapshot of our integration environment's RDS.
There are two EC2 Image Builder pipelines. The Minikube Ingress pipeline creates a AMI based on an Amazon Linux 2 base image. The pipeline installs Minikube, enables its ingress (minikube addons enable ingress), and also installs tools needed to deploy our microservices. The Minikube Service pipeline creates a AMI based on the output of Minikube Ingress and caches the latest Docker images (from AWS ECR) of our microservices. This pipeline is triggered daily.
There is a Lambda function (lambda-devlauncher) that our engineers can invoke to spin up an EC2 instance using the latest AMI from Minikube Service. This EC2 instance sits on the public subnet (with a public IP address) and is only accessible within our internal network (VPN). The EC2 instance's user data fetches the latest Docker images (from AWS ECR) of our microservices as well as the images of the services under test. The user data exposes the APIs, the cluster's K8s dashboard (minikube dashboard), and a centralised logviewer (minikube addons enable logviewer).
The Lambda function also creates a small RDS instance to be used by the feature environment. It is created from the latest snapshot of our integration environment's RDS. In practice we always have n+1 small RDS instances, where n is the number of feature environments running, so the time needed to spin up a dynamic feature environment is reduced to the minimum.
There is an EventBridge rule that invokes the lambda function daily to terminate any old feature environments and to create 1 small RDS instance for usage.
The time from invoking the lambda function to having the environment ready is around 12 minutes.
Top comments (0)