Deep Dive into Elastic Cloud Enterprise (ECE)

Elastic Cloud Enterprise (ECE) is a significant innovation from Elastic, designed to simplify the deployment, management, and scaling of Elasticsearch clusters in various environments. ECE provides a unified, efficient platform for handling Elasticsearch clusters on-premises, in the cloud, or in hybrid setups. It offers a centralized orchestration layer, enhancing operational efficiency through automation, monitoring, and seamless scaling. ECE empowers organizations to utilize the full Elastic Stack (Elasticsearch, Kibana, Beats, Logstash) for robust search, observability, and security solutions while maintaining control over their data.

In this blog, we’ll delve into ECE’s architecture, setup, features, and deployment strategies.

*Components Roles in ECE *

1. Director: The Director is the brain of the ECE platform. It orchestrates and manages the entire ECE deployment, ensuring that all components are working together seamlessly.
2. Coordinator: The Coordinator acts as an intermediary between users and the ECE infrastructure. It is responsible for handling API requests and distributing them efficiently across the system.
3. Proxy: The Proxy handles network traffic between the various ECE components and between ECE and external clients. It ensures secure, efficient, and reliable communication.
4. Allocator: The Allocator is the component responsible for provisioning and managing the physical resources required for running Elasticsearch, Logstash, and Kibana (the ELK stack) clusters. '

*Hardware Requirements *

For this setup, it requires three hosts such as:

1. Nginx as Reverse Proxy, which handles user requests from internet.
2. Director and Coordinator
3. Proxy and Allocator

Prerequisites:

1. Ubuntu 20.04 Operating System
2. Kernel Version: 4.15.x or later on Ubuntu.
3. Elastic Cloud Enterprise is not supported on Linux distributions that use cgroups version.
4. Port 443 should be allowed through the firewall to the reverse proxy host.
5. (Optional) If you plan to deploy with your own domain name in the deployment. You should add the relevant domain name in the DNS configuration.
6. (Optional) If you plan to use your own domain name. You should generate Wildcard SSL certificate for the following URLs:

Note: consider example.com is your domain name.

cloudui.example.com (for accessing Cloud UI)
*.kibana.example.com (for accessing Kibana UI)
*.fleet.kibana.example.com (for Fleet server)
*.apm.kibana.example.com (for APM)

*Note: Fleet and APM endpoint URLs will be created as subdomains of the Kibana endpoint URL. *

*Host Machine Configuration: *

Create a user named elastic with sudo privilege.

`useradd elastic `

*Creating Filesystem and Mount point for ECE Installation *

1. For this Deployment we require XFS filesystem.

sudo mkfs.xfs /dev/< disk path>

1. Create the /mnt/data/ directory as a mount point.

sudo install -o $USER -g $USER -d -m 700 /mnt/data

3/ Add an entry to the /etc/fstab file for the new XFS volume. The default filesystem path used by Elastic Cloud Enterprise is /mnt/data.

/dev/<disk path> /mnt/data xfs defaults,nofail,x-systemd.automount,prjquota,pquota 0 2


1. Regenerate the mount files

sudo systemctl daemon-reload
sudo systemctl restart local-fs.target

*Docker Installation *

Install Docker LTS version 24.0 for Ubuntu 20.04 or 22.04.

Ref Doc: https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository

*Update the configurations settings *

1. Stop the Docker service:

sudo systemctl stop docker

1. Enable cgroup accounting for memory and swap space.

In the /etc/default/grub file, ensure that the GRUB_CMDLINE_LINUX= variable includes these values:

cgroup_enable=memory swapaccount=1 cgroup.memory=nokmem

1. Update your Grub configuration.

sudo update-grub

1. Configure kernel parameters.

cat <<EOF | sudo tee -a /etc/sysctl.conf

The following is an example of the configuration file fields:

#Required by Elasticsearch
vm.max_map_count=262144
#enable forwarding so the Docker networking works as expected
net.ipv4.ip_forward=1
#Decrease the maximum number of TCP retransmissions to 5 as recommended for Elasticsearch TCP retransmission timeout.
#See
https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config-tcpretries.html
net.ipv4.tcp_retries2=5
#Make sure the host doesn't swap too early
vm.swappiness=1
EOF`

1. Apply the settings.

sudo sysctl –p

1. Adjust the system limits.

Add the following configuration values to the /etc/security/limits.conf

•            soft    nofile         1024000 
  

•            hard    nofile         1024000 
  

•            soft    memlock        unlimited 
  

•            hard    memlock        unlimited 
  

elastic soft nofile 1024000

elastic hard nofile 1024000

elastic soft memlock unlimited

elastic hard memlock unlimited

elastic soft nproc unlimited

elastic hard nproc unlimited

root soft nofile 1024000

root hard nofile 1024000

root soft memlock unlimited

*Configure the Docker daemon options *

1. Update /etc/systemd/system/docker.service.d/docker.conf. If the file path and file do not exist, create them first.

[Unit]
Description=Docker Service
After=multi-user.target
[Service]
Environment="DOCKER_OPTS=-H unix:///run/docker.sock --data-root /mnt/data/docker --storage-driver=overlay2 --bip=172.17.42.1/16 --raw-logs --log-opt max-size=500m --log-opt max-file=10 --icc=false"
ExecStart=
ExecStart=/usr/bin/dockerd $DOCKER_OPTS

2.Apply the updated Docker daemon configuration:

sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl enable docker

1. Enable your user to communicate with the Docker subsystem by adding it to the docker group:

sudo usermod -aG docker $USER

1. Tune your network settings.

Create a 70-cloudenterprise.conf file in the /etc/sysctl.d/ file path that includes these network settings:

cat << SETTINGS | sudo tee /etc/sysctl.d/70-cloudenterprise.conf net.ipv4.tcp_max_syn_backlog=65536 net.core.somaxconn=32768 net.core.netdev_max_backlog=32768 SETTINGS

5.Reboot your system to ensure that all configuration changes take effect.

sudo reboot -f

1. After rebooting, verify that your Docker settings persist as expected: sudo docker info | grep Root

If the command returns Docker Root Dir: /mnt/data/docker, then your changes were applied successfully and persist as expected.

Note: Repeat the same steps to configure the other host that we are going it for the role Allocator and Proxy.

This is the high-level overview of the default installation of ECE. The below picture depicts the admin user using Cloud UI for creating and managing deployments. The end users will be accessing Kibana via Proxy, and all the ELK clusters will be running in the Allocator. The proxy communicates and routes the traffic to the allocator for the relevant ELK cluster.

Deploying a Medium Installation

*Install Elastic Cloud Enterprise on the Director and Coordinator Host *

1. Initially, the installation script installs all the roles in the first host, later we will separate the roles on different hosts.

bash <(curl -fsSL https://download.elastic.co/cloud/elastic-cloud-enterprise.sh) install --availability-zone MY_ZONE-1 --memory-settings '{"runner":{"xms":"1G","xmx":"1G"},"zookeeper":{"xms":"4G","xmx":"4G"},"director":{"xms":"1G","xmx":"1G"},"constructor":{"xms":"4G","xmx":"4G"},"admin-console":{"xms":"4G","xmx":"4G"}}'

1. Generate a new role token that persists for one hour on the first host. These roles include proxy and allocator. Let’s start with Proxy, you should generate a token for proxy. The command should be executed on the first host, or it should be replaced with Coordinator Host IP instead of localhost.

curl -k -H 'Content-Type: application/json' -u admin:<password> https://localhost:12443/api/v1/platform/configuration/security/enrollment-tokens -d '{ "persistent": false, "roles": ["proxy”] }'

3.Execute the below command on the second host to enroll the role proxy.

bash <(curl -fsSL https://download.elastic.co/cloud/elastic-cloud-enterprise.sh) install --coordinator-host HOST_IP --roles-token '<Token>' --roles "proxy" --availability-zone MY_ZONE-1 --memory-settings '{"runner":{"xms":"1G","xmx":"1G"}

1. You should generate another token for the Allocator role.

curl -k -H 'Content-Type: application/json' -u admin:<password> https://localhost:12443/api/v1/platform/configuration/security/enrollment-tokens -d '{ "persistent": false, "roles": ["allocator”] }'

1. Execute the below command on the second host to enroll the role allocator.

bash <(curl -fsSL https://download.elastic.co/cloud/elastic-cloud-enterprise.sh) install --coordinator-host HOST_IP --roles-token '<Token>' --roles "allocator" --availability-zone MY_ZONE-1 --memory-settings '{"runner":{"xms":"1G","xmx":"1G"},"allocator ":{"xms":"4G","xmx":"4G"}}'

Separating Roles

1. Login into your Cloud UI which is hosted in the first host where we installed all the roles. 2.Use the IP address of the host which has director and controller roles. Cloud UI will be hosted on port 12443. 3.For separating roles, login to the Cloud UI by using the admin user and navigate to the following path as follows:

Platform –> Hosts –> Select the Host IP which has all the roles
Uncheck the proxy box and update the changes.

4.Now, the host shouldn’t have the proxy role.

Note: Initially it won’t be possible to remove the allocator role from the first host. Because the allocator will be in use by running three instances/clusters which are created by default during the installation and these clusters are known to be a system deployment.

1. Let’s separate the allocator role from the first host. First and foremost, we should move all the clusters to another allocator which we enrolled in the second host.

In the Cloud UI, navigate to Platform –> Hosts –> Select the Host IP which has all the roles –> Allocator. Check the boxes of instances that you want to move it to new allocator and click Move Instances.

In the next screen, you need to select the allocator that you’re moving to (In our case, it is the second host). Once it has been moved to another allocator host. We could be able to remove the allocator role in the first host.

1. For separating roles, login to the Cloud UI by using the admin user and navigate to the following path as follows:

Platform –> Hosts –> Select the Host IP which has all the roles.
Uncheck the allocator box and update the changes.

Here we successfully separated the proxy and allocator roles from the director and controller host.

*Reverse Proxy Configuration *

1. Nginx is used as a reverse proxy to redirect traffic to Cloud UI and Proxy. In this setup the nginx is going to be a public faced system. It listens on port 443 and redirects traffic to Cloud UI and Proxy based on the requests. The below image shows our ECE deployment with nginx being used as reverse proxy.

2. Deploying reverse proxy or load balancer as public faced is more secure than exposing your internal systems to the public.

3. If you want to use https then it requires SSL Certificate of your domain, if not you should generate it from relevant CA.

4. You should create a single SSL Certificate for the following domains:

cloudui.example.com (for accessing Cloud UI)
*.kibana.example.com (for accessing Kibana UI)
*.fleet.kibana.example.com (for Fleet server)
*.apm.kibana.example.com (for APM)

1. Also, the domain names should be added in your DNS records.

In A record:

cloudui.example.com --> < Your Public IP >

In CNAME record:

*.kibana.example.com --> cloudui.example.com
*.fleet.kibana.example.com --> cloudui.example.com
*.apm.kibana.example.com --> cloudui.example.com

For more information call us at: +1 925 918 6565 or email your concern at hello@sennovate.com.

Partner with Sennovate and learn more about the Elastic Cloud Enterprise. Our cybersecurity experts will work closely with you to develop tailored solutions that meet your specific needs and regulatory obligations. Let’s build a secure and compliant future for your bank, together. Contact Sennovate today and ditch the compliance worries!