TL; DR

If you want to use Kubernetes with SELinux and mount PersistentVolume, you have to make sure your mounted FS has labels. You can do it with the mountOptions -o context="system_u:object_r:container_var_lib_t:s0" and if your driver doesn’t support it, you can write an SELinux policy like this one.

The premise

The problem came from a customer who is using Docker Enterprise with CSI Driver for VxFlexOS (now PowerFlex), and SELinux enforced on the nodes.

Anytime a Pod tried to write data on the PersistentVolume, we had Permission denied error from the OS.

SELinux relies on file label (sometimes call context) in the file extended attributes to apply its policies. For examples below :

ls -lZd /root /home /etc
drwxr-xr-x. root root system_u:object_r:etc_t:s0 /etc
drwxr-xr-x. root root system_u:object_r:home_root_t:s0 /home
dr-xr-x---. root root system_u:object_r:admin_home_t:s0 /root

By default, on a newly formatted and mounted FS the files are unlabeled :

mkfs.ext4 /dev/loop0
mount -o loop /dev/loop0 /media/xxx
ls -Z
drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 xxx

On the customer setup, it was forbidden to do any action on an unlabeled file.

How to get the new FS labeled ?

The typical way to relabel a directory is to use the command restorecon against the mount point, which will restore the security context and label.Because we are in a containerized environment with volumes dynamically mounted by Kubernetes, this is not realistic to expect the restorecon command to be executed each time a volume is mounted.

Another option, especially suitable to big FileSystems with many files, is to can create a file named .autorelabel at the root level, so it forces a relabel on the mountpoint. Slightly better but, still, not feasible in a dynamic environment like ours.

A better option is to mount the FS with the context option. That option is my favorite ❣

Unfortunately, the DellEMC CSI drivers don’t have the mountOption capability at the time of that post. That feature is on the roadmap, but in the meantime, I needed a plan B.

The last possibility and the one we implemented is to write a specific policy to allow containers to manipulate unlabeled files and directories.

Since SELinux follows a model of the least-privilege (aka you can’t do anything except if explicitly allowlisted), the challenge was to have all the syscalls a container needs to do their job.

To get inspiration, I hijacked policies given in the SELinux Project and came up with that list : class file { create open getattr setattr read write append rename link unlink ioctl lock };

The full policy is :

module vxflexos-cni 1.0;

require {
    type unlabeled_t;
    type container_t;
    class file { create open getattr setattr read write append rename link unlink ioctl lock };
    class dir { add_name create getattr ioctl link lock open read remove_name rename reparent rmdir search setattr unlink write };
}

#!!!! WARNING: 'unlabeled_t' is a base type.
allow container_t unlabeled_t:dir { create open getattr setattr read write link unlink rename search add_name remove_name reparent rmdir lock ioctl };
allow container_t unlabeled_t:file { create open getattr setattr read write append rename link unlink ioctl lock };

To compile it, you will need the SELinux Devel package (in Fedora : dnf install selinux-policy-devel.noarch will do) and then compile the policy with : make -f /usr/share/selinux/devel/Makefile. To install the newly compiled policy run : semodule -i vxflexos-cni.pp.

Wrap-up

mountOptions capability is coming with every Dell Technologies CSI driver in the incoming months.

The Gentoo website is a gold mine of information for SELinux ! To understand better the issue, I mostly read these pages on Labels, the Tutorial to create a policy, and SELinux Project Policies.