This page explains how to make Cribl Stream work correctly with SELinux in enforcing mode.
If you’re unsure about SELinux and its modes, here’s some background:
- Security-Enhanced Linux (SELinux) is a mandatory access control (MAC) security mechanism implemented in the kernel. (Source: CentOS Wiki.)
- SELinux defines access controls for the applications, processes, and files on a system. It uses security policies, which are a set of rules that tell SELinux what can or can’t be accessed, to enforce the access allowed by a policy. (Source: Red Hat.)
- Processes and files are labeled with an SELinux context that contains additional information, such as an SELinux user, role, type, and, optionally, a level. When running SELinux, all of this information is used to make access control decisions. (Source: Red Hat.)
- SELinux has two modes, enforcing and permissive. When running in enforcing mode, SELinux enforces the SELinux policy and denies access based on SELinux policy rules. (Source: Red Hat.)
SELinux and Cribl Stream
Cribl Stream is typically used for in-stream processing, where the processes and files that Cribl Stream Workers need to access are known beforehand. This makes it easy to write an SELinux policy that’s relatively restrictive of processes and files.
For network ports, though, the policy should be more permissive, because users can configure those within the UI.
SELinux Troubleshooting
SELinux problems usually surface when you try to start the Cribl Stream service. Users immediately receive an error like the following:
Cribl Stream fails to start
Job for cribl.service failed because the control process exited with error code.See "systemctl status cribl.service" and "journalctl -xe" for details.
One indication that the problem is in SELinux configuration, not in the Cribl Stream service itself, is when (1) nothing has been logged in $CRIBL_HOME/log/cribl.log, and (2), when you run journalctl -xe you see errors like the following:
cribl.service: Failed to execute command: Permission denied
cribl.service: Failed at step EXEC spawning /opt/cribl/bin/cribl: Permission denied
cribl.service: Control process exited, code=exited status=203
cribl.service: Failed to execute command: No such file or directory
cribl.service: Failed at step EXEC spawning /bin/rm -f /opt/cribl/pid/cribl.pid: Permission denied
You can confirm that SELinux configuration is the issue by verifying that the Cribl Stream service is able to start up on its own. To do this, run the cribl start command.
Here are three commands that you can think of as a basic SELinux troubleshooting toolkit:
getenforce verifies that SELinux is enabled and enforcing policy.sestatus -v shows details of the SELinux policy that’s in effect.ls with the -Z or --context option shows the SELinux tags on files and/or folders. In the output, the fifth column reports the SELinux context in the form user:role:type:level.
Example Problem
Suppose you run ls -lZ /opt/cribl to see the SElinux context for /opt/cribl.
If the output looks like the following, there is a problem: the presence of “home” among the type part of the Linux context. In this case, user_home_t is the offending type. When the type is user_home_t or admin_home_t, SELinux will not permit the service to read, write, or execute files.
Problematic SELinux context output:
$ ls -laZ /opt/cribl
total 0
drwxr-xr-x. 6 cribl cribl unconfined_u:object_r:user_home_t:s0 62 Feb 16 08:31 .
drwxr-xr-x. 3 root root system_u:object_r:usr_t:s0 19 Feb 16 19:54 ..
drwxr-xr-x. 2 cribl cribl unconfined_u:object_r:user_home_t:s0 202 Feb 16 08:31 bin
drwxr-xr-x. 5 cribl cribl unconfined_u:object_r:user_home_t:s0 49 Feb 16 08:20 data
drwxr-xr-x. 6 cribl cribl unconfined_u:object_r:user_home_t:s0 59 Feb 16 08:31 default
drwxr-xr-x. 3 cribl cribl unconfined_u:object_r:user_home_t:s0 22 Feb 16 08:31 thirdparty
Example Solution
To fix the labeling, download Cribl Stream again directly to /opt. Do not download Cribl Stream to a user’s home directory, or to root (/), and then move it to /opt - that was what caused our example problem.
curl -Lso - $(curl -s https://cdn.cribl.io/dl/latest) | sudo tar zxvf - -C /opt
A simpler option is to revert the labels using restorecon, an SELinux policy utility that’s in the policycoreutils-python-utils package.
restorecon -R -v /opt/cribl
systemctl restart cribl
Once you correct the problem, the output will look like the following. Instead of *home_t listings in the type field, this SELinux context has usr_t, a type that will satisfy SELinux’s criteria for allowing the service to read, write, and execute files.
# ls -laZ /opt/cribl
total 0
drwxr-xr-x. 6 cribl cribl unconfined_u:object_r:usr_t:s0 62 Feb 16 08:31 .
drwxr-xr-x. 3 root root system_u:object_r:usr_t:s0 19 Feb 16 20:05 ..
drwxr-xr-x. 2 cribl cribl unconfined_u:object_r:usr_t:s0 202 Feb 16 08:31 bin
drwxr-xr-x. 5 cribl cribl unconfined_u:object_r:usr_t:s0 49 Feb 16 08:20 data
drwxr-xr-x. 6 cribl cribl unconfined_u:object_r:usr_t:s0 59 Feb 16 08:31 default
drwxr-xr-x. 3 cribl cribl unconfined_u:object_r:usr_t:s0 22 Feb 16 08:31 thirdparty
