I have been spending a good bit of time playing with SELinux, which has required me to learn how to debug issues that arise due to its use. The most common issues that arise are usually related to security context mismatches. This can occur because somebody modified a file and didn’t bother to chcon a file, or because somebody changed the contexts that were once present on a file. To see the SELinux context assigned to a file or directory, you can use the ls “-Z” option:
$ ls -lZ /var/www
-rw-r--r--. root root system_u:object_r:httpd_sys_content_t:s0 index.html
Now say a user approached you because they were getting 403 forbidden messages when they tried to retrieve a file on your Apache web server:
$ wget localhost/index.txt
--2010-10-29 14:44:22-- http://localhost/index.txt
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2010-10-29 14:44:22 ERROR 403: Forbidden.
The apache error log indicated that there was indeed a permission issue:
[Fri Oct 29 14:25:53 2010] [error] [client 127.0.0.1] (13)Permission denied: access to /index.txt denied
[Fri Oct 29 14:44:22 2010] [error] [client 127.0.0.1] (13)Permission denied: access to /index.txt denied
Though the UNIX permissions on the file were correct. To see if the permission issue is caused by SELinux, you can fire up the setroubleshootd daemon and then tail /var/log/messages:
$ /usr/sbin/setroubleshootd
$ grep httpd /var/log/messages
Oct 29 14:44:26 theshack setroubleshoot: SELinux is preventing /usr/sbin/httpd "getattr" access to /var/www/html/index.txt. For complete SELinux messages. run sealert -l f02ee96b-75fb-40d6-a5e4-dd2bc91f52e2
Oct 29 14:44:26 theshack setroubleshoot: SELinux is preventing /usr/sbin/httpd "getattr" access to /var/www/html/index.txt. For complete SELinux messages. run sealert -l f02ee96b-75fb-40d6-a5e4-dd2bc91f52e2
We can see that SELinux was the reason the request failed, and we can get some additional information by running sealert with the “-l” option and the alert number:
$ sealert -l f02ee96b-75fb-40d6-a5e4-dd2bc91f52e2
Summary:
SELinux is preventing /usr/sbin/httpd "getattr" access to
/var/www/html/index.txt.
Detailed Description:
SELinux denied access requested by httpd. /var/www/html/index.txt may be a
mislabeled. /var/www/html/index.txt default SELinux type is httpd_sys_content_t,
but its current type is var_t. Changing this file back to the default type, may
fix your problem.
File contexts can be assigned to a file in the following ways.
* Files created in a directory receive the file context of the parent
directory by default.
* The SELinux policy might override the default label inherited from the
parent directory by specifying a process running in context A which creates
a file in a directory labeled B will instead create the file with label C.
An example of this would be the dhcp client running with the dhclient_t type
and creating a file in the directory /etc. This file would normally receive
the etc_t type due to parental inheritance but instead the file is labeled
with the net_conf_t type because the SELinux policy specifies this.
* Users can change the file context on a file using tools such as chcon, or
restorecon.
This file could have been mislabeled either by user error, or if an normally
confined application was run under the wrong domain.
However, this might also indicate a bug in SELinux because the file should not
have been labeled with this type.
If you believe this is a bug, please file a bug report against this package.
Allowing Access:
You can restore the default system context to this file by executing the
restorecon command. restorecon '/var/www/html/index.txt', if this file is a
directory, you can recursively restore using restorecon -R
'/var/www/html/index.txt'.
Fix Command:
/sbin/restorecon '/var/www/html/index.txt'
Additional Information:
Source Context unconfined_u:system_r:httpd_t:s0
Target Context unconfined_u:object_r:var_t:s0
Target Objects /var/www/html/index.txt [ file ]
Source httpd
Source Path /usr/sbin/httpd
Port
Host theshack
Source RPM Packages httpd-2.2.16-1.fc13
Target RPM Packages
Policy RPM selinux-policy-3.7.19-65.fc13
Selinux Enabled True
Policy Type targeted
Enforcing Mode Enforcing
Plugin Name restorecon
Host Name theshack
Platform Linux theshack 2.6.34.7-61.fc13.x86_64 #1 SMP Tue
Oct 19 04:06:30 UTC 2010 x86_64 x86_64
Alert Count 4
First Seen Fri Oct 29 14:25:53 2010
Last Seen Fri Oct 29 14:44:22 2010
Local ID f02ee96b-75fb-40d6-a5e4-dd2bc91f52e2
Line Numbers
Raw Audit Messages
node=theshack type=AVC msg=audit(1288377862.457:22628): avc: denied { getattr } for pid=2798 comm="httpd" path="/var/www/html/index.txt" dev=dm-0 ino=525951 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file
node=theshack type=SYSCALL msg=audit(1288377862.457:22628): arch=c000003e syscall=6 success=no exit=-13 a0=7f609700e588 a1=7fffabd48f80 a2=7fffabd48f80 a3=1 items=0 ppid=2794 pid=2798 auid=500 uid=48 gid=487 euid=48 suid=48 fsuid=48 egid=487 sgid=487 fsgid=487 tty=(none) ses=1 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)
The error message is pretty self explanatory, and you can use restorecon to fix the security context of the file:
$ /sbin/restorecon '/var/www/html/index.txt'
Once the correct security context has been assigned to the file, we can verify that all is well:
$ ls -lZ /var/www/html/index.txt
-rw-r--r--. root root system_u:object_r:httpd_sys_content_t:s0 /var/www/html/index.txt
$ wget -q -O - localhost/index.txt
hi
If you aren’t able to fire up setroubleshootd, you can still debug this issue. The first thing to look at is the audit log (audit.log) and messages file, which should both be located under /var/log. Let’s see if the httpd process caused any alerts to be generated:
$ grep httpd /var/log/audit/audit.log
type=AVC msg=audit(1288376753.574:22621): avc: denied { getattr } for pid=2803 comm="httpd" path="/var/www/html/index.txt" dev=dm-0 ino=525951 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file
type=SYSCALL msg=audit(1288376753.574:22621): arch=c000003e syscall=4 success=no exit=-13 a0=7f609700e4b8 a1=7fffabd48f80 a2=7fffabd48f80 a3=0 items=0 ppid=2794 pid=2803 auid=500 uid=48 gid=487 euid=48 suid=48 fsuid=48 egid=487 sgid=487 fsgid=487 tty=(none) ses=1 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1288376753.575:22622): avc: denied { getattr } for pid=2803 comm="httpd" path="/var/www/html/index.txt" dev=dm-0 ino=525951 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file
type=SYSCALL msg=audit(1288376753.575:22622): arch=c000003e syscall=6 success=no exit=-13 a0=7f609700e588 a1=7fffabd48f80 a2=7fffabd48f80 a3=1 items=0 ppid=2794 pid=2803 auid=500 uid=48 gid=487 euid=48 suid=48 fsuid=48 egid=487 sgid=487 fsgid=487 tty=(none) ses=1 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)
In the messages above, we can see that SELinux denied access to the index.txt file due to an incorrect security context (I’ll do a follow up post on security contexts and how to tell which ones should be used for a given file). To fix this, we can once again run the restorecon command to recursively restore the context of all of the files in /var/www:
$ restorecon -R /var/www
$ ls -lZ
-rw-r--r--. root root system_u:object_r:httpd_sys_content_t:s0 index.html
SELinux is a pretty cool technology, and I think it’s gotten a bad wrap due to its complexity. I’m going to write up a number of additional SELinux posts in the coming weeks, which will hopefully demystify some things and make debugging SELinux issues a piece of cake!