Bug 152465

Summary: [jail] [patch] devfs is mounted in jails without rules if devfs.rules can't be parsed
Product: Base System Reporter: Andrey Zholos <aaz>
Component: confAssignee: freebsd-jail (Nobody) <jail>
Status: Open ---    
Severity: Affects Only Me CC: linimon
Priority: Normal Keywords: patch
Version: 9.0-CURRENT   
Hardware: Any   
OS: Any   

Description Andrey Zholos 2010-11-21 23:20:09 UTC
If /etc/devfs.rules contains invalid rules and can't be parsed, devfs is
still mounted inside jails, exposing all host devices to a potentially
untrusted environment.

Because parsing of rules stops at the first error, this can happen when
the invalid rule is in a group of rules unrelated to the jail, and even
when a syntactically-correct rule becomes invalid.

For example, the rule

    add path 'ulpt*' mode 0660 group cups

becomes invalid when CUPS is deinstalled (removing the cups group).
This produces a warning, but jails are already started with full access
to devfs before the rule can be removed.

This doesn't affect jails using the standard ruleset (devfsrules_jail in
/etc/defaults/devfs.rules), only those using a custom ruleset in
/etc/devfs.rules which is specified after an invalid rule.

How-To-Repeat: Make a simple jail (replace "ad0"):

# mkdir -p /sandbox/{dev,etc,bin,lib,libexec}
# cp /bin/dd /sandbox/bin
# cp /lib/libc.so.* /sandbox/lib
# cp /libexec/ld-elf.so.* /sandbox/libexec
# echo 'root:*:0:0::0:0:Root:/:' > /sandbox/etc/master.passwd
# pwd_mkdb -p -d /sandbox/etc /sandbox/etc/master.passwd

/etc/rc.conf has:
jail_sandbox_exec_start="/bin/dd if=/dev/ad0 of=ad0_copy count=1"

/etc/devfs.rules has:
add hide

Normal start, jail can't access host disk:

# /etc/rc.d/jail start
Configuring jails:.
Starting jails: cannot start jail "sandbox": 
dd: /dev/ad0: No such file or directory
Comment 1 Andrey Zholos 2010-11-22 00:32:53 UTC
The report got truncated (at a line that was a single dot).

The rest of How-To-Repeat is:

Prevent devfs.rules from being parsed:

/etc/devfs.rules now has:
add path 'ulpt*' group nonexistent
add hide

Watch the jail read /dev/ad0 into /sandbox/ad0_copy:

# /etc/rc.d/jail restart


One possible solution is attached: devfs_mount_jail fails if devfs.rules
can't be parsed or the ruleset can't be applied, and the jail doesn't
start if this happens.

This can leave devfs mounted without rules after a jail fails to start,
so if devfs is already mounted when the jail is being started the next
time, unmount it and try again rather than using it as is.

This patch will prevent all jails using devfs from starting if
devfs.rules can't be parsed, even those using the standard ruleset (from
/etc/defaults/devfs.rules). Another option is to ignore errors in
devfs_init_ruleset, while still failing on errors in devfs_set_ruleset
and devfs_apply_ruleset.


--- etc/rc.subr.orig	2010-11-03 17:39:53.000000000 +0000
+++ etc/rc.subr	2010-11-21 18:52:42.000000000 +0000
@@ -1422,14 +1422,19 @@
 		return 1
 	debug "$_me: mount-point is ($devdir), ruleset is ($rs)"
+	if [ -n "$rs" ]; then
+		if ! devfs_init_rulesets; then
+			warn "$_me: Unable to load ruleset $rs";
+			return 1
+		fi
+	fi
 	if ! mount -t devfs dev "$devdir"; then
 		warn "$_me: Unable to mount devfs on $devdir"
 		return 1
 	if [ -n "$rs" ]; then
-		devfs_init_rulesets
-		devfs_set_ruleset $rs $devdir
-		devfs -m $devdir rule applyset
+		devfs_set_ruleset $rs $devdir || return 1
+		devfs_apply_ruleset $rs $devdir || return 1
 	return 0
@@ -1448,7 +1453,6 @@
 	[ -n "$2" ] && rs=$2 || rs="devfsrules_jail"
-	devfs_init_rulesets
 	if ! devfs_domount "$jdev" $rs; then
 		warn "$_me: devfs was not mounted on $jdev"
 		return 1
--- etc/rc.d/jail.orig	2010-11-03 17:39:53.000000000 +0000
+++ etc/rc.d/jail	2010-11-21 18:49:48.000000000 +0000
@@ -590,22 +590,26 @@
 		if checkyesno _devfs; then
-			# If devfs is already mounted here, skip it.
-			df -t devfs "${_devdir}" >/dev/null
-			if [ $? -ne 0 ]; then
-				if is_symlinked_mountpoint ${_devdir}; then
-					warn "${_devdir} has symlink as parent - not 
starting jail ${_jail}"
-					continue
-				fi
-				info "Mounting devfs on ${_devdir}"
-				devfs_mount_jail "${_devdir}" ${_ruleset}
-				# Transitional symlink for old binaries
-				if [ ! -L "${_devdir}/log" ]; then
-					__pwd="`pwd`"
-					cd "${_devdir}"
-					ln -sf ../var/run/log log
-					cd "$__pwd"
-				fi
+			# If devfs is already mounted here, unmount it first.
+			if df -t devfs "${_devdir}" >/dev/null; then
+				secure_umount "${_devdir}"
+			fi
+			if is_symlinked_mountpoint ${_devdir}; then
+				warn "${_devdir} has symlink as parent - not starting 
jail ${_jail}"
+				continue
+			fi
+			info "Mounting devfs on ${_devdir}"
+			if ! devfs_mount_jail "${_devdir}" ${_ruleset}; then
+				warn "devfs failed to mount - not starting jail 
+				continue
+			fi
+			# Transitional symlink for old binaries
+			if [ ! -L "${_devdir}/log" ]; then
+				__pwd="`pwd`"
+				cd "${_devdir}"
+				ln -sf ../var/run/log log
+				cd "$__pwd"
 			# XXX - It seems symlinks don't work when there
Comment 2 Mark Linimon freebsd_committer freebsd_triage 2010-11-22 18:03:36 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-jail

Over to maintainer(s).
Comment 3 Simon L. B. Nielsen freebsd_committer 2010-12-02 07:35:04 UTC
Responsible Changed
From-To: freebsd-jail->simon

Grab PR to make sure it get dealt with - other committers should feel 
free to grab the PR if they like and can handle it soon.
Comment 4 Mark Linimon freebsd_committer freebsd_triage 2015-11-12 01:23:40 UTC
Reset to default assignee after several years of inactivity on this PR.

To submitter: is this issue still relevant?
Comment 5 Andrey Zholos 2015-11-20 00:07:48 UTC
The same thing can still happen. Below is an updated scenario for 11.0-CURRENT.

Perhaps it's not a likely scenario but it did happen to me once five years ago.

I'd suggest not starting the jail if the configured devfs_ruleset doesn't exist,
but someone might do that on purpose and configure the rules in exec.prestart.

Another option is for the devfs.rules parser to attempt to load subsequent
rulesets after an error, or just documenting that important rulesets go first.


Install cups, following pkg-message to set up devfs:

# pkg install -y cups
# cat >>/etc/devfs.rules
add path 'usb*' mode 0770 group cups
add path 'ugen*' mode 0660 group cups

Create jail:

# cat >>/etc/devfs.rules
add hide
# cat >>/etc/jail.conf
sandbox {
    path = /sandbox;
    ip4.addr =;
    devfs_ruleset = 100;
    exec.start = "/dd if=/dev/ada0 of=ada0_copy count=1";
# cat >>/etc/rc.conf
# mkdir /sandbox /sandbox/dev
# cp /rescue/dd /sandbox/

Reboot. Jailed command can't access /dev/ada0:

# ls /sandbox
dd      dev

Uninstall cups, following the suggestion to remove the user (which removes the
cups group):

# pkg delete -y cups-base
==> You should manually remove the "cups" user.
# rmuser -y cups

Reboot. There's a console warning:

devfs rule: error converting to integer: cups
/etc/rc: WARNING: devfs_init_rulesets: could not read rules from /etc/devfs.rules

But the jailed command starts anyway and can now access /dev/ada0:

# ls /sandbox
ada0_copy       dd              dev
Comment 6 Eitan Adler freebsd_committer freebsd_triage 2018-05-23 10:27:32 UTC
batch change of PRs untouched in 2018 marked "in progress" back to open.