Bug 197170 - [nfs] Kernel panic when receiving invalid RPC requests
Summary: [nfs] Kernel panic when receiving invalid RPC requests
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.1-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Rick Macklem
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-29 11:03 UTC by Thomas Mueller
Modified: 2015-05-02 13:09 UTC (History)
3 users (show)

See Also:


Attachments
Application program exercising the bug. (17.15 KB, text/x-csrc)
2015-01-29 11:03 UTC, Thomas Mueller
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Mueller 2015-01-29 11:03:56 UTC
Created attachment 152333 [details]
Application program exercising the bug.

I've found a problem in the NFS server implementation which can lead to a
kernel crash when issuing invalid NFSv2 RPC calls. I made the observation on
10.1-STABLE amd64 (r277871) and on an older system running 9.2-STABLE amd64
from around November 2013.

The attached program (nfs_usr.c) can be used to trigger the problem as
follows:
  cc nfs_usr.c && ./a.out host path

  With 'host' being an NFS server exporting 'path' and allowing mount
  requests on unprivileged ports (mountd option -n).

The program is interactive and prompts for input for each of the
steps described below.

First the program retrieves MOUNTD and NFSD ports from the portmapper on 'host'
and then issues an RPCMNT_MOUNT RPC call attempting to mount 'path'. Next a
NFSV2PROC_GETTATTR RPC call is issued on the mounted path.

The next RPC call is NFSV2PROC_ROOT (procedure #3) with an RPC call
structure suitable for a 'void NFSPROC_ROOT(void) 'procedure. This call was
denied by the remote host in my tests.

The next RPC call is again NFSV2PROC_ROOT (procedure #3) but this time
with an RPC call structure suitable for a 'void NFSPROC_ROOT(fhandle)' 
procedure, which is incompatible with the definition of the ROOT procedure.
This call leads to a kernel panic in my tests.

[root@tmuvbfbsd10 /var/crash]# uname -a
FreeBSD tmuvbfbsd10.XXXXXX.com 10.1-STABLE FreeBSD 10.1-STABLE #0 r277871: Thu Jan 29 11:31:23 CET 2015     toor@tmuvbfbsd10.XXXXXX.com:/usr/obj/usr/src/sys/GENERIC  amd64

[root@tmuvbfbsd10 /var/crash]# kgdb /usr/obj/usr/src/sys/GENERIC/kernel.debug vmcore.5
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
fault virtual address	= 0x1133dfc04
fault code		= supervisor write data, page not present
instruction pointer	= 0x20:0xffffffff80867371
stack pointer	        = 0x28:0xfffffe0073bcddb0
frame pointer	        = 0x28:0xfffffe0073bcdf70
code segment		= base rx0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 565 (nfsd: master)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff80969990 at kdb_backtrace+0x60
#1 0xffffffff8092de25 at panic+0x155
#2 0xffffffff80d2c07f at trap_fatal+0x38f
#3 0xffffffff80d2c37d at trap_pfault+0x2ed
#4 0xffffffff80d2b9fa at trap+0x47a
#5 0xffffffff80d11ef2 at calltrap+0x8
#6 0xffffffff80876d24 at nfssvc_program+0x554
#7 0xffffffff80b2e717 at svc_run_internal+0xd07
#8 0xffffffff80b2d91e at svc_run+0x1de
#9 0xffffffff808771aa at nfsrvd_nfsd+0x1ca
#10 0xffffffff80887347 at nfssvc_nfsd+0x107
#11 0xffffffff80b15b9c at sys_nfssvc+0x9c
#12 0xffffffff80d2c997 at amd64_syscall+0x357
#13 0xffffffff80d121db at Xfast_syscall+0xfb
Uptime: 2m25s
Dumping 141 out of 1500 MB:..12%..23%..34%..46%..57%..68%..80%..91%

#0  doadump (textdump=<value optimized out>) at pcpu.h:219
219	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) list *0xffffffff80867371
0xffffffff80867371 is in nfsrvd_dorpc (/usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:475).
470		 * to complete the RPC reply for V2. Otherwise, you must do
471		 * the RPC.
472		 */
473		if (nd->nd_repstat && (nd->nd_flag & ND_NFSV2)) {
474			*nd->nd_errp = nfsd_errmap(nd);
475			NFSINCRGLOBAL(newnfsstats.srvrpccnt[nfsv3to4op[nd->nd_procnum]]);
476			if (mp != NULL && nfs_writerpc[nd->nd_procnum] != 0)
477				vn_finished_write(mp);
478			goto out;
479		}
Current language:  auto; currently minimal
(kgdb) backtrace
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
#1  0xffffffff8092daa2 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452
#2  0xffffffff8092de64 in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff80d2c07f in trap_fatal (frame=<value optimized out>, eva=<value optimized out>)
    at /usr/src/sys/amd64/amd64/trap.c:859
#4  0xffffffff80d2c37d in trap_pfault (frame=0xfffffe0073bcdd00, usermode=<value optimized out>)
    at /usr/src/sys/amd64/amd64/trap.c:676
#5  0xffffffff80d2b9fa in trap (frame=0xfffffe0073bcdd00) at /usr/src/sys/amd64/amd64/trap.c:440
#6  0xffffffff80d11ef2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236
#7  0xffffffff80867371 in nfsrvd_dorpc (nd=0xfffffe0073bcdfa0, isdgram=<value optimized out>, tag=<value optimized out>, 
    taglen=<value optimized out>, minorvers=<value optimized out>, p=<value optimized out>)
    at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:475
#8  0xffffffff80876d24 in nfssvc_program (rqst=0xfffff8001979b000, xprt=0xfffff800192bd800)
    at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:371
#9  0xffffffff80b2e717 in svc_run_internal (grp=0xfffffe0000ad0180, ismaster=1) at /usr/src/sys/rpc/svc.c:993
#10 0xffffffff80b2d91e in svc_run (pool=0xfffffe0000ad0000) at /usr/src/sys/rpc/svc.c:1345
#11 0xffffffff808771aa in nfsrvd_nfsd (td=<value optimized out>, args=0xfffffe0073bce4b8)
    at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:506
#12 0xffffffff80887347 in nfssvc_nfsd (td=0xfffff80019364920, uap=<value optimized out>)
    at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:3081
#13 0xffffffff80b15b9c in sys_nfssvc (td=0xfffff80019364920, uap=0xfffffe0073bceb80) at /usr/src/sys/nfs/nfs_nfssvc.c:108
#14 0xffffffff80d2c997 in amd64_syscall (td=0xfffff80019364920, traced=0) at subr_syscall.c:134
#15 0xffffffff80d121db in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396
---Type <return> to continue, or q <return> to quit---
#16 0x000000080089227a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Comment 1 Rick Macklem freebsd_committer freebsd_triage 2015-05-02 13:09:01 UTC
Patch has been committed and MFC'd.