Description Conrad Meyer 2017-12-21 18:35:47 UTC

Rpcbind may crash in signal-induced termination.

E.g.:

> (gdb) bt
> ...
> #12 __free (ptr=0x8006f8140) at jemalloc_jemalloc.c:1277
> #13 0x0000000800d4c8e7 in vector_free (esize=<optimized out>, vec=<optimized out>, count=<optimized out>, esize=<optimized out>, free_elem=<optimized out>)
>     at /b/mnt/src/lib/libc/net/nsdispatch.c:250
> #14 nss_atexit () at /b/mnt/src/lib/libc/net/nsdispatch.c:577
> #15 0x0000000800d4d5b9 in __cxa_finalize (dso=0x0) at /b/mnt/src/lib/libc/stdlib/atexit.c:200
> #16 0x0000000800cfe1ac in exit (status=2) at /b/mnt/src/lib/libc/stdlib/exit.c:67
> #17 0x0000000000404df5 in terminate (signum=-26832) at /b/mnt/src/usr.sbin/rpcbind/rpcbind.c:860
> #18 0x000000080337f67b in handle_signal (actp=<optimized out>, sig=15, info=0x7fffffffa9f0, ucp=0x7fffffffa680) at /b/mnt/src/lib/libthr/thread/thr_sig.c:240
> #19 0x000000080337f263 in thr_sighandler (sig=15, info=0x7fffffffa9f0, _ucp=0x800700210) at /b/mnt/src/lib/libthr/thread/thr_sig.c:183

rpcbind got a sig 15 (TERM) and attempted to call exit(3) from a signal handler.  But, exit(3) is not an async-signal safe function.  Probably these crashes are the result of the program manipulating jemalloc internal structures at the time the SIGTERM is delivered and handled unsafely.

The cause can be explained:

195         /* catch the usual termination signals for graceful exit */
196         (void) signal(SIGCHLD, reap);
197         (void) signal(SIGINT, terminate);
198         (void) signal(SIGTERM, terminate);
199         (void) signal(SIGQUIT, terminate);
...

758 /*
759  * Catch the signal and die
760  */
761 static void
762 terminate(int signum __unused)
763 {
764         close(rpcbindlockfd);
765 #ifdef WARMSTART
766         syslog(LOG_ERR,
767             "rpcbind terminating on signal %d. Restart with \"rpcbind -w\"",
768             signum);
769         write_warmstart();      /* Dump yourself */
770 #endif
771         exit(2);
772 }
...

// close(2) is async-safe -- rpcbindlockfd must be initalized before the signal is
// delivered, though.
//
// syslog() is definitely not safe.
//
// write_warmstart() uses fopen(3), syslog(3), ... definitely not safe.
//
// Finally, exit(3) itself is not safe either.

857 void
858 reap(int dummy __unused)
859 {
860         int save_errno = errno;
861
862         while (wait3(NULL, WNOHANG, NULL) > 0)
863                 ;
864         errno = save_errno;
865 }

// wait(2) and waitpid(2) are allowed, but wait3() is not documented as allowed in
// sigaction(2).  It likely is allowed, though, given it is implemented identically
// to wait(2) (in terms of wait4(2)).  So `reap` for SIGCHLD is probably ok.