Bug 27698

Summary: kernel trap in ptsstop() in all 4.x releases
Product: Base System Reporter: bein <bein>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 4.2-RELEASE   
Hardware: Any   
OS: Any   

Description bein 2001-05-28 03:30:00 UTC
	With the new kernel dev_t conversions done at release 4.X,
	it becomes possible to trap in ptsstop() in kern/tty_pty.c
	if the slave side has never been opened during the life of a kernel.

	What happens is that calls to ttyflush() done from ptyioctl() for the
	controlling side end up calling ptsstop() [via (*tp->t_stop)(tp, <X>)]
	which evaluates the following:

	        struct pt_ioctl *pti = tp->t_dev->si_drv1;

	In order for tp->t_dev to be set, the slave device must first be
	opened in ttyopen() [kern/tty.c].

	It appears that the only problem is calls to (*tp->t_stop)(tp, <n>),
	so this could also happen with other ioctls initiated by the
	controlling side before the slave has been opened.

Fix: 

For 4.0, 4.1 and 4.2 revisions of kern/tty_pty.c:

		src/sys/kern/tty_pty.c,v 1.74 2000/02/09 03:32:11 rwatson

	*** tty_pty.c.dist	Tue Feb  8 22:32:11 2000
	--- tty_pty.c	Sun May 27 19:48:30 2001
	***************
	*** 157,162 ****
	--- 157,170 ----
	
	  	devs->si_drv1 = devc->si_drv1 = pt;
	  	devs->si_tty = devc->si_tty = &pt->pt_tty;
	+ 	/*
	+ 	 * Avoid kernel mode trap in ptsstart, ptsstop and ptcwakeup.
	+ 	 *
	+ 	 * This can trivially happen if the controlling side opens and
	+ 	 * calls [via ptyioctl()] ttyflush() BEFORE the slave side
	+ 	 * is opened for the first time.
	+ 	 */
	+ 	pt->pt_tty.t_dev = devs;
	  	ttyregister(&pt->pt_tty);
	  }

	Other than this fix, there is no known workaround. You could
	run some rc script to open all possible slaves and then close
	them.
How-To-Repeat: 
	The following code fragment is essentially what is happening.
	It might seem that this code should be using openpty(), but because
	it uses vfork() and needs to play with the controlling session
	and the parent needs to keep a hand on the controlling side,
	using openpty() is not quite the right approach. The code
	which does this has been in use for at least 15 years (originating
	with BSD 4.2). It was working until FreeBSD 4.0.

	char *hipty = "/dev/ptypv";
	char *hitty = "/dev/ttypv";
	int cfd;
	int on = 1;
	int newpid;

	if ((cfd = open(hipty, O_RDWR, 0)) < 0) {
		perror("open");
		exit(1);
	}

	/*
	 * XXX: If the slave has not been opened, this is where
	 * XXX: the 4.x kernels will trap inside of ptyioctl().
	 */
	if (ioctl(cfd, TIOCREMOTE, &on) < 0) {
		perror("TIOCREMOTE");
		exit(1);
	}

	if ((newpid = (vfork()) < 0) {
		perror("vfork");
		exit(1);
	}
	if (newpid != 0) {
		int fd;
		if (setsid() < 0) {
			perror("setsid");
			_exit(1);
		}
		close(0);
		close(1);

		if ((fd = open(hitty, O_RDWR, 0)) < 0) {
			perror("open#2");
			_exit(1);
		}
		if (ioctl(fd, TIOCSCTTY, (char *)0) < 0) {
			perror("TIOCSCTTY");
			_exit(1);
		}
		close(2);
		dup(0);
		dup(0);
		/* exec a shell */
		_exit(1);
	}
Comment 1 Poul-Henning Kamp freebsd_committer freebsd_triage 2001-05-28 21:22:18 UTC
State Changed
From-To: open->closed

Right on the spot, good analysis. 

Committed to -current, will be MFCed as well. 

Thanks a lot!