Bug 123892

Hello

First, I meet this bug along time ago
http://lists.freebsd.org/pipermail/freebsd-net/2007-July/014941.html
and write small program, similar to
http://www.freebsd.org/cgi/query-pr.cgi?pr=95665
but, that work with tap device, to reproduce bug.

And no feedback come.

Later, I make small research while debugging kernel 
http://lists.freebsd.org/pipermail/freebsd-net/2007-April/014064.html
and again no help :(

Today I got this bug agan but with another way:

You need to have multicore machine, FreeBSD 7 kernel with 

options SCHED_ULE
options PREEMPTION
options SMP 

From NOTES

# PREEMPTION allows the threads that are in the kernel to be preempted
#     by higher priority threads.  It helps with interactivity and
#     allows interrupt threads to run sooner rather than waiting.
#     WARNING! Only tested on amd64 and i386.

When preempt is possible tap-device have 2 unsafe functions (if_tap.c):
tapifstart and tapread.

static int
tapread(struct cdev *dev, struct uio *uio, int flag)
{
	//
	// skipped
	// ..

	/* sleep until we get a packet */
	do {
		s = splimp();
		IF_DEQUEUE(&ifp->if_snd, m);
		splx(s);

		// (1)
		if (m == NULL) {
			if (flag & O_NONBLOCK)
				return (EWOULDBLOCK);

			mtx_lock(&tp->tap_mtx);
			tp->tap_flags |= TAP_RWAIT;
			mtx_unlock(&tp->tap_mtx);
			// (2)
			error = tsleep(tp,PCATCH|(PZERO+1),"taprd",0);
			if (error)
				return (error);
		}
	} while (m == NULL);

	//
	// skipped
	// ..
}


static void
tapifstart(struct ifnet *ifp)
{
	//
	// skipped
	// ..

	s = splimp();
	ifp->if_drv_flags |= IFF_DRV_OACTIVE;

	if (ifp->if_snd.ifq_len != 0) {
		mtx_lock(&tp->tap_mtx);
		if (tp->tap_flags & TAP_RWAIT) {
			tp->tap_flags &= ~TAP_RWAIT;
			// (3)
			wakeup(tp);
		}

		if ((tp->tap_flags & TAP_ASYNC) && (tp->tap_sigio != NULL)) {
			mtx_unlock(&tp->tap_mtx);
			pgsigio(&tp->tap_sigio, SIGIO, 0);
		} else
			mtx_unlock(&tp->tap_mtx);

		selwakeuppri(&tp->tap_rsel, PZERO+1);
		KNOTE_UNLOCKED(&tp->tap_rsel.si_note, 0);
		ifp->if_opackets ++; /* obytes are counted in ether_output */
	}

	ifp->if_drv_flags &= ~IFF_DRV_OACTIVE;
	splx(s);
}

So, when some process make block read on tap-device, tapread called,
it check  if_snd, and if it empty make sleep until we get a packet for
sending (tapifstart called).

Now imagine, that tapread thread see that queue are empty and going to
sleep, but between (1) and (2) interrupted.
While that thread rest interrupted someplace between (1)-(2)
network stack got or generate many packets for tap-device
(50 e.g. >= if_snd.ifq_maxlen), tapifstart called for each packet,
it make wakeup (3) for tapread thread, but they are lost, because no threads
sleeping. Now really if_snd queue are full.

Next, our tapread thread execution returned and go sleep - never wakeup,
because if_snd queue are full, tcp/ip stack will drop new packets for tap-device
and will not call tapifstart.

p.s. tun device have same problem.

Best regards,
   Oleg Dolgov.

Fix: Here are patch, condition variable with if_snd.ifq_mtx
make necessary synchronization.


Patch attached with submission follows: