Bug 61744 - [tcp] [patch] TCP hangs onto mbufs with no tcp data unnecessily under certain error conditions
Summary: [tcp] [patch] TCP hangs onto mbufs with no tcp data unnecessily under certain...
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-01-22 19:00 UTC by Roselyn Lee
Modified: 2020-10-23 18:04 UTC (History)
3 users (show)

See Also:


Attachments
patch-2.diff (1.17 KB, patch)
2010-08-16 23:13 UTC, Andre Oppermann
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Roselyn Lee 2004-01-22 19:00:35 UTC
      In tcp_reass(), if we get a duplicate ACK with len = 0, we queue the mbuf.  This is not necessary because there is no additional data in this mbuf that's useful for this tcp session.  Under certain error conditions, we've seen VMware clients running freebsd send thousands of FIN/ACKs with duplicate sequence #'s and tcp len = 0.  These result in mbufs being held on the system until the socket is closed.  Sometimes this could result in mbuf exhaustion on the system.

Fix: 

The following patch to tcp_input.c's tcp_reass() routine seems to do the trick.

--- freebsd/sys/netinet/tcp_input.c     5 Jun 2003 02:12:11 -0000       1.3
+++ freebsd/sys/netinet/tcp_input.c     22 Jan 2004 17:52:54 -0000      1.4
@@ -219,20 +219,29 @@
                                 * Try to present any queued data
                                 * at the left window edge to the user.
                                 * This is needed after the 3-WHS
                                 * completes.
                                 */
                                goto present;   /* ??? */
                        }
                        m_adj(m, i);
                        *tlenp -= i;
                        th->th_seq += i;
+               } else if (i == 0 && p->tqe_len == 0 && *tlenp == 0) {
+                       /*
+                        * This means we're getting the same seq# and
+                        * there is no new data. Just drop it.
+                        */
+                       tcpstat.tcps_rcvduppack++;
+                       m_freem(m);
+                       free(te, M_TSEGQ);
+                       return (0);
                }
        }
        tcpstat.tcps_rcvoopack++;
        tcpstat.tcps_rcvoobyte += *tlenp;
 
        /*
         * While we overlap succeeding segments trim them or,
         * if they are completely covered, dequeue them.
         */
        while (q) {
How-To-Repeat: In our environment, we can reproduce the the problem as follows:
1.  setup a freebsd pptp client (mpd) to connect to the freebsd server but have it use the wrong password so it will keep trying to talk.
2.  Have the client 'disappear' for seconds to minutes and 'reappear' to the server. We do this by making the client roam away out of range.
3.  Sometimes, this will result in thousands of FIN/ACKs with same seq#.

Here is a partial tcpdump summary when the error condition is reached.  Starting at froamd 2740, the FIN/ACK exchange continues until frame 20877 - looks like the client roams out of range at this point and returns a few seconds later.  The tcpdump was filtered for this client only and so there a some gaps in the seq.  But 90% of the packets captured from frames 2740 to 20877 were for this client.

2562 13:31:34.3043   42.0.0.1              42.171.25.102         PPP CHAP PPP CHAP Failure                                                127    2562
   2563 13:31:34.3078   42.0.0.1              42.171.25.102         PPP LCP  PPP LCP Termination Request                                     60     2563
   2564 13:31:34.3113   42.171.25.102         42.0.0.1              PPTP     CLEAR-CALL-REQUEST                                              70     2564
   2565 13:31:34.3140   42.0.0.1              42.171.25.102         PPTP     DISCONNECT-NOTIFY                                               202    2565
   2566 13:31:34.3156   42.0.0.1              42.171.25.102         PPTP     STOP-CONTROL-REQUEST                                            70     2566
   2567 13:31:34.3174   42.171.25.102         42.0.0.1              TCP      1194 > pptp [ACK] Seq=3599677643 Ack=1998493514 Win=58236 Len=0 60     2567
   2568 13:31:34.3231   42.171.25.102         42.0.0.1              PPTP     STOP-CONTROL-REQUEST                                            70     2568
   2569 13:31:34.3266   42.171.25.102         42.0.0.1              PPTP     STOP-CONTROL-REPLY                                              70     2569
   2570 13:31:34.3268   42.0.0.1              42.171.25.102         TCP      pptp > 1194 [ACK] Seq=1998493514 Ack=3599677675 Win=65535 Len=0 60     2570
   2571 13:31:34.3272   42.0.0.1              42.171.25.102         PPTP     STOP-CONTROL-REPLY                                              70     2571
   2572 13:31:34.3274   42.0.0.1              42.171.25.102         TCP      pptp > 1194 [FIN, ACK] Seq=1998493530 Ack=3599677675 Win=65535 Len=0 60     2572
   2573 13:31:34.3286   42.171.25.102         42.0.0.1              TCP      1194 > pptp [ACK] Seq=3599677675 Ack=1998493531 Win=58384 Len=0 60     2573
   2574 13:31:34.3304   42.171.25.102         42.0.0.1              TCP      1194 > pptp [FIN, ACK] Seq=3599677675 Ack=1998493531 Win=58400 Len=0 60     2574
   2575 13:31:34.3305   42.0.0.1              42.171.25.102         TCP      pptp > 1194 [ACK] Seq=1998493531 Ack=3599677676 Win=65535 Len=0 60     2575
   2739 13:31:40.3816   42.0.0.1              42.171.25.102         TCP      pptp > 1193 [ACK] Seq=1565198537 Ack=2686410455 Win=32942 Len=0 60     2739
   2740 13:31:40.3835   42.171.25.102         42.0.0.1              TCP      1193 > pptp [FIN, ACK] Seq=2686410655 Ack=1565198538 Win=57920 Len=0 66     2740
   2741 13:31:40.3836   42.0.0.1              42.171.25.102         TCP      pptp > 1193 [FIN, ACK] Seq=1565198702 Ack=2686410455 Win=32942 Len=0 66     2741
   2742 13:31:40.3862   42.171.25.102         42.0.0.1              TCP      1193 > pptp [FIN, ACK] Seq=2686410655 Ack=1565198538 Win=57920 Len=0 66     2742
   2743 13:31:40.3862   42.0.0.1              42.171.25.102         TCP      pptp > 1193 [FIN, ACK] Seq=1565198702 Ack=2686410455 Win=32942 Len=0 66     2743
   2744 13:31:40.3885   42.171.25.102         42.0.0.1              TCP      1193 > pptp [FIN, ACK] Seq=2686410655 Ack=1565198538 Win=57920 Len=0 66     2744
   2745 13:31:40.3885   42.0.0.1              42.171.25.102         TCP      pptp > 1193 [FIN, ACK] Seq=1565198702 Ack=2686410455 Win=32942 Len=0 66     2745
....
  20877 13:31:49.3741   42.171.25.102         42.0.0.1              TCP      1193 > pptp [FIN, ACK] Seq=2686410655 Ack=1565198538 Win=57920 Len=0 66     20877
  20878 13:31:49.3741   42.0.0.1              42.171.25.102         TCP      pptp > 1193 [FIN, ACK] Seq=1565198702 Ack=2686410455 Win=32942 Len=0 66     20878
  21192 13:31:56.2532   42.171.25.102         42.0.0.1              TCP      1196 > pptp [SYN] Seq=926832057 Ack=0 Win=57344 Len=0           74     21192
  21193 13:31:56.2534   42.0.0.1              42.171.25.102         TCP      pptp > 1196 [SYN, ACK] Seq=1341705398 Ack=926832058 Win=65535 Len=0 74     21193
  21194 13:31:56.2549   42.171.25.102         42.0.0.1              TCP      1196 > pptp [ACK] Seq=926832058 Ack=1341705399 Win=57920 Len=0  66     21194
  21195 13:31:56.2550   42.0.0.1              42.171.25.102         TCP      pptp > 1196 [ACK] Seq=1341705399 Ack=926832058 Win=32942 Len=0  66     21195
Comment 1 Andre Oppermann freebsd_committer 2004-01-22 20:35:54 UTC
Responsible Changed
From-To: freebsd-bugs->andre

Take over after conversation with Silby.
Comment 2 Andre Oppermann freebsd_committer 2005-09-14 16:52:43 UTC
State Changed
From-To: open->analyzed

Roselyn, 

this simpler patch should fix the problem too.  Can you confirm? 

--  
Andre 

Index: tcp_input.c 
=================================================================== 
RCS file: /home/ncvs/src/sys/netinet/tcp_input.c,v 
retrieving revision 1.283 
diff -u -p -r1.283 tcp_input.c 
--- tcp_input.c 24 Aug 2005 02:48:45 -0000      1.283 
+++ tcp_input.c 14 Sep 2005 15:51:47 -0000 
@@ -239,7 +239,7 @@ tcp_reass(tp, th, tlenp, m) 
* Call with th==NULL after become established to 
* force pre-ESTABLISHED data up to user socket. 
*/ 
-       if (th == NULL) 
+       if (th == NULL || *tlenp == 0) 
goto present; 

/*
Comment 3 Andre Oppermann freebsd_committer 2010-08-16 23:13:40 UTC
Lawrence,

attached are two possible fixes for the issue described in the PR.

I like the first one better as it is much simpler and the FIN can
easily be resent after the missing segment(s) that caused the reassembly
have been retransmitted.

What's your take on it?  Actually I don't mind either way.

-- 
Andre
Comment 4 Hiren Panchasara freebsd_committer 2016-12-22 18:35:47 UTC
Unsure if this is still a problem. CC'd lawrence as he was mentioned in the last comment and assigning back to -net.