Bug 141843 - [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
Summary: [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 8.0-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: jfv
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2009-12-21 15:00 UTC by Dennis Yusupoff
Modified: 2015-07-25 22:37 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dennis Yusupoff 2009-12-21 15:00:12 UTC
Then txcsum is enabled on interface, there are connectivity problem with
this host: packets sent to wrong MAC address. It was happening instantly
after we assign vlan id and dev to interface vlanXXXX, which was had
created before, see examples below.

Example.
Host 10.78.77.77 connected directly to server 10.78.77.88 by patch cord.
10.78.77.77 has mac 00:03:9d:73:e7:bd
10.78.77.88 has mac 00:30:48:63:db:e7

Establish TCP session by execute "telnet 10.78.77.88 22" from host
tcpdump -lneti fxp0 tcp port 22
----
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.57182: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 66: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 106: 10.78.77.88.22 > 10.78.77.77.57182: tcp 40
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 66: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0
---
IT's OK.

Let's assign id and dev to interface vlan2 at server and repeat attempt
to establish TCP session.

Tcpdump from server:
---
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 62: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
---

All looks fine (except unsuccessful attempts of establish session)...but
let's look at it from host side:
---
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > f0:4e:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > e4:96:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > e4:96:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > d8:16:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > d8:16:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 62: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0
00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0
---

Did you see difference? While server says, that he sends TCP packets to
proper MAC, _IN REAL_ it sends it to almost random MACs! Why "almost"?
Because if you'll take a look at destination MACs closer, you'll see:
---
d8:16:9d:73:e7:bd
e4:96:9d:73:e7:bd
e8:cf:9d:73:e7:bd
f0:4e:9d:73:e7:bd
---

Wow! First two octets are changing and, of course, are incorrect, while
other four octets doesn't changed and are corrects.

Of course, with that conditions no one could establish TCP session.
Nota Bene: ICMP, however, works fine in both cases.
UDP works unknown (doesn't check).

Conclusion:
With assigned vlans and enabled TXCSUM of [Intel] NIC, something in
OS/drivers works incorrectly and broke TCP packets by setting incorrect
MAC destination.


Technical information:
===
#pciconf -lv
hostb0@pci0:0:0:0:      class=0x060000 card=0xd18015d9 chip=0x29f08086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '3200 Chipset (Bearlake) Processor to I/O Controller'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:       class=0x060400 card=0xd18015d9 chip=0x29f18086 rev=0x01 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '3200 Chipset (Bearlake) PCIe Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
uhci0@pci0:0:26:0:      class=0x0c0300 card=0xd18015d9 chip=0x29378086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
uhci1@pci0:0:26:1:      class=0x0c0300 card=0xd18015d9 chip=0x29388086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
uhci2@pci0:0:26:2:      class=0x0c0300 card=0xd18015d9 chip=0x29398086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
ehci0@pci0:0:26:7:      class=0x0c0320 card=0xd18015d9 chip=0x293c8086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller'
    class      = serial bus
    subclass   = USB
pcib3@pci0:0:28:0:      class=0x060400 card=0xd18015d9 chip=0x29408086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:28:4:      class=0x060400 card=0xd18015d9 chip=0x29488086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 5'
    class      = bridge
    subclass   = PCI-PCI
pcib5@pci0:0:28:5:      class=0x060400 card=0xd18015d9 chip=0x294a8086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 6'
    class      = bridge
    subclass   = PCI-PCI
uhci3@pci0:0:29:0:      class=0x0c0300 card=0xd18015d9 chip=0x29348086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
uhci4@pci0:0:29:1:      class=0x0c0300 card=0xd18015d9 chip=0x29358086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
uhci5@pci0:0:29:2:      class=0x0c0300 card=0xd18015d9 chip=0x29368086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
    class      = serial bus
    subclass   = USB
ehci1@pci0:0:29:7:      class=0x0c0320 card=0xd18015d9 chip=0x293a8086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller'
    class      = serial bus
    subclass   = USB
pcib6@pci0:0:30:0:      class=0x060401 card=0xd18015d9 chip=0x244e8086 rev=0x92 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 card=0xd18015d9 chip=0x29168086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IR (ICH9R) LPC Interface Controller'
    class      = bridge
    subclass   = PCI-ISA
atapci1@pci0:0:31:2:    class=0x010601 card=0xd18015d9 chip=0x29228086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controller'
    class      = mass storage
    subclass   = SATA
none0@pci0:0:31:3:      class=0x0c0500 card=0xd18015d9 chip=0x29308086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) SMBus Controller'
    class      = serial bus
    subclass   = SMBus
none1@pci0:0:31:6:      class=0x118000 card=0x000015d9 chip=0x29328086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801IB/IR/IH (ICH9 Family) Thermal Subsystem'
    class      = dasp
pcib2@pci0:1:0:0:       class=0x060400 card=0x00000000 chip=0x032c8086 rev=0x09 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = 'PCI Express-to-PCI Express Bridge (6702PXH)'
    class      = bridge
    subclass   = PCI-PCI
ioapic0@pci0:1:0:1:     class=0x080020 card=0xd18015d9 chip=0x03268086 rev=0x09 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '6700/6702PXH I/OxAPIC Interrupt Controller A'
    class      = base peripheral
    subclass   = interrupt controller
em0@pci0:13:0:0:        class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel Corporation 82573E Gigabit Ethernet Controller (Copper) (82573E)'
    class      = network
    subclass   = ethernet
em1@pci0:15:0:0:        class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel PRO/1000 PL Network Adaptor (82573L)'
    class      = network
    subclass   = ethernet
vgapci0@pci0:17:3:0:    class=0x030000 card=0xd18015d9 chip=0x515e1002 rev=0x02 hdr=0x00
    vendor     = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.'
    device     = 'Radeon ES1000 (Radeon ES1000)'
    class      = display
    subclass   = VGA
atapci0@pci0:17:4:0:    class=0x010185 card=0x82131283 chip=0x82131283 rev=0x00 hdr=0x00
    vendor     = 'Integrated Technology Express (ITE) Inc'
    device     = 'IDE Controller (IT8213F)'
    class      = mass storage
    subclass   = ATA

[root@alpha /]# ps -axl
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT       TIME COMMAND
    0     0     0   0 -68  0     0    64 -      DLs   ??    0:00,09 [kernel]
    0     1     0   0  44  0  2912   508 wait   ILs   ??    0:00,00 /sbin/init --
    0     2     0   0  -8  0     0     8 -      DL    ??    0:00,09 [g_event]
    0     3     0   0  -8  0     0     8 -      DL    ??    0:00,10 [g_up]
    0     4     0   0  -8  0     0     8 -      DL    ??    0:00,09 [g_down]
    0     5     0   0 -16  0     0     8 ccb_sc DL    ??    0:00,00 [xpt_thrd]
    0     6     0   0 -16  0     0     8 waitin DL    ??    0:00,00 [sctp_iterator]
    0     7     0   0 -16  0     0     8 psleep DL    ??    0:00,00 [pagedaemon]
    0     8     0   0 -16  0     0     8 psleep DL    ??    0:00,00 [vmdaemon]
    0     9     0   0  76  0     0     8 pgzero DL    ??    0:00,00 [pagezero]
    0    10     0   0 -16  0     0     8 audit_ DL    ??    0:00,00 [audit]
    0    11     0   0 171  0     0    16 -      RL    ??  132:01,13 [idle]
    0    12     0   0 -64  0     0   144 -      WL    ??    0:02,56 [intr]
    0    13     0   0 -16  0     0     8 -      DL    ??    0:00,09 [yarrow]
    0    14     0   0 -64  0     0   256 -      DL    ??    0:00,09 [usb]
    0    15     0   0 -16  0     0     8 psleep DL    ??    0:00,01 [bufdaemon]
    0    16     0   0 -16  0     0     8 syncer DL    ??    0:00,04 [syncer]
    0    17     0   0 -16  0     0     8 vlruwt DL    ??    0:00,01 [vnlru]
    0    18     0   0 -16  0     0     8 sdflus DL    ??    0:00,02 [softdepflush]
    0    19     0   0 -16  0     0     8 flowcl DL    ??    0:00,00 [flowcleaner]
    0   409     1   0  50  0  1888   540 select Is    ??    0:00,01 /sbin/devd
  101   487     1   0  44  0  4004  1968 select Is    ??    0:00,00 /usr/local/sbin/zebra -d
  101   493     1   0  44  0  4140  2336 select Ss    ??    0:00,05 /usr/local/sbin/ospfd -d
    0   539     1   0  44  0  3344  1328 select Ss    ??    0:00,02 /usr/sbin/syslogd -s
    0   765     1   0  44  0  6676  3768 select Is    ??    0:00,01 /usr/sbin/sshd
    0   774     1   0  44  0  3372  1376 nanslp Ss    ??    0:00,01 /usr/sbin/cron -s
    0   801     1   0  76  0  3404  1272 select Is    ??    0:00,00 /usr/sbin/inetd -wW -C 60
    0  1224   765   0  45  0  9400  4404 sbwait Is    ??    0:00,02 sshd: dyr [priv] (sshd)
 1001  1227  1224   0  44  0  9400  4440 select S     ??    0:00,09 sshd: dyr@pts/0 (sshd)
    0   826     1   0  44  0  3808  1816 wait   Is    v0    0:00,01 login [pam] (login)
 1001   850   826   0  46  0  5584  2520 pause  I     v0    0:00,01 -tcsh (tcsh)
    0   853   850   0  44  0  4468  2424 ttyin  I+    v0    0:00,02 bash
    0   827     1   0  76  0  3344  1180 ttyin  Is+   v1    0:00,00 /usr/libexec/getty Pc ttyv1
    0   828     1   0  76  0  3344  1180 ttyin  Is+   v2    0:00,00 /usr/libexec/getty Pc ttyv2
    0   829     1   0  76  0  3344  1180 ttyin  Is+   v3    0:00,00 /usr/libexec/getty Pc ttyv3
    0   830     1   0  76  0  3344  1180 ttyin  Is+   v4    0:00,00 /usr/libexec/getty Pc ttyv4
    0   831     1   0  76  0  3344  1180 ttyin  Is+   v5    0:00,00 /usr/libexec/getty Pc ttyv5
    0   832     1   0  76  0  3344  1180 ttyin  Is+   v6    0:00,00 /usr/libexec/getty Pc ttyv6
    0   833     1   0  76  0  3344  1180 ttyin  Is+   v7    0:00,00 /usr/libexec/getty Pc ttyv7
 1001  1228  1227   0  46  0  5584  2588 pause  Is     0    0:00,01 -tcsh (tcsh)
    0  1231  1228   0  44  0  4468  2464 wait   S      0    0:00,06 bash
    0  1441  1231   0  44  0  3424  1252 -      R+     0    0:00,00 ps -axl
[root@alpha /]# vmstat -s
  1831477 cpu context switches
    10200 device interrupts
   623396 software interrupts
   115495 traps
   389816 system calls
       19 kernel threads created
     1380  fork() calls
       43 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      303 vnode pager pageins
     1987 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
       57 pages reactivated
    41718 copy-on-write faults
      131 copy-on-write optimized faults
    40215 zero fill pages zeroed
     1783 zero fill pages prezeroed
       14 intransit blocking page faults
   112421 total VM faults taken
        0 pages affected by kernel thread creation
   240965 pages affected by  fork()
     8596 pages affected by vfork()
        0 pages affected by rfork()
       64 pages cached
    96163 pages freed
        0 pages freed by daemon
    68636 pages freed by exiting processes
     2072 pages active
     1754 pages inactive
        6 pages in VM cache
     6774 pages wired down
   500685 pages free
     4096 bytes per page
    48089 total name lookups
          cache hits (86% pos + 9% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%
[root@alpha /]# vmstat -m
         Type InUse MemUse HighUse Requests  Size(s)
         GEOM    71    16K       -      510  16,32,64,128,512,1024
CAM dev queue     1     1K       -        1  64
       isadev     8     1K       -        8  64
    CAM queue     3     1K       -        7  16
     pci_link    16     2K       -       16  64,128
         cdev     9     2K       -        9  128
    acpi_perf     2     1K       -        2  128
        sigio     1     1K       -        1  32
     filedesc    48    17K       -     1746  32,256,2048
         kenv    79     7K       -       83  16,32,64,128,4096
       kqueue     0     0K       -        8  128,1024
    proc-args    23     2K       -      762  16,32,64,128
      ithread   104    10K       -      104  16,64,128
       KTRACE   100    13K       -      100  128
       linker   115    32K       -      153  16,32,256,1024,4096
        lockf    20     2K       -       20  32,64
       ip6ndp     4     1K       -        6  64,128
         temp    25   765K       -     8754  16,32,64,128,256,512,1024,2048,4096
       devbuf  4424  3492K       -     4495  16,32,64,128,256,512,1024,2048,4096
       kbdmux     7    10K       -        7  16,256,1024,2048,4096
       module   464    30K       -      464  64,128
     mtx_pool     2     8K       -        2  4096
      subproc   127   190K       -     1529  256,4096
         proc     2     8K       -        2  4096
      session    18     2K       -       55  64
         pgrp    22     2K       -      362  64
         cred    50     5K       -     9476  64,128
      uidinfo     4     2K       -       58  64,1024
       plimit    14     4K       -      512  256
    sysctltmp     0     0K       -      388  16,32,64,128
    sysctloid  3389   104K       -     3485  16,32,64,128
       sysctl     0     0K       -     1063  16,32,64
      callout     1   256K       -        1
         umtx   161    11K       -      161  64
     p1003.1b     1     1K       -        1  16
         SWAP     2   277K       -        2  64
       bus-sc    72   225K       -     4275  16,32,64,128,256,512,1024,2048,4096
          bus  1178    55K       -     7309  16,32,64,128,512,1024
      devstat     6    13K       -        6  16,4096
 eventhandler    73     4K       -       73  32,64,128
         kobj   325   650K       -      378  2048
      Per-cpu     1     1K       -        1  16
         rman   190    12K       -      693  16,32,64
         sbuf     0     0K       -      340  16,32,64,128,256,512,1024,2048,4096
       acpica  2271   119K       -    80413  16,32,64,128,256,512,1024,2048
     acpitask     1     1K       -        1  1024
      CAM SIM     1     1K       -        1  128
        stack     0     0K       -        2  128
    taskqueue    15     1K       -       15  16,64
       Unitno    11     1K       -       35  16,64
          iov     0     0K       -     1882  16,64,128,256
       select    28     2K       -       28  64
     ioctlops     0     0K       -     5025  16,32,64,128,256,512,1024,4096
          msg     4    25K       -        4  1024,4096
          sem     4     6K       -        4  256,512,1024,4096
          shm     1    12K       -        1
          tty    22    11K       -       25  512,2048
          pts     1     1K       -        2  128
     mbuf_tag     0     0K       -      488  32,64
         ksem     1     4K       -        1  4096
        shmfd     1     4K       -        1  4096
          pcb    18    79K       -       34  16,32,64,512,1024,2048,4096
       soname     7     1K       -     2021  16,32,128
       biobuf     4     8K       -        6  2048
     vfscache     1   512K       -        1
     vfs_hash     1   256K       -        1
       vnodes     2     1K       -        2  128
  vnodemarker     0     0K       -     1298  512
        mount    86     3K       -      130  16,32,64,128,256
          BPF     3     1K       -       41  16,64,256
  ether_multi    40     2K       -       46  16,32,64
       ifaddr    43     9K       -       55  16,32,64,128,256,512,2048
        ifnet     4     4K       -        6  64,1024
        clone     6    24K       -        6  4096
       arpcom     2     1K       -        4  16
      lltable    12     3K       -       18  128,256
      acpisem    19     3K       -       19  64,128
      CAM XPT    11     2K       -       30  16,32,64,1024
   CAM periph     2     1K       -       11  16,32,64,128
  ata_generic     1     1K       -        1  1024
     routetbl    59   260K       -      330  16,32,64,128,256,512
         igmp     3     1K       -        5  128
    ad_driver     1     1K       -        1  32
    ar_driver     0     0K       -        7  512,2048
  ip_moptions     2     1K       -        2  32,128
     in_multi     7     1K       -        7  128
   in_mfilter     1     1K       -        1  512
    sctp_iter     0     0K       -        4  128
     sctp_ifn     3     1K       -        3  128
     sctp_ifa     5     1K       -        5  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        4  16
    hostcache     1    16K       -        1
     syncache     1    72K       -        1
 ip6_moptions     2     1K       -        2  32,128
    in6_multi    12     2K       -       12  16,256
  in6_mfilter     1     1K       -        1  512
      entropy  1024    64K       -     1024  64
          mld     3     1K       -        5  128
      NFS FHA     1     1K       -        1  1024
          rpc     2     5K       -        2  128,4096
audit_evclass   172     3K       -      211  16
     savedino     0     0K       -       33  256
       dirrem     0     0K       -       82  32
        mkdir     0     0K       -       12  32
       diradd     0     0K       -       89  64
     freefile     0     0K       -       31  32
     freeblks     0     0K       -       26  256
     freefrag     0     0K       -        1  32
  allocdirect     1     1K       -       43  128
    bmsafemap     1     1K       -       28  64
       newblk     1     1K       -       44  64,256
     inodedep     2   257K       -      125  128
      pagedep     1    64K       -       23  64
  ufs_dirhash    33     7K       -       33  16,32,64,128,512
    ufs_mount    15    52K       -       15  256,2048,4096
    vm_pgdata     2    65K       -        2  64
     atkbddev     2     1K       -        2  32
      acpidev    81     3K       -       81  32
         UART     6     3K       -        6  16,256,1024
       apmdev     1     1K       -        1  64
       USBdev    41    11K       -       41  32,128,256,1024
          USB    66    79K       -       67  16,32,64,128,1024,4096
       DEVFS1   107    27K       -      118  256
       DEVFS3   123    16K       -      135  128
      io_apic     2     2K       -        2  1024
      memdesc     1     4K       -        1  4096
          msi     2     1K       -        2  64
     nexusdev     4     1K       -        4  16
        DEVFS    19     1K       -       20  16,64
       DEVFSP     0     0K       -        8  32
    pfs_nodes    20     3K       -       20  128
         vlan     0     0K       -        6  64
[root@alpha /]# vmstat -z
ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES

UMA Kegs:                 128,        0,       88,        2,       88,        0
UMA Zones:                888,        0,       88,        0,       88,        0
UMA Slabs:                284,        0,      511,        7,     1262,        0
UMA RCntSlabs:            544,        0,      475,        1,      475,        0
UMA Hash:                 128,        0,        4,       26,        4,        0
16 Bucket:                 76,        0,       53,       47,       74,        0
32 Bucket:                140,        0,       49,        7,       70,        0
64 Bucket:                268,        0,       48,        8,       94,       26
128 Bucket:               524,        0,       63,        0,     1064,      112
VM OBJECT:                136,        0,      915,      216,    21819,        0
MAP:                      140,        0,        7,       21,        7,        0
KMAP ENTRY:                72,    57505,       39,      226,     4277,        0
MAP ENTRY:                 72,        0,      507,      394,    40821,        0
DP fakepg:                 72,        0,        0,        0,        0,        0
SG fakepg:                 72,        0,        0,        0,        0,        0
mt_zone:                 2056,        0,      263,      127,      263,        0
16:                        16,        0,     2900,      348,    48137,        0
32:                        32,        0,     2790,      374,    42407,        0
64:                        64,        0,     4574,      146,    14228,        0
128:                      128,        0,     3329,      151,    16754,        0
256:                      256,        0,      874,      116,     4453,        0
512:                      512,        0,       65,       23,     4054,        0
1024:                    1024,        0,       45,      167,     4456,        0
2048:                    2048,        0,      361,       55,      719,        0
4096:                    4096,        0,       90,       39,     5304,        0
Files:                     56,        0,       66,      202,     8773,        0
TURNSTILE:                 72,        0,      162,       18,      162,        0
umtx pi:                   52,        0,        0,        0,        0,        0
MAC labels:                20,        0,        0,        0,        0,        0
PROC:                     680,        0,       41,       43,     1444,        0
THREAD:                   572,        0,      141,       20,      141,        0
SLEEPQUEUE:                32,        0,      162,      133,      162,        0
VMSPACE:                  232,        0,       23,       79,     1427,        0
cpuset:                    40,        0,        2,      182,        2,        0
audit_record:             816,        0,        0,        0,        0,        0
mbuf_packet:              256,        0,      512,      384,    10689,        0
mbuf:                     256,        0,        1,      393,    10792,        0
mbuf_cluster:            2048,    25600,      896,       26,      896,        0
mbuf_jumbo_page:         4096,    12800,        0,       14,       27,        0
mbuf_jumbo_9k:           9216,     6400,        0,        0,        0,        0
mbuf_jumbo_16k:         16384,     3200,        0,        0,        0,        0
mbuf_ext_refcnt:            4,        0,        0,        0,        0,        0
g_bio:                    140,        0,        0,      224,     9073,        0
ttyinq:                   152,        0,      135,       47,      285,        0
ttyoutq:                  256,        0,       72,       18,      152,        0
ata_request:              200,        0,        0,      114,     2277,        0
ata_composite:            180,        0,        0,        0,        0,        0
VNODE:                    268,        0,      532,       14,      565,        0
VNODEPOLL:                 60,        0,        0,        0,        0,        0
S VFS Cache:               72,        0,      512,      124,     2017,        0
L VFS Cache:              292,        0,        0,        0,        0,        0
NAMEI:                   1024,        0,        0,       36,    19250,        0
DIRHASH:                 1024,        0,       42,       14,       42,        0
NFSMOUNT:                 520,        0,        0,        0,        0,        0
NFSNODE:                  464,        0,        0,        0,        0,        0
pipe:                     392,        0,        2,       48,      835,        0
ksiginfo:                  80,        0,       75,      981,       75,        0
itimer:                   220,        0,        0,        0,        0,        0
KNOTE:                     68,        0,        0,      112,       32,        0
socket:                   412,    25605,       24,       39,      616,        0
unpcb:                    172,    25622,       12,       80,      190,        0
ipq:                       32,      904,        0,        0,        0,        0
udp_inpcb:                220,    25614,        2,       52,      406,        0
udpcb:                      8,    25781,        2,      404,      406,        0
tcp_inpcb:                220,    25614,        7,       47,       17,        0
tcpcb:                    632,    25602,        7,       17,       17,        0
tcptw:                     52,     5184,        0,      144,        4,        0
syncache:                 112,    15365,        0,      105,       19,        0
hostcache:                 76,    15400,        2,      148,        2,        0
tcpreass:                  20,     1690,        0,      169,        6,        0
sackhole:                  20,        0,        0,      169,        1,        0
sctp_ep:                  848,    25600,        0,        0,        0,        0
sctp_asoc:               1460,    40000,        0,        0,        0,        0
sctp_laddr:                24,    80040,        0,      290,        4,        0
sctp_raddr:               420,    80001,        0,        0,        0,        0
sctp_chunk:                96,   400000,        0,        0,        0,        0
sctp_readq:                76,   400000,        0,        0,        0,        0
sctp_stream_msg_out:       64,   400020,        0,        0,        0,        0
sctp_asconf:               24,   400055,        0,        0,        0,        0
sctp_asconf_ack:           24,   400055,        0,        0,        0,        0
ripcb:                    220,    25614,        2,       34,        2,        0
rtentry:                  108,        0,       12,       96,       22,        0
selfd:                     28,        0,       60,      448,    83248,        0
ip4flow:                   40,     4140,        3,      273,       75,        0
ip6flow:                   64,     4118,        0,        0,        0,        0
SWAPMETA:                 276,   121576,        0,        0,        0,        0
Mountpoints:              644,        0,        6,       12,        6,        0
FFS inode:                116,        0,      496,       32,      527,        0
FFS1 dinode:              128,        0,        0,        0,        0,        0
FFS2 dinode:              256,        0,      496,       44,      527,        0

[root@alpha /]# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           1          0
irq16: uhci0                         274          0
irq17: uhci1+                       2276          0
irq18: uhci2 ehci+                     2          0
cpu0: timer                      7637268       1921
irq256: em0                         6672          1
irq257: em1                          999          0
cpu1: timer                      7637143       1921
Total                           15284635       3845
[root@alpha /]# pstat -T
 66/12328 files
0M/2048M swap space
[root@alpha /]# pstat -s
Device          512-blocks     Used    Avail Capacity
/dev/ad6s1b        4194304        0  4194304     0%
[root@alpha /]# iostat
       tty             ad6             cpu
 tin  tout  KB/t tps  MB/s  us ni sy in id
   1    64 14.15   1  0.01   0  0  0  0 100
[root@alpha /]# ipcs -a
Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME

[root@alpha /]# ipcs -T
msginfo:
        msgmax:        16384    (max characters in a message)
        msgmni:           40    (# of message queues)
        msgmnb:         2048    (max characters in a message queue)
        msgtql:           40    (max # of messages in system)
        msgssz:            8    (size of a message segment)
        msgseg:         2048    (# of message segments in system)

shminfo:
        shmmax:     33554432    (max shared memory segment size)
        shmmin:            1    (min shared memory segment size)
        shmmni:          192    (max number of shared memory identifiers)
        shmseg:          128    (max shared memory segments per process)
        shmall:         8192    (max amount of shared memory in pages)

seminfo:
        semmap:           30    (# of entries in semaphore map)
        semmni:           10    (# of semaphore identifiers)
        semmns:           60    (# of semaphores in system)
        semmnu:           30    (# of undo structures in system)
        semmsl:           60    (max # of semaphores per id)
        semopm:          100    (max # of operations per semop call)
        semume:           10    (max # of undo entries per process)
        semusz:          136    (size in bytes of undo structure)
        semvmx:        32767    (semaphore maximum value)
        semaem:        16384    (adjust on exit max value)

[root@alpha /]# w -dn
14:45  up  1:06, 2 users, load averages: 0,00 0,00 0,00
USER             TTY      FROM              LOGIN@  IDLE WHAT
                826       login [pam] (login)
                850       -tcsh (tcsh)
                853       bash
dyr              v0       -                13:39      26 bash
                1228      -tcsh (tcsh)
                1231      bash
                1451      w -dn
dyr              pts/0    217.119.16.88    14:18       - w -dn
[root@alpha /]# nfsstat
Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses
        0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0
[root@alpha /]# netstat -s
tcp:
        3259 packets sent
                3130 data packets (390516 bytes)
                179 data packets (85318 bytes) retransmitted
                0 data packets unnecessarily retransmitted
                0 resends initiated by MTU discovery
                87 ack-only packets (43 delayed)
                0 URG only packets
                0 window probe packets
                1 window update packet
                10 control packets
        4559 packets received
                3105 acks (for 391796 bytes)
                61 duplicate acks
                0 acks for unsent data
                1373 packets (77061 bytes) received in-sequence
                23 completely duplicate packets (5285 bytes)
                0 old duplicate packets
                3 packets with some dup. data (41 bytes duped)
                6 out-of-order packets (336 bytes)
                0 packets (0 bytes) of data after window
                0 window probes
                5 window update packets
                1 packet received after close
                2 discarded for bad checksums
                0 discarded for bad header offset fields
                0 discarded because packet too short
                0 discarded due to memory problems
        0 connection requests
        11 connection accepts
        0 bad connection attempts
        0 listen queue overflows
        1 ignored RSTs in the window
        11 connections established (including accepts)
        10 connections closed (including 1 drop)
                4 connections updated cached RTT on close
                5 connections updated cached RTT variance on close
                1 connection updated cached ssthresh on close
        0 embryonic connections dropped
        3105 segments updated rtt (of 1714 attempts)
        41 retransmit timeouts
                1 connection dropped by rexmit timeout
        0 persist timeouts
                0 connections dropped by persist timeout
        0 Connections (fin_wait_2) dropped because of timeout
        0 keepalive timeouts
                0 keepalive probes sent
                0 connections dropped by keepalive
        0 correct ACK header predictions
        1082 correct data packet header predictions
        19 syncache entries added
                43 retransmitted
                26 dupsyn
                0 dropped
                11 completed
                0 bucket overflow
                0 cache overflow
                3 reset
                5 stale
                0 aborted
                0 badack
                0 unreach
                0 zone failures
        19 cookies sent
        0 cookies received
        2 SACK recovery episodes
        1 segment rexmit in SACK recovery episodes
        480 byte rexmits in SACK recovery episodes
        34 SACK options (SACK blocks) received
        4 SACK options (SACK blocks) sent
        0 SACK scoreboard overflow
        0 packets with ECN CE bit set
        0 packets with ECN ECT(0) bit set
        0 packets with ECN ECT(1) bit set
        0 successful ECN handshakes
        0 times ECN reduced the congestion window
udp:
        24 datagrams received
        0 with incomplete header
        0 with bad data length field
        0 with bad checksum
        0 with no checksum
        16 dropped due to no socket
        8 broadcast/multicast datagrams undelivered
        0 dropped due to full socket buffers
        0 not for hashed pcb
        0 delivered
        16 datagrams output
        0 times multicast source filter matched
sctp:
        0 input packets
                0 datagrams
                0 packets that had data
                0 input SACK chunks
                0 input DATA chunks
                0 duplicate DATA chunks
                0 input HB chunks
                0 HB-ACK chunks
                0 input ECNE chunks
                0 input AUTH chunks
                0 chunks missing AUTH
                0 invalid HMAC ids received
                0 invalid secret ids received
                0 auth failed
                0 fast path receives all one chunk
                0 fast path multi-part data
        0 output packets
                0 output SACKs
                0 output DATA chunks
                0 retransmitted DATA chunks
                0 fast retransmitted DATA chunks
                0 FR's that happened more than once to same chunk
                0 intput HB chunks
                0 output ECNE chunks
                0 output AUTH chunks
                0 ip_output error counter
        Packet drop statistics:
                0 from middle box
                0 from end host
                0 with data
                0 non-data, non-endhost
                0 non-endhost, bandwidth rep only
                0 not enough for chunk header
                0 not enough data to confirm
                0 where process_chunk_drop said break
                0 failed to find TSN
                0 attempt reverse TSN lookup
                0 e-host confirms zero-rwnd
                0 midbox confirms no space
                0 data did not match TSN
                0 TSN's marked for Fast Retran
        Timeouts:
                0 iterator timers fired
                0 T3 data time outs
                0 window probe (T3) timers fired
                0 INIT timers fired
                0 sack timers fired
                0 shutdown timers fired
                0 heartbeat timers fired
                0 a cookie timeout fired
                0 an endpoint changed its cookiesecret
                0 PMTU timers fired
                0 shutdown ack timers fired
                0 shutdown guard timers fired
                0 stream reset timers fired
                0 early FR timers fired
                0 an asconf timer fired
                0 auto close timer fired
                0 asoc free timers expired
                0 inp free timers expired
        0 packet shorter than header
        0 checksum error
        0 no endpoint for port
        0 bad v-tag
        0 bad SID
        0 no memory
        0 number of multiple FR in a RTT window
        0 RFC813 allowed sending
        0 RFC813 does not allow sending
        0 times max burst prohibited sending
        0 look ahead tells us no memory in interface
        0 numbers of window probes sent
        0 times an output error to clamp down on next user send
        0 times sctp_senderrors were caused from a user
        0 number of in data drops due to chunk limit reached
        0 number of in data drops due to rwnd limit reached
        0 times a ECN reduced the cwnd
        0 used express lookup via vtag
        0 collision in express lookup
        0 times the sender ran dry of user data on primary
        0 same for above
        0 sacks the slow way
        0 window update only sacks sent
        0 sends with sinfo_flags !=0
        0 unordered sends
        0 sends with EOF flag set
        0 sends with ABORT flag set
        0 times protocol drain called
        0 times we did a protocol drain
        0 times recv was called with peek
        0 cached chunks used
        0 cached stream oq's used
        0 unread messages abandonded by close
        0 send burst avoidance, already max burst inflight to net
        0 send cwnd full avoidance, already max burst inflight to net
        0 number of map array over-runs via fwd-tsn's
ip:
        5333 total packets received
        0 bad header checksums
        0 with size smaller than minimum
        0 with data size < data length
        0 with ip length > max ip packet size
        0 with header length < data size
        0 with data length < header length
        0 with bad options
        0 with incorrect version number
        0 fragments received
        0 fragments dropped (dup or out of space)
        0 fragments dropped after timeout
        0 packets reassembled ok
        5315 packets for this host
        8 packets for unknown/unsupported protocol
        0 packets forwarded (0 packets fast forwarded)
        10 packets not forwardable
        0 packets received for unknown multicast group
        0 redirects sent
        3597 packets sent from this host
        872 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        191 output packets discarded due to no route
        0 output datagrams fragmented
        0 fragments created
        0 datagrams that can't be fragmented
        0 tunneling packets that can't find gif
        0 datagrams with bad address in header
icmp:
        8 calls to icmp_error
        0 errors not generated in response to an icmp message
        Output histogram:
                echo reply: 87
                destination unreachable: 8
        0 messages with bad code fields
        0 messages less than the minimum length
        0 messages with bad checksum
        0 messages with bad length
        0 multicast echo requests ignored
        0 multicast timestamp requests ignored
        Input histogram:
                destination unreachable: 8
                echo: 87
        87 message responses generated
        0 invalid return addresses
        0 no return routes
        ICMP address mask responses are disabled
igmp:
        0 messages received
        0 messages received with too few bytes
        0 messages received with wrong TTL
        0 messages received with bad checksum
        0 V1/V2 membership queries received
        0 V3 membership queries received
        0 membership queries received with invalid field(s)
        0 general queries received
        0 group queries received
        0 group-source queries received
        0 group-source queries dropped
        0 membership reports received
        0 membership reports received with invalid field(s)
        0 membership reports received for groups to which we belong
        0 V3 reports received without Router Alert
        8 membership reports sent
ip6:
        16 total packets received
        0 with size smaller than minimum
        0 with data size < data length
        0 with bad options
        0 with incorrect version number
        0 fragments received
        0 fragments dropped (dup or out of space)
        0 fragments dropped after timeout
        0 fragments that exceeded limit
        0 packets reassembled ok
        8 packets for this host
        0 packets forwarded
        0 packets not forwardable
        0 redirects sent
        16 packets sent from this host
        0 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        0 output packets discarded due to no route
        0 output datagrams fragmented
        0 fragments created
        0 datagrams that can't be fragmented
        0 packets that violated scope rules
        0 multicast packets which we don't join
        Input histogram:
                UDP: 8
                ICMP6: 8
        Mbuf statistics:
                8 one mbuf
                two or more mbuf:
                        lo0= 8
                0 one ext mbuf
                0 two or more ext mbuf
        0 packets whose headers are not continuous
        0 tunneling packets that can't find gif
        0 packets discarded because of too many headers
        0 failures of source address selection
        Source addresses selection rule applied:
                8 first candidate
                8 same address
icmp6:
        8 calls to icmp6_error
        0 errors not generated in response to an icmp6 message
        0 errors not generated because of rate limitation
        Output histogram:
                unreach: 8
        0 messages with bad code fields
        0 messages < minimum length
        0 bad checksums
        0 messages with bad length
        Input histogram:
                unreach: 8
        Histogram of error messages to be generated:
                0 no route
                0 administratively prohibited
                0 beyond scope
                0 address unreachable
                8 port unreachable
                0 packet too big
                0 time exceed transit
                0 time exceed reassembly
                0 erroneous header field
                0 unrecognized next header
                0 unrecognized option
                0 redirect
                0 unknown
        0 message responses generated
        0 messages with too many ND options
        0 messages with bad ND options
        0 bad neighbor solicitation messages
        0 bad neighbor advertisement messages
        0 bad router solicitation messages
        0 bad router advertisement messages
        0 bad redirect messages
        0 path MTU changes
rip6:
        0 messages received
        0 checksum calculations on inbound
        0 messages with bad checksum
        0 messages dropped due to no socket
        0 multicast messages dropped due to no socket
        0 messages dropped due to full socket buffers
        0 delivered
        0 datagrams output
[root@alpha /]# netstat -m
513/777/1290 mbufs in use (current/cache/total)
512/410/922/25600 mbuf clusters in use (current/cache/total/max)
512/384 mbuf+clusters out of packet secondary zone in use (current/cache)
0/14/14/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
1152K/1070K/2222K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/4/6656 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines
[root@alpha /]# netstat -id
Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs  Coll Drop
em0    1500 <Link#1>      00:30:48:63:db:e6     5943     0     3704     0     0    0
em0    1500 109.71.177.64 109.71.177.88         4509     -     3636     -     -    -
em1    1500 <Link#2>      00:30:48:63:db:e7      400     0      674     0     0    0
em1    1500 10.78.77.0    10.78.77.88            247     -      669     -     -    -
lo0   16384 <Link#3>                              48     0       48     0     0    0
lo0   16384 fe80:3::1     fe80:3::1                0     -        0     -     -    -
lo0   16384 localhost     ::1                      0     -       24     -     -    -
lo0   16384 your-net      localhost               24     -       24     -     -    -
[root@alpha /]# netstat -anr
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            109.71.177.65      UG1         3     2578    em0
10.78.77.0/24      link#2             U           0      249    em1
10.78.77.88        link#2             UHS         0        0    lo0
109.71.177.64/27   link#1             U           0        0    em0
109.71.177.88      link#1             UHS         0        0    lo0
127.0.0.1          link#3             UH          0       24    lo0
217.119.16.252/30  109.71.177.65      UG1         0        0    em0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UH          lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff01:3::/32                       fe80::1%lo0                   U           lo0
ff02::%lo0/32                     fe80::1%lo0                   U           lo0
[root@alpha /]# netstat -anA
Active Internet connections (including servers)
Tcpcb    Proto Recv-Q Send-Q  Local Address      Foreign Address   (state)
c5e71278 tcp4       0      0 109.71.177.88.22   217.119.16.88.5285 ESTABLISHED
c5c8f278 tcp4       0      0 *.22               *.*                LISTEN
c5c8f4f0 tcp6       0      0 *.22               *.*                LISTEN
c5c8f768 tcp4       0      0 *.2604             *.*                LISTEN
c5c8f9e0 tcp6       0      0 *.2604             *.*                LISTEN
c5c8fc58 tcp4       0      0 *.2601             *.*                LISTEN
c5c90000 tcp6       0      0 *.2601             *.*                LISTEN
c5b09528 udp4       0      0 *.514              *.*
c5b09604 udp6       0      0 *.514              *.*
c5c8c000 icm6       0      0 *.*                *.*
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
c5b0cd70 stream      0      0        0 c5b0d560        0        0
c5b0d560 stream      0      0        0 c5b0cd70        0        0
c5b0d764 stream      0      0        0 c5b0d810        0        0 /var/run/quagga/zserv.api
c5b0d810 stream      0      0        0 c5b0d764        0        0
c5b0d8bc stream      0      0 c5c88648        0        0        0 /var/run/quagga/ospfd.vty
c5b0dac0 stream      0      0 c5c8896c        0        0        0 /var/run/quagga/zebra.vty
c5b0db6c stream      0      0 c5c88a78        0        0        0 /var/run/quagga/zserv.api
c5b0cec8 stream      0      0 c5c7c860        0        0        0 /var/run/devd.pipe
c5b0c158 dgram       0      0        0 c5b0d6b8        0        0
c5b0cb6c dgram       0      0        0 c5b0d60c        0        0
c5b0d60c dgram       0      0 c5c8810c        0 c5b0cb6c        0 /var/run/logpriv
c5b0d6b8 dgram       0      0 c5c88218        0 c5b0c158        0 /var/run/log
[root@alpha /]# netstat -aL
Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address
tcp4  0/0/128        *.ssh
tcp6  0/0/128        *.ssh
tcp4  0/0/3          *.ospfd
tcp6  0/0/3          *.ospfd
tcp4  0/0/3          *.zebra
tcp6  0/0/3          *.zebra
unix  0/0/5          /var/run/quagga/ospfd.vty
unix  0/0/5          /var/run/quagga/zebra.vty
unix  0/0/5          /var/run/quagga/zserv.api
unix  0/0/4          /var/run/devd.pipe
[root@alpha /]# fstat
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     fstat       1459 root /             2 drwxr-xr-x     512  r
root     fstat       1459   wd /             2 drwxr-xr-x     512  r
root     fstat       1459 text /usr     1789987 -r-xr-sr-x   17672  r
root     fstat       1459    0 /dev        104 crw--w----   pts/0 rw
root     fstat       1459    1 /dev        104 crw--w----   pts/0 rw
root     fstat       1459    2 /dev        104 crw--w----   pts/0 rw
root     fstat       1459    3 /dev          8 crw-r-----     mem  r
root     fstat       1459    4 /dev          9 crw-r-----    kmem  r
root     fstat       1459    5 /        118128 -rw-------   40960  r
root     bash        1231 root /             2 drwxr-xr-x     512  r
root     bash        1231   wd /             2 drwxr-xr-x     512  r
root     bash        1231 text /usr     1039162 -rwxr-xr-x  620152  r
root     bash        1231    0 /dev        104 crw--w----   pts/0 rw
root     bash        1231    1 /dev        104 crw--w----   pts/0 rw
root     bash        1231    2 /dev        104 crw--w----   pts/0 rw
root     bash        1231  255 /dev        104 crw--w----   pts/0 rw
dyr      tcsh        1228 root /             2 drwxr-xr-x     512  r
dyr      tcsh        1228   wd /usr     2284545 drwxr-xr-x     512  r
dyr      tcsh        1228 text /        235560 -r-xr-xr-x  327760  r
dyr      tcsh        1228   15 /dev        104 crw--w----   pts/0 rw
dyr      tcsh        1228   16 /dev        104 crw--w----   pts/0 rw
dyr      tcsh        1228   17 /dev        104 crw--w----   pts/0 rw
dyr      tcsh        1228   18 /dev        104 crw--w----   pts/0 rw
dyr      tcsh        1228   19 /dev        104 crw--w----   pts/0 rw
dyr      sshd        1227 root /             2 drwxr-xr-x     512  r
dyr      sshd        1227   wd /             2 drwxr-xr-x     512  r
dyr      sshd        1227 text /usr     2331651 -r-xr-xr-x  222192  r
dyr      sshd        1227    0 /dev         11 crw-rw-rw-    null rw
dyr      sshd        1227    1 /dev         11 crw-rw-rw-    null rw
dyr      sshd        1227    2 /dev         11 crw-rw-rw-    null rw
dyr      sshd        1227    3* internet stream tcp c5e71278
dyr      sshd        1227    4* local stream c5b0d560 <-> c5b0cd70
dyr      sshd        1227    5* pipe c5a3c7a8 <-> c5a3c860      0 rw
dyr      sshd        1227    6* pipe c5a3c860 <-> c5a3c7a8      0 rw
dyr      sshd        1227    7* pseudo-terminal master      pts/0 rw
dyr      sshd        1227    9* pseudo-terminal master      pts/0 rw
dyr      sshd        1227   10* pseudo-terminal master      pts/0 rw
root     sshd        1224 root /             2 drwxr-xr-x     512  r
root     sshd        1224   wd /             2 drwxr-xr-x     512  r
root     sshd        1224 text /usr     2331651 -r-xr-xr-x  222192  r
root     sshd        1224    0 /dev         11 crw-rw-rw-    null  r
root     sshd        1224    1 /dev         11 crw-rw-rw-    null rw
root     sshd        1224    2 /dev         11 crw-rw-rw-    null rw
root     sshd        1224    3* internet stream tcp c5e71278
root     sshd        1224    4* pseudo-terminal master      pts/0 rw
root     sshd        1224    5* local stream c5b0cd70 <-> c5b0d560
root     bash         853 root /             2 drwxr-xr-x     512  r
root     bash         853   wd /usr     2284545 drwxr-xr-x     512  r
root     bash         853 text /usr     1039162 -rwxr-xr-x  620152  r
root     bash         853    0 /dev         49 crw-------   ttyv0 rw
root     bash         853    1 /dev         49 crw-------   ttyv0 rw
root     bash         853    2 /dev         49 crw-------   ttyv0 rw
root     bash         853  255 /dev         49 crw-------   ttyv0 rw
dyr      tcsh         850 root /             2 drwxr-xr-x     512  r
dyr      tcsh         850   wd /usr     2284545 drwxr-xr-x     512  r
dyr      tcsh         850 text /        235560 -r-xr-xr-x  327760  r
dyr      tcsh         850   15 /dev         49 crw-------   ttyv0 rw
dyr      tcsh         850   16 /dev         49 crw-------   ttyv0 rw
dyr      tcsh         850   17 /dev         49 crw-------   ttyv0 rw
dyr      tcsh         850   18 /dev         49 crw-------   ttyv0 rw
dyr      tcsh         850   19 /dev         49 crw-------   ttyv0 rw
root     getty        833 root /             2 drwxr-xr-x     512  r
root     getty        833   wd /             2 drwxr-xr-x     512  r
root     getty        833 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        833    0 /dev         56 crw-------   ttyv7 rw
root     getty        833    1 /dev         56 crw-------   ttyv7 rw
root     getty        833    2 /dev         56 crw-------   ttyv7 rw
root     getty        832 root /             2 drwxr-xr-x     512  r
root     getty        832   wd /             2 drwxr-xr-x     512  r
root     getty        832 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        832    0 /dev         55 crw-------   ttyv6 rw
root     getty        832    1 /dev         55 crw-------   ttyv6 rw
root     getty        832    2 /dev         55 crw-------   ttyv6 rw
root     getty        831 root /             2 drwxr-xr-x     512  r
root     getty        831   wd /             2 drwxr-xr-x     512  r
root     getty        831 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        831    0 /dev         54 crw-------   ttyv5 rw
root     getty        831    1 /dev         54 crw-------   ttyv5 rw
root     getty        831    2 /dev         54 crw-------   ttyv5 rw
root     getty        830 root /             2 drwxr-xr-x     512  r
root     getty        830   wd /             2 drwxr-xr-x     512  r
root     getty        830 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        830    0 /dev         53 crw-------   ttyv4 rw
root     getty        830    1 /dev         53 crw-------   ttyv4 rw
root     getty        830    2 /dev         53 crw-------   ttyv4 rw
root     getty        829 root /             2 drwxr-xr-x     512  r
root     getty        829   wd /             2 drwxr-xr-x     512  r
root     getty        829 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        829    0 /dev         52 crw-------   ttyv3 rw
root     getty        829    1 /dev         52 crw-------   ttyv3 rw
root     getty        829    2 /dev         52 crw-------   ttyv3 rw
root     getty        828 root /             2 drwxr-xr-x     512  r
root     getty        828   wd /             2 drwxr-xr-x     512  r
root     getty        828 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        828    0 /dev         51 crw-------   ttyv2 rw
root     getty        828    1 /dev         51 crw-------   ttyv2 rw
root     getty        828    2 /dev         51 crw-------   ttyv2 rw
root     getty        827 root /             2 drwxr-xr-x     512  r
root     getty        827   wd /             2 drwxr-xr-x     512  r
root     getty        827 text /usr     172036 -r-xr-xr-x   21860  r
root     getty        827    0 /dev         50 crw-------   ttyv1 rw
root     getty        827    1 /dev         50 crw-------   ttyv1 rw
root     getty        827    2 /dev         50 crw-------   ttyv1 rw
root     login        826 root /             2 drwxr-xr-x     512  r
root     login        826   wd /usr     2284545 drwxr-xr-x     512  r
root     login        826 text /usr     1790410 -r-sr-xr-x   21804  r
root     login        826    0 /dev         49 crw-------   ttyv0 rw
root     login        826    1 /dev         49 crw-------   ttyv0 rw
root     login        826    2 /dev         49 crw-------   ttyv0 rw
root     login        826    3* local dgram c5b0cb6c <-> c5b0d60c
root     inetd        801 root /             2 drwxr-xr-x     512  r
root     inetd        801   wd /             2 drwxr-xr-x     512  r
root     inetd        801 text /usr     2332017 -r-xr-xr-x   42188  r
root     inetd        801    0 /dev         11 crw-rw-rw-    null rw
root     inetd        801    1 /dev         11 crw-rw-rw-    null rw
root     inetd        801    2 /dev         11 crw-rw-rw-    null rw
root     inetd        801    3 /var     212003 -rw-------       3  w
root     inetd        801    4* pipe c5a3c620 <-> c5a3c6d8      0 rw
root     inetd        801    5* pipe c5a3c6d8 <-> c5a3c620      0 rw
root     cron         774 root /             2 drwxr-xr-x     512  r
root     cron         774   wd /var     635904 drwxr-x---     512  r
root     cron         774 text /usr     2331971 -r-xr-xr-x   34080  r
root     cron         774    0 /dev         11 crw-rw-rw-    null rw
root     cron         774    1 /dev         11 crw-rw-rw-    null rw
root     cron         774    2 /dev         11 crw-rw-rw-    null rw
root     cron         774    3 /var     212002 -rw-------       3  w
root     sshd         765 root /             2 drwxr-xr-x     512  r
root     sshd         765   wd /             2 drwxr-xr-x     512  r
root     sshd         765 text /usr     2331651 -r-xr-xr-x  222192  r
root     sshd         765    0 /dev         11 crw-rw-rw-    null rw
root     sshd         765    1 /dev         11 crw-rw-rw-    null rw
root     sshd         765    2 /dev         11 crw-rw-rw-    null rw
root     sshd         765    3* internet6 stream tcp c5c8f4f0
root     sshd         765    4* internet stream tcp c5c8f278
root     syslogd      539 root /             2 drwxr-xr-x     512  r
root     syslogd      539   wd /             2 drwxr-xr-x     512  r
root     syslogd      539 text /usr     2332129 -r-xr-xr-x   35916  r
root     syslogd      539    0 /dev         11 crw-rw-rw-    null rw
root     syslogd      539    1 /dev         11 crw-rw-rw-    null rw
root     syslogd      539    2 /dev         11 crw-rw-rw-    null rw
root     syslogd      539    3 /var     211990 -rw-------       3  w
root     syslogd      539    4* local dgram c5b0d6b8
root     syslogd      539    5* local dgram c5b0d60c
root     syslogd      539    6* internet6 dgram udp c5b09604
root     syslogd      539    7* internet dgram udp c5b09528
root     syslogd      539    8 /dev         10 crw-------    klog  r
root     syslogd      539   10 -         -         bad    -
root     syslogd      539   11 /var     235541 -rw-r--r--   71647  w
root     syslogd      539   12 /var     235531 -rw-------      55  w
root     syslogd      539   13 /var     235525 -rw-------   31796  w
root     syslogd      539   14 /var     235536 -rw-r-----    7571  w
root     syslogd      539   15 /var     235527 -rw-r--r--      55  w
root     syslogd      539   16 /var     235532 -rw-------    9873  w
root     syslogd      539   17 /var     235546 -rw-------   63558  w
root     syslogd      539   18 /var     235526 -rw-------      55  w
root     syslogd      539   19 /var     235530 -rw-r-----      55  w
quagga   ospfd        493 root /             2 drwxr-xr-x     512  r
quagga   ospfd        493   wd /             2 drwxr-xr-x     512  r
quagga   ospfd        493 text /usr     1060003 -r-xr-xr-x    8496  r
quagga   ospfd        493    0 /dev         11 crw-rw-rw-    null rw
quagga   ospfd        493    1 /dev         11 crw-rw-rw-    null rw
quagga   ospfd        493    2 /dev         11 crw-rw-rw-    null rw
quagga   ospfd        493    3 /var     211988 -rw-r--r--       4 rw
quagga   ospfd        493    4* internet raw ip c5c8be9c
quagga   ospfd        493    5* internet6 stream tcp c5c8f9e0
quagga   ospfd        493    6* internet stream tcp c5c8f768
quagga   ospfd        493    7* local stream c5b0d8bc
quagga   ospfd        493    8* local stream c5b0d810 <-> c5b0d764
quagga   zebra        487 root /             2 drwxr-xr-x     512  r
quagga   zebra        487   wd /             2 drwxr-xr-x     512  r
quagga   zebra        487 text /usr     1060007 -r-xr-xr-x  169888  r
quagga   zebra        487    0 /dev         11 crw-rw-rw-    null rw
quagga   zebra        487    1 /dev         11 crw-rw-rw-    null rw
quagga   zebra        487    2 /dev         11 crw-rw-rw-    null rw
quagga   zebra        487    3* internet6 raw ipv6-icmp c5c8c000
quagga   zebra        487    4* route raw 0 c5b07b44
quagga   zebra        487    5 /var     211985 -rw-r--r--       4 rw
quagga   zebra        487    6* local stream c5b0db6c
quagga   zebra        487    7* internet6 stream tcp c5c90000
quagga   zebra        487    8* internet stream tcp c5c8fc58
quagga   zebra        487    9* local stream c5b0dac0
quagga   zebra        487   10* local stream c5b0d764 <-> c5b0d810
quagga   zebra        487   11* local dgram c5b0c158 <-> c5b0d6b8
root     devd         409 root /             2 drwxr-xr-x     512  r
root     devd         409   wd /             2 drwxr-xr-x     512  r
root     devd         409 text /        141327 -r-xr-xr-x  381640  r
root     devd         409    0 /dev         11 crw-rw-rw-  
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2009-12-21 23:51:33 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-net

Over to maintainer(s).
Comment 2 pyunyh 2009-12-29 18:51:19 UTC
On Mon, Dec 21, 2009 at 11:53:22PM +0000, linimon@freebsd.org wrote:
> Old Synopsis: Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
> New Synopsis: [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
> 
> Responsible-Changed-From-To: freebsd-bugs->freebsd-net
> Responsible-Changed-By: linimon
> Responsible-Changed-When: Mon Dec 21 23:51:33 UTC 2009
> Responsible-Changed-Why: 
> Over to maintainer(s).
> 
> http://www.freebsd.org/cgi/query-pr.cgi?pr=141843

I'm not sure whether you're seeing one of edge case of em(4)/igb(4)
checksum offload issue. Personally I couldn't reproduce the issue
but I guess the checksum offload/TSO handling might cause
unexpected result. If you disable Tx checksum offload or TSO em(4)
work as expected, right? If so, would you try the following patch
and let me know whether it makes any difference?

http://people.freebsd.org/~yongari/em.csum_tso.20091229.patch
Comment 3 Dennis Yusupoff 2009-12-30 10:34:30 UTC
This is a multi-part message in MIME format.
Comment 4 pyunyh 2009-12-30 20:20:14 UTC
On Wed, Dec 30, 2009 at 01:34:30PM +0300, Dennis Yusupoff wrote:
> 29.12.2009 21:51, Pyun YongHyeon ?????:
> > On Mon, Dec 21, 2009 at 11:53:22PM +0000, linimon@freebsd.org wrote:
> >   
> >> Old Synopsis: Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
> >> New Synopsis: [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets
> >>
> >> Responsible-Changed-From-To: freebsd-bugs->freebsd-net
> >> Responsible-Changed-By: linimon
> >> Responsible-Changed-When: Mon Dec 21 23:51:33 UTC 2009
> >> Responsible-Changed-Why: 
> >> Over to maintainer(s).
> >>
> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=141843
> >>     
> > I'm not sure whether you're seeing one of edge case of em(4)/igb(4)
> > checksum offload issue. Personally I couldn't reproduce the issue
> > but I guess the checksum offload/TSO handling might cause
> > unexpected result. If you disable Tx checksum offload or TSO em(4)
> > work as expected, right? 
> Not exactly.
> Only checksum offload gives this issue, TSO on/off doesn't matter.
> By the way, we checked UDP also and found something interested.
> For it, we run echo service on server ("|echo    dgram   udp     wait   
> root    internal|"  in //etc/inetd.conf/) and send strings by "|nc -4 -u
> server 7|" from client.
> So...UDP works fine before, while and after "ifconfig vlanX vlan X
> vlandev em0" command given at server. And the most interesting things is
> what while server "hangs" for TCP-connections, "establishing" UDP
> connections between client and server...helps it to "unhang" TCP, so it
> begins to works correctly.
> 
> > If so, would you try the following patch
> > and let me know whether it makes any difference?
> >
> > http://people.freebsd.org/~yongari/em.csum_tso.20091229.patch
> >   
> Doesn't help, same behavior. And moreover, UDP also doesn't "help" and
> doesn't work so.
> 

Sorry, that wasn't expected one. From your explanation above I
think I found a possible cause of checksum offload issue of em(4).
The issue was not I initially thought though. It seems the checksum
offload context configured in em(4) was incorrectly reused even if
a frame requires a new context as IP/TCP header length, checksum
offload offset was changed. Setting up new context put more burden
to hardware such that em(4) used to avoid new context setup as
possible as it can. However it seems em(4) failed to compare all
required field in the checksum offload context. Would you try the
following patch again?

http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch

> If you'd like, I could give you an access to this server via SSH, would you?

Thanks for the offer.
ATM I'm somewhat busy to address other driver issues. If the patch
does not work and I find spare time I'll let you know.
Comment 5 Dennis Yusupoff 2010-01-11 09:20:46 UTC
30.12.2009 23:20, Pyun YongHyeon пиÑеÑ:
>
> Sorry, that wasn't expected one. From your explanation above I
> think I found a possible cause of checksum offload issue of em(4).
> The issue was not I initially thought though. It seems the checksum
> offload context configured in em(4) was incorrectly reused even if
> a frame requires a new context as IP/TCP header length, checksum
> offload offset was changed. Setting up new context put more burden
> to hardware such that em(4) used to avoid new context setup as
> possible as it can. However it seems em(4) failed to compare all
> required field in the checksum offload context. Would you try the
> following patch again?
>
> http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch
Gotcha!
It works!

---
With best regards,
Dennis Yusupoff,
network engineer of
Smart-Telecom ISP
Russia, Saint-Petersburg
Comment 6 Jean-Luc.Richier 2010-02-24 08:19:29 UTC
A other cause (or the same) seems that when the interface is reset (for 
example with a ifconfig em0 down, the chip is reset, but the 
last_hw_offload in the interface context is not reset

I just had the following problem in FreeBSD8.0 RELEASE
# ping www.google.com
	It works
# ifconfig em0 down up
# ping www.google.com
	It does not work
# telnet -N <ipv4 router addr>
	Normal error message
# ping www.google.com
	It works

I made the following path to solve the problem (in em_stop):

--- /sys/dev/e1000/if_em.c.DIST 2009-10-25 02:10:29.000000000 +0100
+++ /sys/dev/e1000/if_em.c      2010-02-23 18:22:53.000000000 +0100
@@ -2710,6 +2710,7 @@
         ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);

         e1000_reset_hw(&adapter->hw);
+       adapter->last_hw_offload = 0;
         if (adapter->hw.mac.type >= e1000_82544)
                 E1000_WRITE_REG(&adapter->hw, E1000_WUC, 0);
  }




-- 
Jean-Luc RICHIER (Jean-Luc.Richier@Imag.Fr)
Laboratoire d'Informatique de Grenoble (LIG)
Campus de Grenoble
681 rue de la Passerelle
BP 72, F-38402 St Martin d'Hères Cedex
Tel : +33 4 76 82 72 32 Fax : +33 4 76 82 72 87
Comment 7 Jean-Luc.Richier 2010-02-24 09:35:01 UTC
Jean-Luc Richier wrote:

 > A other cause (or the same) seems that when the interface is reset 
(for example with a ifconfig em0 down, the chip is reset, but the
 > last_hw_offload in the interface context is not reset
....
 > I made the following path to solve the problem (in em_stop)
 > --- /sys/dev/e1000/if_em.c.DIST 2009-10-25 02:10:29.000000000 +0100
 > +++ /sys/dev/e1000/if_em.c 2010-02-23 18:22:53.000000000 +0100
...
 > + adapter->last_hw_offload = 0;
...

I tested the suggested patch
http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch
and it also solved my problem

Regards

-- 
Jean-Luc RICHIER (Jean-Luc.Richier@Imag.Fr)
Laboratoire d'Informatique de Grenoble (LIG)
Campus de Grenoble
681 rue de la Passerelle
BP 72, F-38402 St Martin d'Hères Cedex
Tel : +33 4 76 82 72 32 Fax : +33 4 76 82 72 87
Comment 8 mike 2010-08-23 16:54:47 UTC
This is still a problem in RELENG_8 as of Aug 28th.  The patch referenced in

<http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058305.html>http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058305.html 


fixes the issue for me

         ---Mike

--------------------------------------------------------------------
Mike Tancsa,                                      tel +1 519 651 3400
Sentex Communications,                            mike@sentex.net
Providing Internet since 1994                    www.sentex.net
Cambridge, Ontario Canada                         www.sentex.net/mike
Comment 9 Andre Oppermann freebsd_committer 2010-08-23 18:32:50 UTC
Responsible Changed
From-To: freebsd-net->jfv

Over to maintainer.
Comment 10 Dennis Yusupoff 2011-11-18 10:25:36 UTC
It's seems that you can close this PR since all works fine at least from
8.1-RELEASE.

-- 
With best regards,
Dennis Yusupoff,
network engineer of
Smart-Telecom ISP
Russia, Saint-Petersburg 

Comment 11 Sean Bruno freebsd_committer 2015-07-25 22:37:07 UTC
Closed at request of submitter.