Then txcsum is enabled on interface, there are connectivity problem with this host: packets sent to wrong MAC address. It was happening instantly after we assign vlan id and dev to interface vlanXXXX, which was had created before, see examples below. Example. Host 10.78.77.77 connected directly to server 10.78.77.88 by patch cord. 10.78.77.77 has mac 00:03:9d:73:e7:bd 10.78.77.88 has mac 00:30:48:63:db:e7 Establish TCP session by execute "telnet 10.78.77.88 22" from host tcpdump -lneti fxp0 tcp port 22 ---- 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.57182: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 66: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 106: 10.78.77.88.22 > 10.78.77.77.57182: tcp 40 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 66: 10.78.77.77.57182 > 10.78.77.88.22: tcp 0 --- IT's OK. Let's assign id and dev to interface vlan2 at server and repeat attempt to establish TCP session. Tcpdump from server: --- 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 62: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > 00:03:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 --- All looks fine (except unsuccessful attempts of establish session)...but let's look at it from host side: --- 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > f0:4e:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > e4:96:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > e4:96:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 78: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > d8:16:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > d8:16:9d:73:e7:bd, IPv4, length 74: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:03:9d:73:e7:bd > 00:30:48:63:db:e7, IPv4, length 62: 10.78.77.77.64722 > 10.78.77.88.22: tcp 0 00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 00:30:48:63:db:e7 > e8:cf:9d:73:e7:bd, IPv4, length 66: 10.78.77.88.22 > 10.78.77.77.64722: tcp 0 --- Did you see difference? While server says, that he sends TCP packets to proper MAC, _IN REAL_ it sends it to almost random MACs! Why "almost"? Because if you'll take a look at destination MACs closer, you'll see: --- d8:16:9d:73:e7:bd e4:96:9d:73:e7:bd e8:cf:9d:73:e7:bd f0:4e:9d:73:e7:bd --- Wow! First two octets are changing and, of course, are incorrect, while other four octets doesn't changed and are corrects. Of course, with that conditions no one could establish TCP session. Nota Bene: ICMP, however, works fine in both cases. UDP works unknown (doesn't check). Conclusion: With assigned vlans and enabled TXCSUM of [Intel] NIC, something in OS/drivers works incorrectly and broke TCP packets by setting incorrect MAC destination. Technical information: === #pciconf -lv hostb0@pci0:0:0:0: class=0x060000 card=0xd18015d9 chip=0x29f08086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '3200 Chipset (Bearlake) Processor to I/O Controller' class = bridge subclass = HOST-PCI pcib1@pci0:0:1:0: class=0x060400 card=0xd18015d9 chip=0x29f18086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = '3200 Chipset (Bearlake) PCIe Root Port 1' class = bridge subclass = PCI-PCI uhci0@pci0:0:26:0: class=0x0c0300 card=0xd18015d9 chip=0x29378086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci1@pci0:0:26:1: class=0x0c0300 card=0xd18015d9 chip=0x29388086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci2@pci0:0:26:2: class=0x0c0300 card=0xd18015d9 chip=0x29398086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci0@pci0:0:26:7: class=0x0c0320 card=0xd18015d9 chip=0x293c8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB pcib3@pci0:0:28:0: class=0x060400 card=0xd18015d9 chip=0x29408086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1' class = bridge subclass = PCI-PCI pcib4@pci0:0:28:4: class=0x060400 card=0xd18015d9 chip=0x29488086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 5' class = bridge subclass = PCI-PCI pcib5@pci0:0:28:5: class=0x060400 card=0xd18015d9 chip=0x294a8086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 6' class = bridge subclass = PCI-PCI uhci3@pci0:0:29:0: class=0x0c0300 card=0xd18015d9 chip=0x29348086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci4@pci0:0:29:1: class=0x0c0300 card=0xd18015d9 chip=0x29358086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci5@pci0:0:29:2: class=0x0c0300 card=0xd18015d9 chip=0x29368086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci1@pci0:0:29:7: class=0x0c0320 card=0xd18015d9 chip=0x293a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB pcib6@pci0:0:30:0: class=0x060401 card=0xd18015d9 chip=0x244e8086 rev=0x92 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0xd18015d9 chip=0x29168086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IR (ICH9R) LPC Interface Controller' class = bridge subclass = PCI-ISA atapci1@pci0:0:31:2: class=0x010601 card=0xd18015d9 chip=0x29228086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controller' class = mass storage subclass = SATA none0@pci0:0:31:3: class=0x0c0500 card=0xd18015d9 chip=0x29308086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) SMBus Controller' class = serial bus subclass = SMBus none1@pci0:0:31:6: class=0x118000 card=0x000015d9 chip=0x29328086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) Thermal Subsystem' class = dasp pcib2@pci0:1:0:0: class=0x060400 card=0x00000000 chip=0x032c8086 rev=0x09 hdr=0x01 vendor = 'Intel Corporation' device = 'PCI Express-to-PCI Express Bridge (6702PXH)' class = bridge subclass = PCI-PCI ioapic0@pci0:1:0:1: class=0x080020 card=0xd18015d9 chip=0x03268086 rev=0x09 hdr=0x00 vendor = 'Intel Corporation' device = '6700/6702PXH I/OxAPIC Interrupt Controller A' class = base peripheral subclass = interrupt controller em0@pci0:13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel Corporation 82573E Gigabit Ethernet Controller (Copper) (82573E)' class = network subclass = ethernet em1@pci0:15:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet vgapci0@pci0:17:3:0: class=0x030000 card=0xd18015d9 chip=0x515e1002 rev=0x02 hdr=0x00 vendor = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.' device = 'Radeon ES1000 (Radeon ES1000)' class = display subclass = VGA atapci0@pci0:17:4:0: class=0x010185 card=0x82131283 chip=0x82131283 rev=0x00 hdr=0x00 vendor = 'Integrated Technology Express (ITE) Inc' device = 'IDE Controller (IT8213F)' class = mass storage subclass = ATA [root@alpha /]# ps -axl UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -68 0 0 64 - DLs ?? 0:00,09 [kernel] 0 1 0 0 44 0 2912 508 wait ILs ?? 0:00,00 /sbin/init -- 0 2 0 0 -8 0 0 8 - DL ?? 0:00,09 [g_event] 0 3 0 0 -8 0 0 8 - DL ?? 0:00,10 [g_up] 0 4 0 0 -8 0 0 8 - DL ?? 0:00,09 [g_down] 0 5 0 0 -16 0 0 8 ccb_sc DL ?? 0:00,00 [xpt_thrd] 0 6 0 0 -16 0 0 8 waitin DL ?? 0:00,00 [sctp_iterator] 0 7 0 0 -16 0 0 8 psleep DL ?? 0:00,00 [pagedaemon] 0 8 0 0 -16 0 0 8 psleep DL ?? 0:00,00 [vmdaemon] 0 9 0 0 76 0 0 8 pgzero DL ?? 0:00,00 [pagezero] 0 10 0 0 -16 0 0 8 audit_ DL ?? 0:00,00 [audit] 0 11 0 0 171 0 0 16 - RL ?? 132:01,13 [idle] 0 12 0 0 -64 0 0 144 - WL ?? 0:02,56 [intr] 0 13 0 0 -16 0 0 8 - DL ?? 0:00,09 [yarrow] 0 14 0 0 -64 0 0 256 - DL ?? 0:00,09 [usb] 0 15 0 0 -16 0 0 8 psleep DL ?? 0:00,01 [bufdaemon] 0 16 0 0 -16 0 0 8 syncer DL ?? 0:00,04 [syncer] 0 17 0 0 -16 0 0 8 vlruwt DL ?? 0:00,01 [vnlru] 0 18 0 0 -16 0 0 8 sdflus DL ?? 0:00,02 [softdepflush] 0 19 0 0 -16 0 0 8 flowcl DL ?? 0:00,00 [flowcleaner] 0 409 1 0 50 0 1888 540 select Is ?? 0:00,01 /sbin/devd 101 487 1 0 44 0 4004 1968 select Is ?? 0:00,00 /usr/local/sbin/zebra -d 101 493 1 0 44 0 4140 2336 select Ss ?? 0:00,05 /usr/local/sbin/ospfd -d 0 539 1 0 44 0 3344 1328 select Ss ?? 0:00,02 /usr/sbin/syslogd -s 0 765 1 0 44 0 6676 3768 select Is ?? 0:00,01 /usr/sbin/sshd 0 774 1 0 44 0 3372 1376 nanslp Ss ?? 0:00,01 /usr/sbin/cron -s 0 801 1 0 76 0 3404 1272 select Is ?? 0:00,00 /usr/sbin/inetd -wW -C 60 0 1224 765 0 45 0 9400 4404 sbwait Is ?? 0:00,02 sshd: dyr [priv] (sshd) 1001 1227 1224 0 44 0 9400 4440 select S ?? 0:00,09 sshd: dyr@pts/0 (sshd) 0 826 1 0 44 0 3808 1816 wait Is v0 0:00,01 login [pam] (login) 1001 850 826 0 46 0 5584 2520 pause I v0 0:00,01 -tcsh (tcsh) 0 853 850 0 44 0 4468 2424 ttyin I+ v0 0:00,02 bash 0 827 1 0 76 0 3344 1180 ttyin Is+ v1 0:00,00 /usr/libexec/getty Pc ttyv1 0 828 1 0 76 0 3344 1180 ttyin Is+ v2 0:00,00 /usr/libexec/getty Pc ttyv2 0 829 1 0 76 0 3344 1180 ttyin Is+ v3 0:00,00 /usr/libexec/getty Pc ttyv3 0 830 1 0 76 0 3344 1180 ttyin Is+ v4 0:00,00 /usr/libexec/getty Pc ttyv4 0 831 1 0 76 0 3344 1180 ttyin Is+ v5 0:00,00 /usr/libexec/getty Pc ttyv5 0 832 1 0 76 0 3344 1180 ttyin Is+ v6 0:00,00 /usr/libexec/getty Pc ttyv6 0 833 1 0 76 0 3344 1180 ttyin Is+ v7 0:00,00 /usr/libexec/getty Pc ttyv7 1001 1228 1227 0 46 0 5584 2588 pause Is 0 0:00,01 -tcsh (tcsh) 0 1231 1228 0 44 0 4468 2464 wait S 0 0:00,06 bash 0 1441 1231 0 44 0 3424 1252 - R+ 0 0:00,00 ps -axl [root@alpha /]# vmstat -s 1831477 cpu context switches 10200 device interrupts 623396 software interrupts 115495 traps 389816 system calls 19 kernel threads created 1380 fork() calls 43 vfork() calls 0 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 303 vnode pager pageins 1987 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 57 pages reactivated 41718 copy-on-write faults 131 copy-on-write optimized faults 40215 zero fill pages zeroed 1783 zero fill pages prezeroed 14 intransit blocking page faults 112421 total VM faults taken 0 pages affected by kernel thread creation 240965 pages affected by fork() 8596 pages affected by vfork() 0 pages affected by rfork() 64 pages cached 96163 pages freed 0 pages freed by daemon 68636 pages freed by exiting processes 2072 pages active 1754 pages inactive 6 pages in VM cache 6774 pages wired down 500685 pages free 4096 bytes per page 48089 total name lookups cache hits (86% pos + 9% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% [root@alpha /]# vmstat -m Type InUse MemUse HighUse Requests Size(s) GEOM 71 16K - 510 16,32,64,128,512,1024 CAM dev queue 1 1K - 1 64 isadev 8 1K - 8 64 CAM queue 3 1K - 7 16 pci_link 16 2K - 16 64,128 cdev 9 2K - 9 128 acpi_perf 2 1K - 2 128 sigio 1 1K - 1 32 filedesc 48 17K - 1746 32,256,2048 kenv 79 7K - 83 16,32,64,128,4096 kqueue 0 0K - 8 128,1024 proc-args 23 2K - 762 16,32,64,128 ithread 104 10K - 104 16,64,128 KTRACE 100 13K - 100 128 linker 115 32K - 153 16,32,256,1024,4096 lockf 20 2K - 20 32,64 ip6ndp 4 1K - 6 64,128 temp 25 765K - 8754 16,32,64,128,256,512,1024,2048,4096 devbuf 4424 3492K - 4495 16,32,64,128,256,512,1024,2048,4096 kbdmux 7 10K - 7 16,256,1024,2048,4096 module 464 30K - 464 64,128 mtx_pool 2 8K - 2 4096 subproc 127 190K - 1529 256,4096 proc 2 8K - 2 4096 session 18 2K - 55 64 pgrp 22 2K - 362 64 cred 50 5K - 9476 64,128 uidinfo 4 2K - 58 64,1024 plimit 14 4K - 512 256 sysctltmp 0 0K - 388 16,32,64,128 sysctloid 3389 104K - 3485 16,32,64,128 sysctl 0 0K - 1063 16,32,64 callout 1 256K - 1 umtx 161 11K - 161 64 p1003.1b 1 1K - 1 16 SWAP 2 277K - 2 64 bus-sc 72 225K - 4275 16,32,64,128,256,512,1024,2048,4096 bus 1178 55K - 7309 16,32,64,128,512,1024 devstat 6 13K - 6 16,4096 eventhandler 73 4K - 73 32,64,128 kobj 325 650K - 378 2048 Per-cpu 1 1K - 1 16 rman 190 12K - 693 16,32,64 sbuf 0 0K - 340 16,32,64,128,256,512,1024,2048,4096 acpica 2271 119K - 80413 16,32,64,128,256,512,1024,2048 acpitask 1 1K - 1 1024 CAM SIM 1 1K - 1 128 stack 0 0K - 2 128 taskqueue 15 1K - 15 16,64 Unitno 11 1K - 35 16,64 iov 0 0K - 1882 16,64,128,256 select 28 2K - 28 64 ioctlops 0 0K - 5025 16,32,64,128,256,512,1024,4096 msg 4 25K - 4 1024,4096 sem 4 6K - 4 256,512,1024,4096 shm 1 12K - 1 tty 22 11K - 25 512,2048 pts 1 1K - 2 128 mbuf_tag 0 0K - 488 32,64 ksem 1 4K - 1 4096 shmfd 1 4K - 1 4096 pcb 18 79K - 34 16,32,64,512,1024,2048,4096 soname 7 1K - 2021 16,32,128 biobuf 4 8K - 6 2048 vfscache 1 512K - 1 vfs_hash 1 256K - 1 vnodes 2 1K - 2 128 vnodemarker 0 0K - 1298 512 mount 86 3K - 130 16,32,64,128,256 BPF 3 1K - 41 16,64,256 ether_multi 40 2K - 46 16,32,64 ifaddr 43 9K - 55 16,32,64,128,256,512,2048 ifnet 4 4K - 6 64,1024 clone 6 24K - 6 4096 arpcom 2 1K - 4 16 lltable 12 3K - 18 128,256 acpisem 19 3K - 19 64,128 CAM XPT 11 2K - 30 16,32,64,1024 CAM periph 2 1K - 11 16,32,64,128 ata_generic 1 1K - 1 1024 routetbl 59 260K - 330 16,32,64,128,256,512 igmp 3 1K - 5 128 ad_driver 1 1K - 1 32 ar_driver 0 0K - 7 512,2048 ip_moptions 2 1K - 2 32,128 in_multi 7 1K - 7 128 in_mfilter 1 1K - 1 512 sctp_iter 0 0K - 4 128 sctp_ifn 3 1K - 3 128 sctp_ifa 5 1K - 5 128 sctp_vrf 1 1K - 1 64 sctp_a_it 0 0K - 4 16 hostcache 1 16K - 1 syncache 1 72K - 1 ip6_moptions 2 1K - 2 32,128 in6_multi 12 2K - 12 16,256 in6_mfilter 1 1K - 1 512 entropy 1024 64K - 1024 64 mld 3 1K - 5 128 NFS FHA 1 1K - 1 1024 rpc 2 5K - 2 128,4096 audit_evclass 172 3K - 211 16 savedino 0 0K - 33 256 dirrem 0 0K - 82 32 mkdir 0 0K - 12 32 diradd 0 0K - 89 64 freefile 0 0K - 31 32 freeblks 0 0K - 26 256 freefrag 0 0K - 1 32 allocdirect 1 1K - 43 128 bmsafemap 1 1K - 28 64 newblk 1 1K - 44 64,256 inodedep 2 257K - 125 128 pagedep 1 64K - 23 64 ufs_dirhash 33 7K - 33 16,32,64,128,512 ufs_mount 15 52K - 15 256,2048,4096 vm_pgdata 2 65K - 2 64 atkbddev 2 1K - 2 32 acpidev 81 3K - 81 32 UART 6 3K - 6 16,256,1024 apmdev 1 1K - 1 64 USBdev 41 11K - 41 32,128,256,1024 USB 66 79K - 67 16,32,64,128,1024,4096 DEVFS1 107 27K - 118 256 DEVFS3 123 16K - 135 128 io_apic 2 2K - 2 1024 memdesc 1 4K - 1 4096 msi 2 1K - 2 64 nexusdev 4 1K - 4 16 DEVFS 19 1K - 20 16,64 DEVFSP 0 0K - 8 32 pfs_nodes 20 3K - 20 128 vlan 0 0K - 6 64 [root@alpha /]# vmstat -z ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 128, 0, 88, 2, 88, 0 UMA Zones: 888, 0, 88, 0, 88, 0 UMA Slabs: 284, 0, 511, 7, 1262, 0 UMA RCntSlabs: 544, 0, 475, 1, 475, 0 UMA Hash: 128, 0, 4, 26, 4, 0 16 Bucket: 76, 0, 53, 47, 74, 0 32 Bucket: 140, 0, 49, 7, 70, 0 64 Bucket: 268, 0, 48, 8, 94, 26 128 Bucket: 524, 0, 63, 0, 1064, 112 VM OBJECT: 136, 0, 915, 216, 21819, 0 MAP: 140, 0, 7, 21, 7, 0 KMAP ENTRY: 72, 57505, 39, 226, 4277, 0 MAP ENTRY: 72, 0, 507, 394, 40821, 0 DP fakepg: 72, 0, 0, 0, 0, 0 SG fakepg: 72, 0, 0, 0, 0, 0 mt_zone: 2056, 0, 263, 127, 263, 0 16: 16, 0, 2900, 348, 48137, 0 32: 32, 0, 2790, 374, 42407, 0 64: 64, 0, 4574, 146, 14228, 0 128: 128, 0, 3329, 151, 16754, 0 256: 256, 0, 874, 116, 4453, 0 512: 512, 0, 65, 23, 4054, 0 1024: 1024, 0, 45, 167, 4456, 0 2048: 2048, 0, 361, 55, 719, 0 4096: 4096, 0, 90, 39, 5304, 0 Files: 56, 0, 66, 202, 8773, 0 TURNSTILE: 72, 0, 162, 18, 162, 0 umtx pi: 52, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0 PROC: 680, 0, 41, 43, 1444, 0 THREAD: 572, 0, 141, 20, 141, 0 SLEEPQUEUE: 32, 0, 162, 133, 162, 0 VMSPACE: 232, 0, 23, 79, 1427, 0 cpuset: 40, 0, 2, 182, 2, 0 audit_record: 816, 0, 0, 0, 0, 0 mbuf_packet: 256, 0, 512, 384, 10689, 0 mbuf: 256, 0, 1, 393, 10792, 0 mbuf_cluster: 2048, 25600, 896, 26, 896, 0 mbuf_jumbo_page: 4096, 12800, 0, 14, 27, 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0 g_bio: 140, 0, 0, 224, 9073, 0 ttyinq: 152, 0, 135, 47, 285, 0 ttyoutq: 256, 0, 72, 18, 152, 0 ata_request: 200, 0, 0, 114, 2277, 0 ata_composite: 180, 0, 0, 0, 0, 0 VNODE: 268, 0, 532, 14, 565, 0 VNODEPOLL: 60, 0, 0, 0, 0, 0 S VFS Cache: 72, 0, 512, 124, 2017, 0 L VFS Cache: 292, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 36, 19250, 0 DIRHASH: 1024, 0, 42, 14, 42, 0 NFSMOUNT: 520, 0, 0, 0, 0, 0 NFSNODE: 464, 0, 0, 0, 0, 0 pipe: 392, 0, 2, 48, 835, 0 ksiginfo: 80, 0, 75, 981, 75, 0 itimer: 220, 0, 0, 0, 0, 0 KNOTE: 68, 0, 0, 112, 32, 0 socket: 412, 25605, 24, 39, 616, 0 unpcb: 172, 25622, 12, 80, 190, 0 ipq: 32, 904, 0, 0, 0, 0 udp_inpcb: 220, 25614, 2, 52, 406, 0 udpcb: 8, 25781, 2, 404, 406, 0 tcp_inpcb: 220, 25614, 7, 47, 17, 0 tcpcb: 632, 25602, 7, 17, 17, 0 tcptw: 52, 5184, 0, 144, 4, 0 syncache: 112, 15365, 0, 105, 19, 0 hostcache: 76, 15400, 2, 148, 2, 0 tcpreass: 20, 1690, 0, 169, 6, 0 sackhole: 20, 0, 0, 169, 1, 0 sctp_ep: 848, 25600, 0, 0, 0, 0 sctp_asoc: 1460, 40000, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 290, 4, 0 sctp_raddr: 420, 80001, 0, 0, 0, 0 sctp_chunk: 96, 400000, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0 sctp_stream_msg_out: 64, 400020, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0 ripcb: 220, 25614, 2, 34, 2, 0 rtentry: 108, 0, 12, 96, 22, 0 selfd: 28, 0, 60, 448, 83248, 0 ip4flow: 40, 4140, 3, 273, 75, 0 ip6flow: 64, 4118, 0, 0, 0, 0 SWAPMETA: 276, 121576, 0, 0, 0, 0 Mountpoints: 644, 0, 6, 12, 6, 0 FFS inode: 116, 0, 496, 32, 527, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 496, 44, 527, 0 [root@alpha /]# vmstat -i interrupt total rate irq1: atkbd0 1 0 irq16: uhci0 274 0 irq17: uhci1+ 2276 0 irq18: uhci2 ehci+ 2 0 cpu0: timer 7637268 1921 irq256: em0 6672 1 irq257: em1 999 0 cpu1: timer 7637143 1921 Total 15284635 3845 [root@alpha /]# pstat -T 66/12328 files 0M/2048M swap space [root@alpha /]# pstat -s Device 512-blocks Used Avail Capacity /dev/ad6s1b 4194304 0 4194304 0% [root@alpha /]# iostat tty ad6 cpu tin tout KB/t tps MB/s us ni sy in id 1 64 14.15 1 0.01 0 0 0 0 100 [root@alpha /]# ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME [root@alpha /]# ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 33554432 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 8192 (max amount of shared memory in pages) seminfo: semmap: 30 (# of entries in semaphore map) semmni: 10 (# of semaphore identifiers) semmns: 60 (# of semaphores in system) semmnu: 30 (# of undo structures in system) semmsl: 60 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 10 (max # of undo entries per process) semusz: 136 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) [root@alpha /]# w -dn 14:45 up 1:06, 2 users, load averages: 0,00 0,00 0,00 USER TTY FROM LOGIN@ IDLE WHAT 826 login [pam] (login) 850 -tcsh (tcsh) 853 bash dyr v0 - 13:39 26 bash 1228 -tcsh (tcsh) 1231 bash 1451 w -dn dyr pts/0 217.119.16.88 14:18 - w -dn [root@alpha /]# nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 0 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 [root@alpha /]# netstat -s tcp: 3259 packets sent 3130 data packets (390516 bytes) 179 data packets (85318 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 87 ack-only packets (43 delayed) 0 URG only packets 0 window probe packets 1 window update packet 10 control packets 4559 packets received 3105 acks (for 391796 bytes) 61 duplicate acks 0 acks for unsent data 1373 packets (77061 bytes) received in-sequence 23 completely duplicate packets (5285 bytes) 0 old duplicate packets 3 packets with some dup. data (41 bytes duped) 6 out-of-order packets (336 bytes) 0 packets (0 bytes) of data after window 0 window probes 5 window update packets 1 packet received after close 2 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 0 connection requests 11 connection accepts 0 bad connection attempts 0 listen queue overflows 1 ignored RSTs in the window 11 connections established (including accepts) 10 connections closed (including 1 drop) 4 connections updated cached RTT on close 5 connections updated cached RTT variance on close 1 connection updated cached ssthresh on close 0 embryonic connections dropped 3105 segments updated rtt (of 1714 attempts) 41 retransmit timeouts 1 connection dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 0 correct ACK header predictions 1082 correct data packet header predictions 19 syncache entries added 43 retransmitted 26 dupsyn 0 dropped 11 completed 0 bucket overflow 0 cache overflow 3 reset 5 stale 0 aborted 0 badack 0 unreach 0 zone failures 19 cookies sent 0 cookies received 2 SACK recovery episodes 1 segment rexmit in SACK recovery episodes 480 byte rexmits in SACK recovery episodes 34 SACK options (SACK blocks) received 4 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 24 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 16 dropped due to no socket 8 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 16 datagrams output 0 times multicast source filter matched sctp: 0 input packets 0 datagrams 0 packets that had data 0 input SACK chunks 0 input DATA chunks 0 duplicate DATA chunks 0 input HB chunks 0 HB-ACK chunks 0 input ECNE chunks 0 input AUTH chunks 0 chunks missing AUTH 0 invalid HMAC ids received 0 invalid secret ids received 0 auth failed 0 fast path receives all one chunk 0 fast path multi-part data 0 output packets 0 output SACKs 0 output DATA chunks 0 retransmitted DATA chunks 0 fast retransmitted DATA chunks 0 FR's that happened more than once to same chunk 0 intput HB chunks 0 output ECNE chunks 0 output AUTH chunks 0 ip_output error counter Packet drop statistics: 0 from middle box 0 from end host 0 with data 0 non-data, non-endhost 0 non-endhost, bandwidth rep only 0 not enough for chunk header 0 not enough data to confirm 0 where process_chunk_drop said break 0 failed to find TSN 0 attempt reverse TSN lookup 0 e-host confirms zero-rwnd 0 midbox confirms no space 0 data did not match TSN 0 TSN's marked for Fast Retran Timeouts: 0 iterator timers fired 0 T3 data time outs 0 window probe (T3) timers fired 0 INIT timers fired 0 sack timers fired 0 shutdown timers fired 0 heartbeat timers fired 0 a cookie timeout fired 0 an endpoint changed its cookiesecret 0 PMTU timers fired 0 shutdown ack timers fired 0 shutdown guard timers fired 0 stream reset timers fired 0 early FR timers fired 0 an asconf timer fired 0 auto close timer fired 0 asoc free timers expired 0 inp free timers expired 0 packet shorter than header 0 checksum error 0 no endpoint for port 0 bad v-tag 0 bad SID 0 no memory 0 number of multiple FR in a RTT window 0 RFC813 allowed sending 0 RFC813 does not allow sending 0 times max burst prohibited sending 0 look ahead tells us no memory in interface 0 numbers of window probes sent 0 times an output error to clamp down on next user send 0 times sctp_senderrors were caused from a user 0 number of in data drops due to chunk limit reached 0 number of in data drops due to rwnd limit reached 0 times a ECN reduced the cwnd 0 used express lookup via vtag 0 collision in express lookup 0 times the sender ran dry of user data on primary 0 same for above 0 sacks the slow way 0 window update only sacks sent 0 sends with sinfo_flags !=0 0 unordered sends 0 sends with EOF flag set 0 sends with ABORT flag set 0 times protocol drain called 0 times we did a protocol drain 0 times recv was called with peek 0 cached chunks used 0 cached stream oq's used 0 unread messages abandonded by close 0 send burst avoidance, already max burst inflight to net 0 send cwnd full avoidance, already max burst inflight to net 0 number of map array over-runs via fwd-tsn's ip: 5333 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 5315 packets for this host 8 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 10 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 3597 packets sent from this host 872 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 191 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 8 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: echo reply: 87 destination unreachable: 8 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 8 echo: 87 87 message responses generated 0 invalid return addresses 0 no return routes ICMP address mask responses are disabled igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 8 membership reports sent ip6: 16 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 8 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 16 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Input histogram: UDP: 8 ICMP6: 8 Mbuf statistics: 8 one mbuf two or more mbuf: lo0= 8 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not continuous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: 8 first candidate 8 same address icmp6: 8 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation Output histogram: unreach: 8 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Input histogram: unreach: 8 Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 8 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output [root@alpha /]# netstat -m 513/777/1290 mbufs in use (current/cache/total) 512/410/922/25600 mbuf clusters in use (current/cache/total/max) 512/384 mbuf+clusters out of packet secondary zone in use (current/cache) 0/14/14/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1152K/1070K/2222K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/4/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines [root@alpha /]# netstat -id Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll Drop em0 1500 <Link#1> 00:30:48:63:db:e6 5943 0 3704 0 0 0 em0 1500 109.71.177.64 109.71.177.88 4509 - 3636 - - - em1 1500 <Link#2> 00:30:48:63:db:e7 400 0 674 0 0 0 em1 1500 10.78.77.0 10.78.77.88 247 - 669 - - - lo0 16384 <Link#3> 48 0 48 0 0 0 lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - - lo0 16384 localhost ::1 0 - 24 - - - lo0 16384 your-net localhost 24 - 24 - - - [root@alpha /]# netstat -anr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 109.71.177.65 UG1 3 2578 em0 10.78.77.0/24 link#2 U 0 249 em1 10.78.77.88 link#2 UHS 0 0 lo0 109.71.177.64/27 link#1 U 0 0 em0 109.71.177.88 link#1 UHS 0 0 lo0 127.0.0.1 link#3 UH 0 24 lo0 217.119.16.252/30 109.71.177.65 UG1 0 0 em0 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UH lo0 fe80::%lo0/64 link#3 U lo0 fe80::1%lo0 link#3 UHS lo0 ff01:3::/32 fe80::1%lo0 U lo0 ff02::%lo0/32 fe80::1%lo0 U lo0 [root@alpha /]# netstat -anA Active Internet connections (including servers) Tcpcb Proto Recv-Q Send-Q Local Address Foreign Address (state) c5e71278 tcp4 0 0 109.71.177.88.22 217.119.16.88.5285 ESTABLISHED c5c8f278 tcp4 0 0 *.22 *.* LISTEN c5c8f4f0 tcp6 0 0 *.22 *.* LISTEN c5c8f768 tcp4 0 0 *.2604 *.* LISTEN c5c8f9e0 tcp6 0 0 *.2604 *.* LISTEN c5c8fc58 tcp4 0 0 *.2601 *.* LISTEN c5c90000 tcp6 0 0 *.2601 *.* LISTEN c5b09528 udp4 0 0 *.514 *.* c5b09604 udp6 0 0 *.514 *.* c5c8c000 icm6 0 0 *.* *.* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr c5b0cd70 stream 0 0 0 c5b0d560 0 0 c5b0d560 stream 0 0 0 c5b0cd70 0 0 c5b0d764 stream 0 0 0 c5b0d810 0 0 /var/run/quagga/zserv.api c5b0d810 stream 0 0 0 c5b0d764 0 0 c5b0d8bc stream 0 0 c5c88648 0 0 0 /var/run/quagga/ospfd.vty c5b0dac0 stream 0 0 c5c8896c 0 0 0 /var/run/quagga/zebra.vty c5b0db6c stream 0 0 c5c88a78 0 0 0 /var/run/quagga/zserv.api c5b0cec8 stream 0 0 c5c7c860 0 0 0 /var/run/devd.pipe c5b0c158 dgram 0 0 0 c5b0d6b8 0 0 c5b0cb6c dgram 0 0 0 c5b0d60c 0 0 c5b0d60c dgram 0 0 c5c8810c 0 c5b0cb6c 0 /var/run/logpriv c5b0d6b8 dgram 0 0 c5c88218 0 c5b0c158 0 /var/run/log [root@alpha /]# netstat -aL Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen Local Address tcp4 0/0/128 *.ssh tcp6 0/0/128 *.ssh tcp4 0/0/3 *.ospfd tcp6 0/0/3 *.ospfd tcp4 0/0/3 *.zebra tcp6 0/0/3 *.zebra unix 0/0/5 /var/run/quagga/ospfd.vty unix 0/0/5 /var/run/quagga/zebra.vty unix 0/0/5 /var/run/quagga/zserv.api unix 0/0/4 /var/run/devd.pipe [root@alpha /]# fstat USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root fstat 1459 root / 2 drwxr-xr-x 512 r root fstat 1459 wd / 2 drwxr-xr-x 512 r root fstat 1459 text /usr 1789987 -r-xr-sr-x 17672 r root fstat 1459 0 /dev 104 crw--w---- pts/0 rw root fstat 1459 1 /dev 104 crw--w---- pts/0 rw root fstat 1459 2 /dev 104 crw--w---- pts/0 rw root fstat 1459 3 /dev 8 crw-r----- mem r root fstat 1459 4 /dev 9 crw-r----- kmem r root fstat 1459 5 / 118128 -rw------- 40960 r root bash 1231 root / 2 drwxr-xr-x 512 r root bash 1231 wd / 2 drwxr-xr-x 512 r root bash 1231 text /usr 1039162 -rwxr-xr-x 620152 r root bash 1231 0 /dev 104 crw--w---- pts/0 rw root bash 1231 1 /dev 104 crw--w---- pts/0 rw root bash 1231 2 /dev 104 crw--w---- pts/0 rw root bash 1231 255 /dev 104 crw--w---- pts/0 rw dyr tcsh 1228 root / 2 drwxr-xr-x 512 r dyr tcsh 1228 wd /usr 2284545 drwxr-xr-x 512 r dyr tcsh 1228 text / 235560 -r-xr-xr-x 327760 r dyr tcsh 1228 15 /dev 104 crw--w---- pts/0 rw dyr tcsh 1228 16 /dev 104 crw--w---- pts/0 rw dyr tcsh 1228 17 /dev 104 crw--w---- pts/0 rw dyr tcsh 1228 18 /dev 104 crw--w---- pts/0 rw dyr tcsh 1228 19 /dev 104 crw--w---- pts/0 rw dyr sshd 1227 root / 2 drwxr-xr-x 512 r dyr sshd 1227 wd / 2 drwxr-xr-x 512 r dyr sshd 1227 text /usr 2331651 -r-xr-xr-x 222192 r dyr sshd 1227 0 /dev 11 crw-rw-rw- null rw dyr sshd 1227 1 /dev 11 crw-rw-rw- null rw dyr sshd 1227 2 /dev 11 crw-rw-rw- null rw dyr sshd 1227 3* internet stream tcp c5e71278 dyr sshd 1227 4* local stream c5b0d560 <-> c5b0cd70 dyr sshd 1227 5* pipe c5a3c7a8 <-> c5a3c860 0 rw dyr sshd 1227 6* pipe c5a3c860 <-> c5a3c7a8 0 rw dyr sshd 1227 7* pseudo-terminal master pts/0 rw dyr sshd 1227 9* pseudo-terminal master pts/0 rw dyr sshd 1227 10* pseudo-terminal master pts/0 rw root sshd 1224 root / 2 drwxr-xr-x 512 r root sshd 1224 wd / 2 drwxr-xr-x 512 r root sshd 1224 text /usr 2331651 -r-xr-xr-x 222192 r root sshd 1224 0 /dev 11 crw-rw-rw- null r root sshd 1224 1 /dev 11 crw-rw-rw- null rw root sshd 1224 2 /dev 11 crw-rw-rw- null rw root sshd 1224 3* internet stream tcp c5e71278 root sshd 1224 4* pseudo-terminal master pts/0 rw root sshd 1224 5* local stream c5b0cd70 <-> c5b0d560 root bash 853 root / 2 drwxr-xr-x 512 r root bash 853 wd /usr 2284545 drwxr-xr-x 512 r root bash 853 text /usr 1039162 -rwxr-xr-x 620152 r root bash 853 0 /dev 49 crw------- ttyv0 rw root bash 853 1 /dev 49 crw------- ttyv0 rw root bash 853 2 /dev 49 crw------- ttyv0 rw root bash 853 255 /dev 49 crw------- ttyv0 rw dyr tcsh 850 root / 2 drwxr-xr-x 512 r dyr tcsh 850 wd /usr 2284545 drwxr-xr-x 512 r dyr tcsh 850 text / 235560 -r-xr-xr-x 327760 r dyr tcsh 850 15 /dev 49 crw------- ttyv0 rw dyr tcsh 850 16 /dev 49 crw------- ttyv0 rw dyr tcsh 850 17 /dev 49 crw------- ttyv0 rw dyr tcsh 850 18 /dev 49 crw------- ttyv0 rw dyr tcsh 850 19 /dev 49 crw------- ttyv0 rw root getty 833 root / 2 drwxr-xr-x 512 r root getty 833 wd / 2 drwxr-xr-x 512 r root getty 833 text /usr 172036 -r-xr-xr-x 21860 r root getty 833 0 /dev 56 crw------- ttyv7 rw root getty 833 1 /dev 56 crw------- ttyv7 rw root getty 833 2 /dev 56 crw------- ttyv7 rw root getty 832 root / 2 drwxr-xr-x 512 r root getty 832 wd / 2 drwxr-xr-x 512 r root getty 832 text /usr 172036 -r-xr-xr-x 21860 r root getty 832 0 /dev 55 crw------- ttyv6 rw root getty 832 1 /dev 55 crw------- ttyv6 rw root getty 832 2 /dev 55 crw------- ttyv6 rw root getty 831 root / 2 drwxr-xr-x 512 r root getty 831 wd / 2 drwxr-xr-x 512 r root getty 831 text /usr 172036 -r-xr-xr-x 21860 r root getty 831 0 /dev 54 crw------- ttyv5 rw root getty 831 1 /dev 54 crw------- ttyv5 rw root getty 831 2 /dev 54 crw------- ttyv5 rw root getty 830 root / 2 drwxr-xr-x 512 r root getty 830 wd / 2 drwxr-xr-x 512 r root getty 830 text /usr 172036 -r-xr-xr-x 21860 r root getty 830 0 /dev 53 crw------- ttyv4 rw root getty 830 1 /dev 53 crw------- ttyv4 rw root getty 830 2 /dev 53 crw------- ttyv4 rw root getty 829 root / 2 drwxr-xr-x 512 r root getty 829 wd / 2 drwxr-xr-x 512 r root getty 829 text /usr 172036 -r-xr-xr-x 21860 r root getty 829 0 /dev 52 crw------- ttyv3 rw root getty 829 1 /dev 52 crw------- ttyv3 rw root getty 829 2 /dev 52 crw------- ttyv3 rw root getty 828 root / 2 drwxr-xr-x 512 r root getty 828 wd / 2 drwxr-xr-x 512 r root getty 828 text /usr 172036 -r-xr-xr-x 21860 r root getty 828 0 /dev 51 crw------- ttyv2 rw root getty 828 1 /dev 51 crw------- ttyv2 rw root getty 828 2 /dev 51 crw------- ttyv2 rw root getty 827 root / 2 drwxr-xr-x 512 r root getty 827 wd / 2 drwxr-xr-x 512 r root getty 827 text /usr 172036 -r-xr-xr-x 21860 r root getty 827 0 /dev 50 crw------- ttyv1 rw root getty 827 1 /dev 50 crw------- ttyv1 rw root getty 827 2 /dev 50 crw------- ttyv1 rw root login 826 root / 2 drwxr-xr-x 512 r root login 826 wd /usr 2284545 drwxr-xr-x 512 r root login 826 text /usr 1790410 -r-sr-xr-x 21804 r root login 826 0 /dev 49 crw------- ttyv0 rw root login 826 1 /dev 49 crw------- ttyv0 rw root login 826 2 /dev 49 crw------- ttyv0 rw root login 826 3* local dgram c5b0cb6c <-> c5b0d60c root inetd 801 root / 2 drwxr-xr-x 512 r root inetd 801 wd / 2 drwxr-xr-x 512 r root inetd 801 text /usr 2332017 -r-xr-xr-x 42188 r root inetd 801 0 /dev 11 crw-rw-rw- null rw root inetd 801 1 /dev 11 crw-rw-rw- null rw root inetd 801 2 /dev 11 crw-rw-rw- null rw root inetd 801 3 /var 212003 -rw------- 3 w root inetd 801 4* pipe c5a3c620 <-> c5a3c6d8 0 rw root inetd 801 5* pipe c5a3c6d8 <-> c5a3c620 0 rw root cron 774 root / 2 drwxr-xr-x 512 r root cron 774 wd /var 635904 drwxr-x--- 512 r root cron 774 text /usr 2331971 -r-xr-xr-x 34080 r root cron 774 0 /dev 11 crw-rw-rw- null rw root cron 774 1 /dev 11 crw-rw-rw- null rw root cron 774 2 /dev 11 crw-rw-rw- null rw root cron 774 3 /var 212002 -rw------- 3 w root sshd 765 root / 2 drwxr-xr-x 512 r root sshd 765 wd / 2 drwxr-xr-x 512 r root sshd 765 text /usr 2331651 -r-xr-xr-x 222192 r root sshd 765 0 /dev 11 crw-rw-rw- null rw root sshd 765 1 /dev 11 crw-rw-rw- null rw root sshd 765 2 /dev 11 crw-rw-rw- null rw root sshd 765 3* internet6 stream tcp c5c8f4f0 root sshd 765 4* internet stream tcp c5c8f278 root syslogd 539 root / 2 drwxr-xr-x 512 r root syslogd 539 wd / 2 drwxr-xr-x 512 r root syslogd 539 text /usr 2332129 -r-xr-xr-x 35916 r root syslogd 539 0 /dev 11 crw-rw-rw- null rw root syslogd 539 1 /dev 11 crw-rw-rw- null rw root syslogd 539 2 /dev 11 crw-rw-rw- null rw root syslogd 539 3 /var 211990 -rw------- 3 w root syslogd 539 4* local dgram c5b0d6b8 root syslogd 539 5* local dgram c5b0d60c root syslogd 539 6* internet6 dgram udp c5b09604 root syslogd 539 7* internet dgram udp c5b09528 root syslogd 539 8 /dev 10 crw------- klog r root syslogd 539 10 - - bad - root syslogd 539 11 /var 235541 -rw-r--r-- 71647 w root syslogd 539 12 /var 235531 -rw------- 55 w root syslogd 539 13 /var 235525 -rw------- 31796 w root syslogd 539 14 /var 235536 -rw-r----- 7571 w root syslogd 539 15 /var 235527 -rw-r--r-- 55 w root syslogd 539 16 /var 235532 -rw------- 9873 w root syslogd 539 17 /var 235546 -rw------- 63558 w root syslogd 539 18 /var 235526 -rw------- 55 w root syslogd 539 19 /var 235530 -rw-r----- 55 w quagga ospfd 493 root / 2 drwxr-xr-x 512 r quagga ospfd 493 wd / 2 drwxr-xr-x 512 r quagga ospfd 493 text /usr 1060003 -r-xr-xr-x 8496 r quagga ospfd 493 0 /dev 11 crw-rw-rw- null rw quagga ospfd 493 1 /dev 11 crw-rw-rw- null rw quagga ospfd 493 2 /dev 11 crw-rw-rw- null rw quagga ospfd 493 3 /var 211988 -rw-r--r-- 4 rw quagga ospfd 493 4* internet raw ip c5c8be9c quagga ospfd 493 5* internet6 stream tcp c5c8f9e0 quagga ospfd 493 6* internet stream tcp c5c8f768 quagga ospfd 493 7* local stream c5b0d8bc quagga ospfd 493 8* local stream c5b0d810 <-> c5b0d764 quagga zebra 487 root / 2 drwxr-xr-x 512 r quagga zebra 487 wd / 2 drwxr-xr-x 512 r quagga zebra 487 text /usr 1060007 -r-xr-xr-x 169888 r quagga zebra 487 0 /dev 11 crw-rw-rw- null rw quagga zebra 487 1 /dev 11 crw-rw-rw- null rw quagga zebra 487 2 /dev 11 crw-rw-rw- null rw quagga zebra 487 3* internet6 raw ipv6-icmp c5c8c000 quagga zebra 487 4* route raw 0 c5b07b44 quagga zebra 487 5 /var 211985 -rw-r--r-- 4 rw quagga zebra 487 6* local stream c5b0db6c quagga zebra 487 7* internet6 stream tcp c5c90000 quagga zebra 487 8* internet stream tcp c5c8fc58 quagga zebra 487 9* local stream c5b0dac0 quagga zebra 487 10* local stream c5b0d764 <-> c5b0d810 quagga zebra 487 11* local dgram c5b0c158 <-> c5b0d6b8 root devd 409 root / 2 drwxr-xr-x 512 r root devd 409 wd / 2 drwxr-xr-x 512 r root devd 409 text / 141327 -r-xr-xr-x 381640 r root devd 409 0 /dev 11 crw-rw-rw-
Responsible Changed From-To: freebsd-bugs->freebsd-net Over to maintainer(s).
On Mon, Dec 21, 2009 at 11:53:22PM +0000, linimon@freebsd.org wrote: > Old Synopsis: Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets > New Synopsis: [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets > > Responsible-Changed-From-To: freebsd-bugs->freebsd-net > Responsible-Changed-By: linimon > Responsible-Changed-When: Mon Dec 21 23:51:33 UTC 2009 > Responsible-Changed-Why: > Over to maintainer(s). > > http://www.freebsd.org/cgi/query-pr.cgi?pr=141843 I'm not sure whether you're seeing one of edge case of em(4)/igb(4) checksum offload issue. Personally I couldn't reproduce the issue but I guess the checksum offload/TSO handling might cause unexpected result. If you disable Tx checksum offload or TSO em(4) work as expected, right? If so, would you try the following patch and let me know whether it makes any difference? http://people.freebsd.org/~yongari/em.csum_tso.20091229.patch
This is a multi-part message in MIME format.
On Wed, Dec 30, 2009 at 01:34:30PM +0300, Dennis Yusupoff wrote: > 29.12.2009 21:51, Pyun YongHyeon ?????: > > On Mon, Dec 21, 2009 at 11:53:22PM +0000, linimon@freebsd.org wrote: > > > >> Old Synopsis: Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets > >> New Synopsis: [em] [vlan] Intel txcsum and assigned vlan invoke wrong dst MAC in TCP packets > >> > >> Responsible-Changed-From-To: freebsd-bugs->freebsd-net > >> Responsible-Changed-By: linimon > >> Responsible-Changed-When: Mon Dec 21 23:51:33 UTC 2009 > >> Responsible-Changed-Why: > >> Over to maintainer(s). > >> > >> http://www.freebsd.org/cgi/query-pr.cgi?pr=141843 > >> > > I'm not sure whether you're seeing one of edge case of em(4)/igb(4) > > checksum offload issue. Personally I couldn't reproduce the issue > > but I guess the checksum offload/TSO handling might cause > > unexpected result. If you disable Tx checksum offload or TSO em(4) > > work as expected, right? > Not exactly. > Only checksum offload gives this issue, TSO on/off doesn't matter. > By the way, we checked UDP also and found something interested. > For it, we run echo service on server ("|echo dgram udp wait > root internal|" in //etc/inetd.conf/) and send strings by "|nc -4 -u > server 7|" from client. > So...UDP works fine before, while and after "ifconfig vlanX vlan X > vlandev em0" command given at server. And the most interesting things is > what while server "hangs" for TCP-connections, "establishing" UDP > connections between client and server...helps it to "unhang" TCP, so it > begins to works correctly. > > > If so, would you try the following patch > > and let me know whether it makes any difference? > > > > http://people.freebsd.org/~yongari/em.csum_tso.20091229.patch > > > Doesn't help, same behavior. And moreover, UDP also doesn't "help" and > doesn't work so. > Sorry, that wasn't expected one. From your explanation above I think I found a possible cause of checksum offload issue of em(4). The issue was not I initially thought though. It seems the checksum offload context configured in em(4) was incorrectly reused even if a frame requires a new context as IP/TCP header length, checksum offload offset was changed. Setting up new context put more burden to hardware such that em(4) used to avoid new context setup as possible as it can. However it seems em(4) failed to compare all required field in the checksum offload context. Would you try the following patch again? http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch > If you'd like, I could give you an access to this server via SSH, would you? Thanks for the offer. ATM I'm somewhat busy to address other driver issues. If the patch does not work and I find spare time I'll let you know.
30.12.2009 23:20, Pyun YongHyeon пиÑеÑ: > > Sorry, that wasn't expected one. From your explanation above I > think I found a possible cause of checksum offload issue of em(4). > The issue was not I initially thought though. It seems the checksum > offload context configured in em(4) was incorrectly reused even if > a frame requires a new context as IP/TCP header length, checksum > offload offset was changed. Setting up new context put more burden > to hardware such that em(4) used to avoid new context setup as > possible as it can. However it seems em(4) failed to compare all > required field in the checksum offload context. Would you try the > following patch again? > > http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch Gotcha! It works! --- With best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg
A other cause (or the same) seems that when the interface is reset (for example with a ifconfig em0 down, the chip is reset, but the last_hw_offload in the interface context is not reset I just had the following problem in FreeBSD8.0 RELEASE # ping www.google.com It works # ifconfig em0 down up # ping www.google.com It does not work # telnet -N <ipv4 router addr> Normal error message # ping www.google.com It works I made the following path to solve the problem (in em_stop): --- /sys/dev/e1000/if_em.c.DIST 2009-10-25 02:10:29.000000000 +0100 +++ /sys/dev/e1000/if_em.c 2010-02-23 18:22:53.000000000 +0100 @@ -2710,6 +2710,7 @@ ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); e1000_reset_hw(&adapter->hw); + adapter->last_hw_offload = 0; if (adapter->hw.mac.type >= e1000_82544) E1000_WRITE_REG(&adapter->hw, E1000_WUC, 0); } -- Jean-Luc RICHIER (Jean-Luc.Richier@Imag.Fr) Laboratoire d'Informatique de Grenoble (LIG) Campus de Grenoble 681 rue de la Passerelle BP 72, F-38402 St Martin d'Hères Cedex Tel : +33 4 76 82 72 32 Fax : +33 4 76 82 72 87
Jean-Luc Richier wrote: > A other cause (or the same) seems that when the interface is reset (for example with a ifconfig em0 down, the chip is reset, but the > last_hw_offload in the interface context is not reset .... > I made the following path to solve the problem (in em_stop) > --- /sys/dev/e1000/if_em.c.DIST 2009-10-25 02:10:29.000000000 +0100 > +++ /sys/dev/e1000/if_em.c 2010-02-23 18:22:53.000000000 +0100 ... > + adapter->last_hw_offload = 0; ... I tested the suggested patch http://people.freebsd.org/~yongari/em.csum_tso.20091230.patch and it also solved my problem Regards -- Jean-Luc RICHIER (Jean-Luc.Richier@Imag.Fr) Laboratoire d'Informatique de Grenoble (LIG) Campus de Grenoble 681 rue de la Passerelle BP 72, F-38402 St Martin d'Hères Cedex Tel : +33 4 76 82 72 32 Fax : +33 4 76 82 72 87
This is still a problem in RELENG_8 as of Aug 28th. The patch referenced in <http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058305.html>http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058305.html fixes the issue for me ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike
Responsible Changed From-To: freebsd-net->jfv Over to maintainer.
It's seems that you can close this PR since all works fine at least from 8.1-RELEASE. -- With best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg
Closed at request of submitter.