Bug 93530

Summary: [pf] Incorrect checksums when using pf's route-to on sparc64
Product: Base System Reporter: Pieter de Boer <pieter>
Component: sparc64Assignee: freebsd-sparc64 (Nobody) <sparc64>
Status: Closed Overcome By Events    
Severity: Affects Only Me CC: kp
Priority: Normal    
Version: 6.1-PRERELEASE   
Hardware: Any   
OS: Any   

Description Pieter de Boer 2006-02-18 16:00:19 UTC 
Using route-to statements in pf gives incorrect checksums on sparc64.
Take for instance the following statement:

 pass out quick route-to (hme0 10.0.0.1) from 10.0.0.2 to any

If hme0 has 10.0.0.2 configured and the system sends a packet to
10.0.0.1, it has an incorrect checksum (tcpdumped on the receiving system):

 16:27:45.879243 IP (tos 0x0, ttl  64, id 27, offset 0, flags [none],
 proto: ICMP (1), length: 84, bad cksum 9041 (->b4a4)!) 10.0.0.2 >
 10.0.0.1: ICMP echo reply, id 35127, seq 8, length 64

Default install, tcxsum and rxcsum don't make a difference. Traffic
that's not routed using route-to has correct checksums.

How-To-Repeat: 
See 'Description', in short: use pf route-to on sparc64 and see wrong
checksums.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2006-02-18 20:31:45 UTC 
Responsible Changed
From-To: gnats-admin->freebsd-pf

Over to maintainer(s).
Comment 2 Pieter de Boer 2006-02-26 14:48:18 UTC 
I've investigated some more and found an interesting heisenbug.

From pf.c, pf_route():

                ip->ip_sum = 0;
                if (sw_csum & CSUM_DELAY_IP) {
                        /* From KAME */
                        if (ip->ip_v == IPVERSION &&
                            (ip->ip_hl << 2) == sizeof(*ip)) {
                                ip->ip_sum = in_cksum_hdr(ip);
                        } else {
                                ip->ip_sum = in_cksum(m0, ip->ip_hl << 2);
                        }
                }

In the tests I've run, the in_cksum_hdr()-function is called, not the
in_cksum() function. Ok,  I inserted some printf's to find the problem. My

new code looks like this:

                ip->ip_sum = 0;
                if (sw_csum & CSUM_DELAY_IP) {
//                      printf("pf_route(): B1\n");
                        /* From KAME */
                        if (ip->ip_v == IPVERSION &&
                            (ip->ip_hl << 2) == sizeof(*ip)) {      
//                              printf("pf_route: B2\n");     
                                ip->ip_sum = in_cksum_hdr(ip);    
                        } else {
                        //      printf("pf_route: B3\n");
                                ip->ip_sum = in_cksum(m0, ip->ip_hl << 2);
                        }
                }
With the printf B1 and B2 commented out, the checksums are wrong. With either
printf B1 or B2 not commented out, then the checksums are correct. My theory
is that there's some caching issue between the ip->ip_sum = 0; at the top and
the assembly-code of in_cksum_hdr(). When a printf is inserted between the
ip->ip_sum = 0; and the in_cksum_hdr(), the cache is invalidated long before
in_cksum_hdr() is called.

Perhaps a Sparc64-hacker could take a look at the assembly output of
pf_route() and determine whether this could be the case? :)
Comment 3 Pieter de Boer 2006-02-26 17:32:08 UTC 
After some hints from a nice Swedish guy, I found out that compiling
pf.ko with CFLAGS=-O instead of -O2 seems to fix this problem.
Comment 4 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:46 UTC 
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped