Bug 203445 - Receive side scaling (RSS) for more than 16 queues not working in "ixl" driver
Summary: Receive side scaling (RSS) for more than 16 queues not working in "ixl" driver
Status: Closed Feedback Timeout
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Some People
Assignee: Eric Joyner
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2015-09-29 19:59 UTC by tushar.swain
Modified: 2017-09-13 16:26 UTC (History)
4 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description tushar.swain 2015-09-29 19:59:14 UTC 
Issue -

Receive side scaling (RSS) for more than 16 queues not working in "ixl" driver.

Version:
The FreeBSD driver version is 1.4.1 

Description:

My system has 20 CPU cores. The packet-processing application uses NIC RSS feature to filter packets to dedicated receive queues for each core.

NIC is able to do RSS properly for Queue-0 to Queue-15, and packets reach the right receive queues. 

But not a single packet reach Queue-16/17/18/19. 

Instead Packets for Queue-16/17/18/18 reach Queue-0/1/2/3 

Possibly some RSS specific register settings missing in the drivers that restricts NIC from using more than 16 receive queues

Linux driver doesn't seem to exhibit this behavior.
Comment 1 Eric Joyner freebsd_committer freebsd_triage 2015-10-01 01:07:38 UTC 
So it looks like the hash is being used to index into the HLUT, but the resulting queue is then mod 16.

To illustrate:

HLUT(45): 0x24252627 (where bits 21:16 contain LUT entry 4*45+2) and a packet received on queue 5 has a hash of 0xa33600b6 in its RX descriptor. According to the HLUT, it should really have gone to queue 37.

This is really odd.
Comment 2 Eric Joyner freebsd_committer freebsd_triage 2015-10-01 02:11:39 UTC 
Try replacing the line

ctxt.info.tc_mapping[0] = 0x0800

with

ctxt.info.tc_mapping[0] = 0x0c00
Comment 3 Eric Joyner freebsd_committer freebsd_triage 2015-10-02 15:54:17 UTC 
Tushar has said this enabled RSS on all of his queues, so it looks like the change I posted worked and should be included in the driver.
Comment 4 Sean Bruno freebsd_committer freebsd_triage 2016-05-06 17:26:11 UTC 
(In reply to Eric Joyner from comment #3)
Eric:

Did this get dropped into the ixl(4) update you pushed to phabricator?
https://reviews.freebsd.org/D6211
Comment 5 Eric Joyner freebsd_committer freebsd_triage 2016-05-06 17:37:02 UTC 
(In reply to Sean Bruno from comment #4)

I fixed this in a previous phabricator review that was committed (https://reviews.freebsd.org/D5203), but I enhanced it slightly there and need to include those changes in the review I posted above, though.
Comment 6 Eric Joyner freebsd_committer freebsd_triage 2017-08-24 21:38:23 UTC 
Did the submitter get a chance to see if this fixed his problem? Otherwise, I'm going to close this.