Bug 233395 - In FreeBSD 11.x contigfree takes too much time on Xeon Gold 5115 CPU
Summary: In FreeBSD 11.x contigfree takes too much time on Xeon Gold 5115 CPU
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 11.2-STABLE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-11-22 05:04 UTC by Steevan
Modified: 2018-11-22 05:08 UTC (History)
0 users

See Also:


Attachments
Contains simple kernel module source and a makefile (1.16 KB, application/zip)
2018-11-22 05:04 UTC, Steevan
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steevan 2018-11-22 05:04:20 UTC
Created attachment 199442 [details]
Contains simple kernel module source and a makefile

Hi

I am seeing a FreeBSD 11.x OS poor performance issue on Xeon Gold CPU.
CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with 10 cores per socket )

I have attached a simple kernel module program ( for FreeBSD) and its Makefile.
This just allocates  32KB contiguous memory using contigmalloc and then frees it using contigfree.
This repeated 1000 times and time taken by contigmalloc and contigfree is computed .

I can see that contigfree takes way too much time .

I have another older server running FreeBSD 11.2   Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz  (dual socket 6 core per socket and in this I do not see such an issue .

Any idea why contigfree takes so much time on FreeBSD 11.x running on Xeon Gold 5115 CPU ?

It looks like this issue affects only the systems which have more than 16 physical CPU cores.

Because of this issue the driver for our PCIe card takes more than 2 minutes to load and upto 15 minutes to unload. 

System and CPU details: 
CPU: Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz (2400.07-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 40 CPUs

System Information
    Manufacturer: Supermicro
    Product Name: Super Server
    Version: 0123456789
Base Board Information
    Manufacturer: Supermicro
    Product Name: X11DPH-Tq
    Version: 1.01

Regards,
Steevan
Comment 1 Steevan 2018-11-22 05:08:16 UTC

I tried contacting SuperMicro and then Intel tech support. Both suggested that the issue could be specific to FreeBSD OS. 

So later I tried FreeBSD 12 Beta 3 version  on this server with Xeon Gold 5115 CPU.

All these problems have disappeared.  In this FreeBSD 12 Beta3 also contigfree takes much more time compared to contigmalloc. 

However  when I compare the absolute values to FreeBSD 11.x number I can see huge improvement in FreeBSD 12 Beta 3.

Now in FreeBSD 12 Beta3 my PCIe card driver takes just a few seconds to load and few seconds to unload. 
However on FreeBSD 11.x it takes 2+ minutes to load and upto 15 minutes to unload. 
 
Hence it looks like the problem is with FreeBSD 11.x. 
Hope the attached simple program in original bug report helps you guys to reproduce this issue. 

Best Regards,
Steevan