Since upgrading from 11.1-RELEASE p10 to 11.2-BETA3 on amd64 we are seeing excessive swapping, even when the system is idle.
Swap is on zvol, with system constantly swapping in/out for no apparent reason. There is no load on the system:
5:22PM up 1 day, 7 mins, 1 user, load averages: 0.06, 0.20, 0.18
However the following lines from top confirm that excessive swapping is occurring:
Swap: 4096M Total, 158M Used, 3938M Free, 3% Inuse, 448K In, 136K Out
Swap: 4096M Total, 156M Used, 3940M Free, 3% Inuse, 8K In, 44K Out
Swap: 4096M Total, 157M Used, 3939M Free, 3% Inuse, 332K In, 24K Out
System has 64GB of RAM, 8-core Xeon E5-2620 v4 CPU, all memory and ZFS tunables are at defaults.
ZFS settings that were changed on swap zvol are:
The following process seems to constantly active:
Did not exhibit this behaviour on 11.1-RELEASE with exactly the same configuration.
issue still exists on 11.2-RC2.
have deployed 11.2-RC2 onto another system and are seeing the same behaviour of excessive swapping.
reducing vfs.zfs.arc_max seems to help reduce the swapping.
I have the same issue, albeit with swapping to a gmirror device instead of a zvol.
Bug #224975 may be related.
The following thread seems to be related: https://lists.freebsd.org/pipermail/freebsd-stable/2018-June/089132.html
I am now trying the patch in https://reviews.freebsd.org/D7538 (mentioned in that thread).
In general one can say that there has been a massive regression regarding ZFS + swap between 11.1 and 11.2.
I have now deployed D7538 on three machines that had all been affected by excessive swapping. So far their swap usage stays at zero.
I have yet to check what happens under memory pressure.
It seems that with this patch the ARC still grows until the system starts to swap again.
"swapoff -a" results in "swapoff: /dev/mirror/swap: Cannot allocate memory"
In 11.1 this used to work.
Also, using "sysctl vfs.zfs.arc_max=xxxxxxxxxx" to reduce the size of the ARC cache does not seem to free memory (top shows that wired memory remains unchanged).
ZFS + swap is very broken under 11.2.