Bug 223831

Summary: [patch] swap_pager.c to release multiple swap blocks at a time
Product: Base System Reporter: ota
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: alc, dougm, emaste, kib, markj
Priority: --- Keywords: patch
Version: 11.1-STABLE   
Hardware: Any   
OS: Any   
Description Flags
release multiple blocks at a time
swapoff to release multiple blocks at a time
swapoff to release multiple blocks at a time for 11-stable
swapoff to release multiple blocks at a time for 12-head none

Description ota 2017-11-23 21:41:46 UTC
Created attachment 188225 [details]
release multiple blocks at a time

This allows to release multiple swap blocks via swp_pager_freeswapspace().

For each call, swp_pager_freeswapspace() locks.  Releasing multiple
blocks helps to reduce number of locks.  If a swblk contains
continuous blocks, it reduces up to SWAP_META_PAGES-1 locks.

This patch covers 2 case: 1. when a process exits and all of its
swap blocks are released and 2. when a process is paged in from
swap space.
Comment 1 ota 2017-12-02 03:22:42 UTC
Created attachment 188460 [details]
swapoff to release multiple blocks at a time

https://svnweb.freebsd.org/base?view=revision&revision=326329 change works better for the first patchset and already committed to the head.

This one is to release multiple blocks from "swapoff" command.
The patch allows to take SWB_NPAGES blocks at once from backing store.
On i386, it is 32 blocks maximum compare to 1 block at a time.
Comment 2 Konstantin Belousov freebsd_committer 2017-12-02 13:43:55 UTC
The patch is not applicable to HEAD, please update it.

For the swap_pager_swapoff() chunk, it seems that the new variable offset can be used before initialized.

That said, why optimize swapoff at all ?
Comment 3 ota 2017-12-03 04:59:39 UTC
(In reply to Konstantin Belousov from comment #2)

Indeed, 1st patch chunk failed and I overlooked it.
Also, I missed offset initialization.

To answer your question, I started with 2 reasons.
#1 - I started looking at optimizing page-in cases and process exit cases where they had released 1 block at a time although swp_pager_freeswapspace() takes npages to free.  This was the 3rd one.
#2 - swp_pager_force_pagein() comment says "it doesn't work" and I wondered why.  The reason I found is we need to call vm_page_grab for the number of pages we want to page in.

Summing up, after I figured vm_page part, the code started working okay and based on my observation of "top", multi-block page-in was considerably faster - reading 1 block 32 times compare to 32 blocks a time.  It looksvm_object_t is locked during swapoff operation (while paging-in), too.

During research, I found swap_pager_getpages() calls swap_pager_haspage() anyway and swap_pager_haspage() checks how many continuous blocks are behind.  So, I started researching from swap_pager_haspage() side instead of swblk side.

I had few questions like when to lock and how and when to increment vm_object_pip.  I posted here for some feed backs.

While checking patch failure against HEAD, I realized swp_pager_force_pagein() in HEAD had a change for (m->valid == VM_PAGE_BITS_ALL) case.  This change cleared some of my questions.  I will update my changes and post a new one for 11-STABLE and HEAD.
Comment 4 ota 2017-12-07 03:47:43 UTC
Created attachment 188607 [details]
swapoff to release multiple blocks at a time for 11-stable

Initialize offset.
Split vm operations into functions.

It looks I need to call vm_page_grab and cannot use rbehind to call 
vm_page_alloc() is called by swap_pager_getpages() but I got a crash.
Comment 5 ota 2017-12-07 03:48:36 UTC
Created attachment 188608 [details]
swapoff to release multiple blocks at a time for 12-head
Comment 6 Konstantin Belousov freebsd_committer 2017-12-09 11:55:22 UTC
(In reply to ota from comment #5)
1. Generate the patch for HEAD using svn diff -x -U 999999
2. Publish the patch on https://reviews.freebsd.org, assigning me as the reviewer.
Comment 7 ota 2017-12-14 06:27:53 UTC
(In reply to Konstantin Belousov from comment #6)

I created https://reviews.freebsd.org/D13484 for this case.

I have another change-set for a improvement and also created https://reviews.freebsd.org/D13485
Comment 8 Ed Maste freebsd_committer 2019-05-21 19:33:39 UTC
Triaged during 2019 Waterloo Hackathon, Doug would you be interested in shepherding this change in?
Comment 9 ota 2019-08-12 09:20:18 UTC
Yes, he addressed this, https://reviews.freebsd.org/D20635.