Bug 230773 - [bhyve] GDT limit needs reset on VMX exit
Summary: [bhyve] GDT limit needs reset on VMX exit
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-20 12:46 UTC by John Levon
Modified: 2018-11-26 17:47 UTC (History)
6 users (show)

See Also:
jhb: mfc-stable11+
jhb: mfc-stable10-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description John Levon 2018-08-20 12:46:47 UTC
Intel vol 3 27.5.2:

"The base addresses for GDTR and IDTR are loaded from the GDTR base-address field and the IDTR base-address field, respectively. If the processor supports the Intel 64 architecture and the processor supports N < 64 linear-address bits, each of bits 63:N of each base address is set to the value of bit Nā€“1 of that base address. The GDTR and IDTR limits are each set to FFFFH."

As far as I could tell, bhyve doesn't do this in current FreeBSD, leaving the GDT limit significantly larger than the previous NGDT-based one.

We found and fixed this in our port as seen here:

https://github.com/joyent/illumos-joyent/commit/1d0338f3f33eec2ed45ec5a6cae30c399a3ef769#diff-5b270bdc09d0e88fd39cfed0ccd0c44b
Comment 1 Marcelo Araujo freebsd_committer 2018-08-21 21:48:11 UTC
Could you please make a patch against FreeBSD HEAD and attach it here?

Best,
Comment 2 John Levon 2018-08-22 09:02:04 UTC
Hi, just to clarify, the patch I referred to is just for reference.

I mainly filed this bug upstream as the implications are pretty unpleasant, hopefully one of you can handle the fix.
Comment 3 commit-hook freebsd_committer 2018-10-11 18:27:46 UTC
A commit references this bug:

Author: jhb
Date: Thu Oct 11 18:27:20 UTC 2018
New revision: 339312
URL: https://svnweb.freebsd.org/changeset/base/339312

Log:
  Fully restore the GDTR, IDTR, and LDTR after VT-x VM exits.

  The VT-x VMCS only stores the base address of the GDTR and IDTR.  As a
  result, VM exits use a fixed limit of 0xffff for the host GDTR and
  IDTR losing the smaller limits set in when the initial GDT is loaded
  on each CPU during boot.  Explicitly save and restore the full GDTR
  and IDTR contents around VM entries and exits to restore the correct
  limit.

  Similarly, explicitly save and restore the LDT selector.  VM exits
  always clear the host LDTR as if the LDT was loaded with a NULL
  selector and a userspace hypervisor is probably using a NULL selector
  anyway, but save and restore the LDT explicitly just to be safe.

  PR:		230773
  Reported by:	John Levon <levon@movementarian.org>
  Reviewed by:	kib
  Tested by:	araujo
  Approved by:	re (rgrimes)
  MFC after:	1 week

Changes:
  head/sys/amd64/include/cpufunc.h
  head/sys/amd64/vmm/intel/vmx.c
Comment 4 Ed Maste freebsd_committer 2018-11-12 17:15:59 UTC
Still waiting on merge to 12?
Comment 5 John Baldwin freebsd_committer freebsd_triage 2018-11-12 17:19:13 UTC
No, it was committed to head before the 12 branch.  But I do think that means it is no longer a 12.0 blocker.
Comment 6 commit-hook freebsd_committer 2018-11-18 01:08:11 UTC
A commit references this bug:

Author: jhb
Date: Sun Nov 18 01:07:37 UTC 2018
New revision: 340545
URL: https://svnweb.freebsd.org/changeset/base/340545

Log:
  MFC 339312,339364: Restore more descriptors during VM exits.

  339312:
  Fully restore the GDTR, IDTR, and LDTR after VT-x VM exits.

  The VT-x VMCS only stores the base address of the GDTR and IDTR.  As a
  result, VM exits use a fixed limit of 0xffff for the host GDTR and
  IDTR losing the smaller limits set in when the initial GDT is loaded
  on each CPU during boot.  Explicitly save and restore the full GDTR
  and IDTR contents around VM entries and exits to restore the correct
  limit.

  Similarly, explicitly save and restore the LDT selector.  VM exits
  always clear the host LDTR as if the LDT was loaded with a NULL
  selector and a userspace hypervisor is probably using a NULL selector
  anyway, but save and restore the LDT explicitly just to be safe.

  339364:
  Reload the LDT selector after an AMD-v #VMEXIT.

  cpu_switch() always reloads the LDT, so this can only affect the
  hypervisor process itself.  Fix this by explicitly reloading the host
  LDT selector after each #VMEXIT.  The stock bhyve process on FreeBSD
  never uses a custom LDT, so this change is cosmetic.

  PR:		230773

Changes:
_U  stable/11/
  stable/11/sys/amd64/include/cpufunc.h
  stable/11/sys/amd64/vmm/amd/svm.c
  stable/11/sys/amd64/vmm/intel/vmx.c