Page Menu
Home
FreeBSD
Search
Configure Global Search
Log In
Files
F177866510
D61924.1795465500.diff
No One
Temporary
Actions
View File
Edit File
Delete File
View Transforms
Subscribe
Flag For Later
Award Token
Authored By
Unknown
Size
3 KB
Referenced Files
None
Subscribers
None
D61924.1795465500.diff
View Options
diff --git a/share/man/man9/fpu_kern.9 b/share/man/man9/fpu_kern.9
--- a/share/man/man9/fpu_kern.9
+++ b/share/man/man9/fpu_kern.9
@@ -21,7 +21,7 @@
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
-.Dd October 13, 2020
+.Dd November 20, 2026
.Dt FPU_KERN 9
.Os
.Sh NAME
@@ -129,6 +129,12 @@
which can be executed in a critical section.
It avoids the need to allocate the FPU context by the cost
of increased system latency.
+The content of the FPU registers at the start of such block is
+unspecified.
+On amd64, only the SSE control and status register
+.Pq Va MXCSR
+is initialized to its default value; the code which uses x87 or MMX
+instructions must initialize the x87 state itself.
.El
.El
.Pp
diff --git a/sys/amd64/amd64/efirt_machdep.c b/sys/amd64/amd64/efirt_machdep.c
--- a/sys/amd64/amd64/efirt_machdep.c
+++ b/sys/amd64/amd64/efirt_machdep.c
@@ -331,6 +331,13 @@
PMAP_LOCK_ASSERT(curpmap, MA_OWNED);
curthread->td_md.md_efirt_dis_pf = vm_fault_disable_pagefaults();
+ /*
+ * The UEFI calling convention requires the default x87 control
+ * word on entry to the firmware, in addition to the default
+ * MXCSR which fpu_kern_enter() provides.
+ */
+ __asm __volatile("fninit");
+
/*
* IPI TLB shootdown handler invltlb_pcid_handler() reloads
* %cr3 from the curpmap->pm_cr3, which would disable runtime
diff --git a/sys/amd64/amd64/fpu.c b/sys/amd64/amd64/fpu.c
--- a/sys/amd64/amd64/fpu.c
+++ b/sys/amd64/amd64/fpu.c
@@ -1234,6 +1234,49 @@
return ((struct savefpu *)p);
}
+/*
+ * Set up the FPU for a FPU_KERN_NOCTX section.
+ *
+ * Only the control state that SSE and AVX code depends on is initialized:
+ * MXCSR gets its default value, so that rounding is predictable and all
+ * exceptions are masked. The data registers, and the rest of the extended
+ * state, keep whatever the previous user of the FPU on this CPU left in
+ * them, same as for a function called according to the ABI. This is
+ * much cheaper than loading the complete initial state.
+ *
+ * The exception is the upper halves of the %ymm registers, which are
+ * cleared. Otherwise the code using legacy SSE encodings, which is what
+ * the compiler generates for the kernel unless told otherwise, would run
+ * with the upper halves dirty and pay the SSE/AVX transition or merge
+ * penalty that many CPUs impose. VZEROUPPER also clears the corresponding
+ * bits of %zmm0-%zmm15. %zmm16-%zmm31 and the opmask registers cannot be
+ * referenced by legacy encodings and are left alone.
+ *
+ * x87 state is not initialized. The control word may have exceptions
+ * unmasked, the register stack may be not empty, and an unmasked
+ * exception may be pending, which the next x87 or MMX instruction would
+ * report. The section which needs x87 or MMX must start with FNINIT.
+ *
+ * PKRU is not a part of the FPU state for the kernel, it is switched
+ * with the thread.
+ *
+ * No XRSTOR is needed for the sake of the XSAVEOPT modified optimization.
+ * The section runs with fpcurthread set to NULL, so the registers are
+ * not saved into any thread's save area during or after it, until some
+ * thread gets the ownership of the FPU. That always starts with XRSTOR
+ * from the save area which the thread's state is later saved into.
+ */
+static void
+fpu_kern_noctx_init(void)
+{
+ u_int mxcsr;
+
+ mxcsr = __INITIAL_MXCSR__;
+ ldmxcsr(mxcsr);
+ if (use_xsave && (xsave_mask & XFEATURE_ENABLED_AVX) != 0)
+ __asm __volatile("vzeroupper");
+}
+
void
fpu_kern_enter(struct thread *td, struct fpu_kern_ctx *ctx, u_int flags)
{
@@ -1258,12 +1301,7 @@
("invalid fpcurthread"));
}
- /*
- * This breaks XSAVEOPT tracker, but
- * PCB_FPUNOSAVE state is supposed to never need to
- * save FPU context at all.
- */
- fpurestore(fpu_initialstate);
+ fpu_kern_noctx_init();
set_pcb_flags(pcb, PCB_KERNFPU | PCB_FPUNOSAVE |
PCB_FPUINITDONE);
return;
File Metadata
Details
Attached
Mime Type
text/plain
Expires
Mon, Nov 23, 8:25 PM (22 h, 48 s)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
41939508
Default Alt Text
D61924.1795465500.diff (3 KB)
Attached To
Mode
D61924: amd64: Do not load the initial FPU state for FPU_KERN_NOCTX
Attached
Detach File
Event Timeline