Change Details

1. pmap_emulate_modify was assuming that no changes to the pmap could take place between the TLB signaling the fault and pmap_emulate_modify's acquisition of the pmap lock, but that's clearly not even true in the uniprocessor case, nevermind the SMP case. Address each possibility in turn. 2. MipsDoTLBMiss will load a segmap entry or pde, check that it isn't zero, and then chase that pointer to a physical page. If that page has been freed in the interim, it will read garbage and go on to populate the TLB with it. This can happen because pmap_unwire_ptp zeros out the pde and vm_page_free_zero()s the ptp (or, recursively, zeros out the segmap entry and vm_page_free_zero()s the pdp) without interlocking against MipsDoTLBMiss. The pmap is locked, and pvh_global_lock may or may not be held, but this is not enough. Solve this issue by inserting TLB shootdowns within _pmap_unwire_ptp; as MipsDoTLBMiss runs with IRQs deferred, the IPIs involved in TLB shootdown are sufficient to ensure that MipsDoTLBMiss sees either a zero segmap entry / pde or a non-zero entry and the pointed-to page still not freed. Thanks to Konstantin Belousov and Mark Johnston for confirming the bugs.

MipsDoTLBMiss will load a segmap entry or pde, check that it isn't zero, and then chase that pointer to a physical page. If that page has been freed in the interim, it will read garbage and go on to populate the TLB with it. This can happen because pmap_unwire_ptp zeros out the pde and vm_page_free_zero()s the ptp (or, recursively, zeros out the segmap entry and vm_page_free_zero()s the pdp) without interlocking against MipsDoTLBMiss. The pmap is locked, and pvh_global_lock may or may not be held, but this is not enough. Solve this issue by inserting TLB shootdowns within _pmap_unwire_ptp; as MipsDoTLBMiss runs with IRQs deferred, the IPIs involved in TLB shootdown are sufficient to ensure that MipsDoTLBMiss sees either a zero segmap entry / pde or a non-zero entry and the pointed-to page still not freed. Thanks to Konstantin Belousov and Mark Johnston for confirming the bug.

1.MipsDoTLBMiss will load a segmap entry or pde, pmap_emulate_modify was assumingcheck that no changes to the pmap could takeit isn't zero, place between the TLB signaling the fault and pmap_emulate_modify's acquisition of the pmap lock, but that's clearly not even true in the uniprocessor case, nevermind the SMP case. Address each possibility in turn. 2. MipsDoTLBMiss will load a segmap entry or pde, check that it isn't zero, and then chase that pointer to a physical page. If that page has been freed in the interim, it will read garbage and go on to populate the TLB with it. This can happen because pmap_unwire_ptp zeros out the pde and vm_page_free_zero()s the ptp (or, recursively, zeros out the segmap entry and vm_page_free_zero()s the pdp) without interlocking against MipsDoTLBMiss. The pmap is locked, and pvh_global_lock may or may not be held, but this is not enough. Solve this issue by inserting TLB shootdowns within _pmap_unwire_ptp; as MipsDoTLBMiss runs with IRQs deferred, the IPIs involved in TLB shootdown are sufficient to ensure that MipsDoTLBMiss sees either a zero segmap entry / pde or a non-zero entry and the pointed-to page still not freed. Thanks to Konstantin Belousov and Mark Johnston for confirming the bugs.