Page MenuHomeFreeBSD

Live Migration feature for bhyve [Part 1]
Needs ReviewPublic

Authored by on Mar 31 2022, 6:57 AM.
Referenced Files
Unknown Object (File)
Wed, May 8, 11:58 AM
Unknown Object (File)
Apr 22 2024, 6:49 AM
Unknown Object (File)
Mar 22 2024, 8:17 AM
Unknown Object (File)
Mar 22 2024, 8:15 AM
Unknown Object (File)
Mar 22 2024, 8:13 AM
Unknown Object (File)
Mar 7 2024, 11:17 PM
Unknown Object (File)
Mar 7 2024, 9:56 PM
Unknown Object (File)
Jan 19 2024, 12:52 AM


Group Reviewers

These 2 tickets represent an implementation for the live migration feature, as specified in the first ticket.

This feature sends the guest's memory while it is still running. To do this, it uses migration rounds. In the first round, all the memory is sent over. In subsequent rounds, because the guest is still running and its memory can be modified anytime, only the modified pages since the last round are sent over in order to update the destination's memory. Currently, it uses a fixed number of rounds, 4 (this may be changed in the future). After these rounds finish, the source guest is stopped and the remaining modified memory pages are sent over, alongside the kernel structures' and emulated devices' states. Once this is completed, the guest can be resumed at the destination.

For detecting the pages that were modified between rounds, we use a flag (VPO_VMM_DIRTY) for each vm_page. This flag is set each time vm_page->dirty field is updated but can only be reset when the page is migrated.

The features transfers only through IPv4. The migration procedure works only between identical workstations.

More information regarding the implementation can be found here:

The usage of the feature is detailed here: Virtual Machine Migration using bhyve.

This first part adds the VPO_VMM_DIRTY bit and related functions in the virtual memory system, as well as an ioctl to search for such dirty pages.

Diff Detail

rG FreeBSD src repository
Lint Skipped
Tests Skipped

Event Timeline retitled this revision from Live Migration feature for bhyve [Part 5] to Live Migration feature for bhyve [Part 1]. added a child revision: Restricted Differential Revision.Mar 31 2022, 6:59 AM removed a child revision: Restricted Differential Revision.Apr 6 2022, 4:31 PM changed the visibility from "All Users" to "Public (No Login Required)".Apr 1 2023, 5:00 AM