+ <url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Virtual-Machine-Migration-using-bhyve">Github wiki - How to Live and Warm Migrate a bhyve guest</url>
+ <url href="https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_migration_dev">Github - Live Migration branch</url>
+ </links>
+
+ <body>
+ <p>The Migration feature uses the Save/Restore feature to
+ migrate a bhyve guest
+ from a FreeBSD host to another FreeBSD host. To migrate a
+ bhyve guest,
+ one needs to start an empty guest on the destination host
+ from a shared guest
+ image using the bhyve tool with the <tt>-R</tt> option
+ followed by the source host
+ IP and the port to listen to migration request. On the
+ source host, the
+ migration is started by executing the bhyvectl command
+ with the <tt>--migrate</tt>
+ or <tt>--migrate-live</tt> option, followed by the
+ destination host IP and the
+ port to send to the messages.</p>
+
+ <p>New features added:</p>
+
+ <ul>
+ <li>Clear the dirty bit after each migration round</li>
+
+ <li>Extend live migration to highmem segment</li>
+ </ul>
+
+ <p>
+ Future tasks:</p>
+
+ <ul>
+ <li>Refactor live migration branch</li>
+
+ <li>Rebase live migration</li>
+
+ <li>Extend live migration to unwired memory</li>
+ </ul>
+
+ </body>
+
+ <sponsor>
+ Matthew Grooms
+ </sponsor>
+
+ </project>
+
+ <project cat='proj'>
+ <title>bhyve - Save/Restore</title>
+
+ <contact>
+ <person>
+ <name>Elena Mihailescu</name>
+ <email>elenamihailescu22@gmail.com</email>
+ </person>
+ <person>
+ <name>Darius Mihai</name>
+ <email>dariusmihaim@gmail.com</email>
+ </person>
+ <person>
+ <name>Mihai Carabas</name>
+ <email>mihai@freebsd.org</email>
+ </person>
+ </contact>
+
+ <links>
+ <url href="https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_snapshot">Github repository for the snapshot feature for bhyve</url>
+ <url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Save-and-Restore-a-virtual-machine-using-bhyve">Github wiki - How to Save and Restore a bhyve guest</url>
+ <url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Suspend-Resume-test-matrix">Github wiki - Suspend/resume test matrix</url>
+ <url href="https://reviews.freebsd.org/D19495">Phabricator review - bhyve Snapshot Save and Restore</url>
+ </links>
+
+ <body>
+ <p>The Save/Restore for bhyve feature is a suspend and resume
+ facility added to the
+ FreeBSD/amd64's hypervisor, bhyve. The bhyvectl tool is
+ used to save the guest
+ state in three files (a file for the guest memory, a file
+ for the states of
+ various devices and the state of the CPU, and another one
+ for some metadata that
+ is used in the restore process).
+ To suspend a bhyve guest, the bhyvectl tool must be run
+ with the <tt>--suspend
+ <state_file_name></tt>
+ option followed by the guest name.</p>
+
+ <p>To restore a bhyve guest from a checkpoint, one simply has
+ to add the <tt>-r</tt> option
+ followed by the main state file (the same file that was
+ given to the <tt>--suspend</tt>
+ option for bhyvectl) when starting the VM.</p>
+
+ <p>New features added:</p>
+
+ <ul>
+ <li>Open ticket on Phabricator</li>
+
+ <li>Apply feedback received from community</li>
+ </ul>
+
+ <p>
+ Future tasks:</p>
+
+ <ul>
+ <li>Add suspend/resume support for nvme</li>
+
+ <li>Add suspend/resume support for virtio-console</li>
+
+ <li>Add suspend/resume support for virtio-scsi</li>
+
+ <li>Add TSC offsetting for restore for AMD CPUs</li>