loader: set hw.vga.textmode=1 on hypervisors

[decui_microsoft.com]

Summary

loader: set hw.vga.textmode=1 on hypervisors

Since Nov 2014, vt(4) has been the default console driver, but it's slow
in graphics mode when FreeBSD guest runs on hypervisors, especially on
Hyper-V, but luckily it's fast enough in text mode, so we can resolve
the issue by setting hw.vga.textmode=1 in the loader (I think it's not
good to ask the user to explicitly set this in /boot/loader.conf).

With the patch, the user can still overwrite the value hw.vga.textmode
by manual "set hw.vga.textmode=0" via the loader command interface, or
adding "hw.vga.textmode=0" into /boot/loader.conf.

[decui_microsoft.com]

fix tab/space indentation and function declaration's style.

[royger]

Hm, it might we worth to do this for all hypervisors and not only HyperV, this can be easily done by checking the hypervisor bit in CPUID.

Also, I guess this can be overwritten by setting hw.vga.textmode=0, in which case it will not be forced to 1?

Roger.

[sepherosa_gmail.com]

In D5727#122310, @royger wrote:

Hm, it might we worth to do this for all hypervisors and not only HyperV, this can be easily done by checking the hypervisor bit in CPUID.

Did you notice the console slowness in Xen or other hypervisors?

Also, I guess this can be overwritten by setting hw.vga.textmode=0, in which case it will not be forced to 1?

Roger.

Yeah, if the user set it, Dexuan's code will not overwrite user's setting.

[howard0su_gmail.com]

Even failed to boot in one of VM with 24 vCPU

[decui_microsoft.com]

In D5727#122310, @royger wrote:

Hm, it might we worth to do this for all hypervisors and not only HyperV, this can be easily done by checking the hypervisor bit in CPUID.

Roger, did you notice 11-CURRENT boots very slow on Xen?

On Hyper-V, it boots very slow, the boot messages are printed at a rate of about one line per second! This is probably Hyper-V specific. I tested on KVM: it's much faster than Hyper-V, so I don't think KVM needs this.
I don't have Xen or VMware at hand, so I can't test them.

Also, I guess this can be overwritten by setting hw.vga.textmode=0, in which case it will not be forced to 1?

Roger.

Yes, we can still overwrite this by manually specifying "set hw.vga.textmode=0" from the loader screen, or adding "hw.vga.textmode=0" into /boot/loader.conf.

[royger]

It's been some time since I've tested this on Xen, but there was a difference between using hw.vga.textmode or not. It's not like 1 line per second, but it used to be noticeable.

[decui_microsoft.com]

In D5727#122322, @royger wrote:

It's been some time since I've tested this on Xen, but there was a difference between using hw.vga.textmode or not. It's not like 1 line per second, but it used to be noticeable.

Can you please confirm Xen also conforms to "CPUID(EAX==1).ECX.bit31 == 1" ? I can update my patch to cover all the hypervisors conforming to this?

[royger]

Yes, it does.

[kib]

In D5727#122342, @decui_microsoft.com wrote:

In D5727#122322, @royger wrote:

It's been some time since I've tested this on Xen, but there was a difference between using hw.vga.textmode or not. It's not like 1 line per second, but it used to be noticeable.

Can you please confirm Xen also conforms to "CPUID(EAX==1).ECX.bit31 == 1" ? I can update my patch to cover all the hypervisors conforming to this?

See identify_hypervisor() in x86/identcpu.c.

Wouldn't it be less hackish to make the decision in vt_vga.c:vga_init() in kernel, without creating fake kenv var ?

[decui_microsoft.com]

a small coding style change -- use brace for the return value of running_on_hypervisor()

[decui_microsoft.com]

In D5727#122354, @kib wrote:

In D5727#122342, @decui_microsoft.com wrote:

In D5727#122322, @royger wrote:

It's been some time since I've tested this on Xen, but there was a difference between using hw.vga.textmode or not. It's not like 1 line per second, but it used to be noticeable.

Can you please confirm Xen also conforms to "CPUID(EAX==1).ECX.bit31 == 1" ? I can update my patch to cover all the hypervisors conforming to this?

See identify_hypervisor() in x86/identcpu.c.

Wouldn't it be less hackish to make the decision in vt_vga.c:vga_init() in kernel, without creating fake kenv var ?

Yes, that may be less hackish.

Here I'm trying to use this method because I plan to add more Hyper-V specific code into the loader. :-)

Currently the Hyper-V (and probably the other hypervisors too) drivers, i.e., sys/dev/hyperv/, are built in with the kernel binary. We're trying to modulize the drivers by building them as separates .ko files -- when doing this, we need a mechanism to load the drivers and this is what I'm staring to do.

Does this plan make sense to you?

[kib]

In D5727#122358, @decui_microsoft.com wrote:

Wouldn't it be less hackish to make the decision in vt_vga.c:vga_init() in kernel, without creating fake kenv var ?

Yes, that may be less hackish.

Here I'm trying to use this method because I plan to add more Hyper-V specific code into the loader. :-)

Currently the Hyper-V (and probably the other hypervisors too) drivers, i.e., sys/dev/hyperv/, are built in with the kernel binary. We're trying to modulize the drivers by building them as separates .ko files -- when doing this, we need a mechanism to load the drivers and this is what I'm staring to do.

Does this plan make sense to you?

Allowing the hyper-v support be modularized is quite useful and very much welcomed.

If your intent is to detect hyper-v in loader and auto-load hyperv.ko, this is fine, assuming there is a knob to turn it off. This is similar to how acpi.ko was autoloaded on i386 long time ago when loader found ACPI table.

But injecting tunables on behalf of user to adjust the kernel defaults when hyper-v is detected, is IMO too hackish.

[sepherosa_gmail.com]

In D5727#122354, @kib wrote:

In D5727#122342, @decui_microsoft.com wrote:

In D5727#122322, @royger wrote:

It's been some time since I've tested this on Xen, but there was a difference between using hw.vga.textmode or not. It's not like 1 line per second, but it used to be noticeable.

Can you please confirm Xen also conforms to "CPUID(EAX==1).ECX.bit31 == 1" ? I can update my patch to cover all the hypervisors conforming to this?

See identify_hypervisor() in x86/identcpu.c.

Wouldn't it be less hackish to make the decision in vt_vga.c:vga_init() in kernel, without creating fake kenv var ?

Hmm, will it be too late? :) I am not sure whether the vm_guest is set or not there. If it is not set there yet, maybe we can factor out the VM guest detection code give it an MI API and use it in the vga_init().

[kib]

In D5727#122488, @sepherosa_gmail.com wrote:

Hmm, will it be too late? :) I am not sure whether the vm_guest is set or not there. If it is not set there yet, maybe we can factor out the VM guest detection code give it an MI API and use it in the vga_init().

I really hope that it is not. identcpu() is performed very early since some other fundamental parts of the kernel startup, like initial page tables configuration, depend on the identified CPU features. But I did not verified the order for vt_vga.

[decui_microsoft.com]

In D5727#122540, @kib wrote:

In D5727#122488, @sepherosa_gmail.com wrote:

Hmm, will it be too late? :) I am not sure whether the vm_guest is set or not there. If it is not set there yet, maybe we can factor out the VM guest detection code give it an MI API and use it in the vga_init().

I really hope that it is not. identcpu() is performed very early since some other fundamental parts of the kernel startup, like initial page tables configuration, depend on the identified CPU features. But I did not verified the order for vt_vga.

Yes, I believe kib is correct: in hammer_time(), we first call identify_cpu() then cninit(): the former will call identify_hypervisor() and the latter will finally call vga_init().

[decui_microsoft.com]

I made a new patch according to kib's suggestion:
https://reviews.freebsd.org/D5739

This looks not so hackish. Thank you, kib!