Bug 294513 - hw.ncpu loader(8) tunable for x86 architectures

[jim.chen.1827_gmail.com]

Summary

Most architectures, excluding i386 and amd64, allow you to set artificial limits on the number of logical processors available to FreeBSD through the hw.ncpu variable in loader(8).

This patch increases consistency with other architectures when handling mp_ncpus and mp_maxid through two changes:

1. Allowing cpu_mp_setmaxid() to read the loader(8) tunable and set mp_ncpus and mp_maxid accordingly
2. Preventing mp_start() from changing mp_ncpus through the assign_cpu_ids() method after mp_setmaxid() has already done so

[kib]

sys/x86/x86/mp_x86.c

727

So where is mp_ncpus is assigned to after the patch, when the hw.ncpu tunable is not set?

1023

If this comment is right, then setting mp_ncpus below is only makes the kernel CPU structures inconsistent.

[jim.chen.1827_gmail.com]

sys/x86/x86/mp_x86.c

727

I think mp_ncpus is set earlier in the boot process in mptable_probe_cpus, through the callback mptable_probe_cpus_handler. Looking through topo_probe, I see that the same variables set in mptable_probe_cpus and mptable_setup_local are in-use while generating topo_root.

Since the TOPO_FOREACH loop skips over the nodes in the topology unless they are logical cpus, I believe that incrementing mp_ncpus in this loop yields the same value as was acquired originally. Since this also has the undesirable consequence of overriding the value set by cpu_mp_setmaxid, I chose not to modify it here.

Have I made any mistakes in my reasoning?

1023

I think the comment may be incorrect since cpu_add has no mention of mp_ncpus or mp_maxid. Instead, both are set earlier in boot while iterating over the MP tables.

Since cpu_mp_setmaxid is called after APIC enumeration but before assigning logical cpu ids, I thought this was a reasonable place to set it. During cpu id assignment, the cpus, which will be correctly recorded in the topology, will simply be disabled if mp_ncpus have already been added. I believe this mimics the behavior of other architectures for which this tunable is already available.

I thought that the kernel CPU structures would remain consistent since boot otherwise proceeds normally, except that the remainder of the cpus are explicitly disabled. Is this not correct?

[kib]

By consistency I mean that blindly ignoring some CPU IDs might result in some devices not working. For instance, there might be interrupts hard-coded to be routed through specific APICs, which would break.

We already have the knob to disable SMP, and knobs to disable specific LAPICs. I am not sure that blindly adding the cap on the OS-assigned CPUIDs it good idea, at least on x86. Although I probably think that the knob is fine as far as people who use it know the consequences. Might be a wording in smp(4) should be added to explain why the knob is there but broken.