Page MenuHomeFreeBSD

sys: add hmp
Needs ReviewPublic

Authored by minsoochoo0122_proton.me on Mon, Jan 12, 8:36 PM.
Tags
None
Referenced Files
F142179910: D54674.diff
Fri, Jan 16, 9:32 PM
F142151636: D54674.id169542.diff
Fri, Jan 16, 12:42 PM
F142151629: D54674.id169542.diff
Fri, Jan 16, 12:42 PM
F142095700: D54674.id169730.diff
Fri, Jan 16, 12:22 AM
Unknown Object (File)
Thu, Jan 15, 10:07 PM
Unknown Object (File)
Thu, Jan 15, 9:55 PM
Unknown Object (File)
Thu, Jan 15, 5:59 PM
Unknown Object (File)
Thu, Jan 15, 4:42 PM

Details

Reviewers
andrew
manu
olce
Summary

This adds initial support for HMP.

The proposed framework, hmp(4), sits between scheduler and provider. sched_ule(4) will invoke hmp(4) APIs for thread placement (score) and load balancing (capacity). Providers such as Intel ITD, device trees, and ACPI CPPC will give static or dynamic scores and capacity information to hmp so the scheduler can utilize this information. Providers like Arm SCMI let scheduler to control processor performance on demand.

hmp is designed to be architecture agnoistic, that is, kern/sched_ule.c and kern/subr_hmp shouldn't have any machine-dependent code, and all machine-dependent code happens in sys/dev (dynamic provider using device drivers) and sys/arch (static provider such as device tree). For architectures that doesn't have homogeneous cores like powerpc and riscv, default values will be used for capacity.

The proposed coredirector(4) patch (D44456) is a example of provider, although it doesn't use hmp API yet. Once this patch is adopted, coredirector(4) can utilize this framework to store dynamic scores from Intel Thread Director and scheduler can utilize it.

This is only enabled when HMP option is on. The framework structure is subject to be changed at any time for better scheduler/provider interaction.

References:
sys/contrib/device-tree/Bindings/cpu/cpu-capacity.txt gives a good insight on utilizing device-tree files for static capacity scores.

Next steps:

  • add sysctl for hmp
  • document hmp(4) on man page
  • integrate hmp into sched_ule(4)
  • add providers such as ITD, ACPI, device trees, etc.
Test Plan

Currently no functional change, so only build success is required.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 69836
Build 66719: arc lint + arc unit

Event Timeline

minsoochoo0122_proton.me updated this revision to Diff 169539.

Fix old header guards

minsoochoo0122_proton.me retitled this revision from options: add HMP to sys: add machine-independent cpucap stuff.Mon, Jan 12, 8:40 PM
minsoochoo0122_proton.me edited the summary of this revision. (Show Details)
minsoochoo0122_proton.me retitled this revision from sys: add machine-independent cpucap stuff to sys: add cpucap.Mon, Jan 12, 8:44 PM

No need to initialize score. If there is no dynamic score, capacity will be used. If there is dynamic score, it will be filled by machine dependent initializer.

Harbormaster completed remote builds in B69833: Diff 169539.
Harbormaster completed remote builds in B69834: Diff 169540.
Harbormaster completed remote builds in B69836: Diff 169542.
Harbormaster completed remote builds in B69837: Diff 169543.
Harbormaster completed remote builds in B69839: Diff 169545.
Harbormaster completed remote builds in B69840: Diff 169546.
sys/kern/subr_hmp.c
175

I suggest adding a note here about the computational complexity of stuff.

Some systems have hundreds of cores, not 8, so all the functions you have here that iterate over a mask or allcpus is going to vary wildly in its runtime overhead.

i suggest adding some documentation to each of the functions here saying what they do, what their overhead can be, and whether they're expected to be called once (eg during init); relatively infrequently (eg every time the system onlines/offlines a CPU), frequently (eg every context switch.)

sys/sys/hmp.h
72

how many of these should be bools instead of ints?

91

Would you mind writing some documentation for each of these functions?

eg i figure cpucap_has_capacity() returns true if the system is initialised and cpus have different capacities, but does that capacity_varies change over time? if so, in what situations would it change?

Under options HMP, cpucap is always enabled. Thus initialized field doesn't have anyeffect. capacity is always initialized; if it isn't provided, default value will be used. Thus ccapacity_varies and related functions aren't needed. The only thing left is has_scores, and this can be queried by accessing the field without using accessors.

Clarified types and added documentation by @adrien 's request.

sys/sys/hmp.h
72

I agree that bool is ideal but I was considering memory alignment as it is accessed frequently by scheduler. Or is it over-optimization?

91

capacity is static, initialized once on boot time. Scores are dynamic and updated from hardware feedback.

I'm removing feature checks and initialized and has_capacity fields because cpucap is always enabled under options HMP and capacity has default value if it is not provided by hardware.

sys/kern/subr_hmp.c
17

Is this needed? opt_global.h should automatically be included

Remove unused functions, match types, and mark unimplented functions as todos.

Default implementation of cpucap_md_init should initialize capacity with CPUCAP_SCORE_DEFAULT

I'm renaming cpucap to hmp. I know this is an undesirable action during review process, but I found that that the name cpucap can have dual meaning (capability and capacity) and thus confuse developers. I decided to follow smp's convention to namespace hmp stuff by prefixing hmp_ in names.

minsoochoo0122_proton.me retitled this revision from sys: add cpucap to sys: add hmp.Wed, Jan 14, 8:14 PM
minsoochoo0122_proton.me edited the summary of this revision. (Show Details)

Utilize hmp_set_capacity for default initialization