Paths

Table of Contentst

powernv: Include NUMA locality information in the CPU topology
ClosedPublic
Actions

Authored by markj on Feb 10 2021, 9:11 PM.

Details

Reviewers

jhibbits
bdragon
luporl

Commits

rG6f19077eaf50: powernv: Include NUMA locality information in the CPU topology
rGcb5f8694a522: powernv: Include NUMA locality information in the CPU topology

Summary

This helps ULE make locality-preserving scheduling decisions. To make
use of the generic subr_smp.c topology-building routes, I have to assume
that all NUMA domains are the same size and that the set of CPUs
belonging to any domain is contiguous.

Test Plan

I don't have a NUMA POWER system to test on. Would anyone be willing to try it and show output from the kern.sched.topology_spec sysctl?

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

markj created this revision.Feb 10 2021, 9:11 PM

Herald added subscribers: jhibbits, imp. · View Herald TranscriptFeb 10 2021, 9:11 PM

markj requested review of this revision.Feb 10 2021, 9:11 PM

Harbormaster completed remote builds in B36877: Diff 83664.Feb 10 2021, 9:11 PM

markj added a parent revision: D28579: arm64: Include NUMA locality info in the CPU topology.Feb 10 2021, 9:11 PM

markj edited the test plan for this revision. (Show Details)Feb 10 2021, 9:13 PM

markj added reviewers: jhibbits, bdragon, luporl.

Will try this on Talos II.

bdragon changed the repository for this revision from rS FreeBSD src repository - subversion to rG FreeBSD src repository.Feb 10 2021, 9:47 PM

bdragon added a project: PowerPC.

Herald added a subscriber: alfredo. · View Herald TranscriptFeb 10 2021, 9:47 PM

On a dual socket POWER9:

<groups>
 <group level="1" cache-level="0">
  <cpu count="144" mask="ffffffffffffffff,ffffffffffffffff,ffff,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143</cpu>
  <children>
   <group level="2" cache-level="3">
    <cpu count="72" mask="ffffffffffffffff,ff,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f,0,0,0">0, 1, 2, 3</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0,0,0,0">4, 5, 6, 7</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00,0,0,0">8, 9, 10, 11</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000,0,0,0">12, 13, 14, 15</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000,0,0,0">16, 17, 18, 19</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000,0,0,0">20, 21, 22, 23</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000,0,0,0">24, 25, 26, 27</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000,0,0,0">28, 29, 30, 31</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000,0,0,0">32, 33, 34, 35</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000,0,0,0">36, 37, 38, 39</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000000,0,0,0">40, 41, 42, 43</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000000,0,0,0">44, 45, 46, 47</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000000,0,0,0">48, 49, 50, 51</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000000000,0,0,0">52, 53, 54, 55</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000000000,0,0,0">56, 57, 58, 59</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000000000,0,0,0">60, 61, 62, 63</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f,0,0">64, 65, 66, 67</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0,0,0">68, 69, 70, 71</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
    </children>
   </group>
   <group level="2" cache-level="3">
    <cpu count="72" mask="0,ffffffffffffff00,ffff,0">72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00,0,0">72, 73, 74, 75</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000,0,0">76, 77, 78, 79</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000,0,0">80, 81, 82, 83</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000,0,0">84, 85, 86, 87</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000,0,0">88, 89, 90, 91</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000,0,0">92, 93, 94, 95</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000,0,0">96, 97, 98, 99</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000,0,0">100, 101, 102, 103</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000000,0,0">104, 105, 106, 107</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000000,0,0">108, 109, 110, 111</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000000,0,0">112, 113, 114, 115</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000000000,0,0">116, 117, 118, 119</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000000000,0,0">120, 121, 122, 123</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000000000,0,0">124, 125, 126, 127</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f,0">128, 129, 130, 131</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f0,0">132, 133, 134, 135</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f00,0">136, 137, 138, 139</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f000,0">140, 141, 142, 143</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
    </children>
   </group>
  </children>
 </group>
</groups>

This revision is now accepted and ready to land.Feb 10 2021, 11:14 PM

Since jhibbits tried for Talos II already, I will try on TYAN POWER8 instead.

On a deca-core POWER8 (SMT-8):

<groups>
 <group level="1" cache-level="0">
  <cpu count="80" mask="ffffffffffffffff,ffff,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79</cpu>
  <children>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff00,0,0,0">8, 9, 10, 11, 12, 13, 14, 15</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff0000,0,0,0">16, 17, 18, 19, 20, 21, 22, 23</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff000000,0,0,0">24, 25, 26, 27, 28, 29, 30, 31</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff00000000,0,0,0">32, 33, 34, 35, 36, 37, 38, 39</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff0000000000,0,0,0">40, 41, 42, 43, 44, 45, 46, 47</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff000000000000,0,0,0">48, 49, 50, 51, 52, 53, 54, 55</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="ff00000000000000,0,0,0">56, 57, 58, 59, 60, 61, 62, 63</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="0,ff,0,0">64, 65, 66, 67, 68, 69, 70, 71</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="1">
    <cpu count="8" mask="0,ff00,0,0">72, 73, 74, 75, 76, 77, 78, 79</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
  </children>
 </group>
</groups>

One possible problem with your assumptions is that arbitray cores may be GUARDed out for a given boot. So some cores may exist but be offline due to being deconfig'd during IPL. I'm not sure whether that would have an effect on this or not. The IPL logic will always deconfig the entire core, and therefore the entire thread/smt group when there is a fault entry. So the domains might have differing numbers of online cores.

In D28580#639940, @bdragon wrote:

One possible problem with your assumptions is that arbitray cores may be GUARDed out for a given boot. So some cores may exist but be offline due to being deconfig'd during IPL. I'm not sure whether that would have an effect on this or not. The IPL logic will always deconfig the entire core, and therefore the entire thread/smt group when there is a fault entry. So the domains might have differing numbers of online cores.

When is this done in practice?

I'm not a huge fan of assuming that all domains have an identical number of cores, this assumption is just needed to use the somewhat simplistic smp_topo_*() functions. I'll revisit this and see if I can handle asymmetric domains without having to complicate the code too much.

In D28580#639939, @bdragon wrote:

On a deca-core POWER8 (SMT-8):

Presumably this is not a NUMA system? Or should we be seeing an additional layer in the topology?

In D28580#640084, @markj wrote:

In D28580#639940, @bdragon wrote:

One possible problem with your assumptions is that arbitray cores may be GUARDed out for a given boot. So some cores may exist but be offline due to being deconfig'd during IPL. I'm not sure whether that would have an effect on this or not. The IPL logic will always deconfig the entire core, and therefore the entire thread/smt group when there is a fault entry. So the domains might have differing numbers of online cores.

When is this done in practice?

I'm not a huge fan of assuming that all domains have an identical number of cores, this assumption is just needed to use the somewhat simplistic smp_topo_*() functions. I'll revisit this and see if I can handle asymmetric domains without having to complicate the code too much.

The most common way for this to happen is on POWER9 DD2.2, where driver bugs can cause an erroneous checksum fault in the load/store unit when trying to mix a CI mapping with a non-CI mapping for the same physical page. The RAS recovery code assumes that it was a problem with the CPU hardware and takes the core that the driver had been running on out of service until the fault is manually cleared.

We currently warn on boot when the system is in that state but allow the boot to continue.

In D28580#640085, @markj wrote:

In D28580#639939, @bdragon wrote:

On a deca-core POWER8 (SMT-8):

Presumably this is not a NUMA system? Or should we be seeing an additional layer in the topology?

It's a single-ram-domain system, yeah. (although the ram is in a fanout hierarchy behind the Centaur buffer chips, I think it still counts as uniform access.) I mainly posted it because I wanted to verify that the 8-way multithreading was being represented correctly.

Stop assuming that domain masks are contiguous and symmetric.

We still assume that adjacent CPU IDs correspond to sibling threads.

This revision now requires review to proceed.Feb 25 2021, 4:12 PM

Harbormaster completed remote builds in B37370: Diff 84679.Feb 25 2021, 4:13 PM

I did a full test run with the updated version of this patch in (among others) and it worked fine.

Here's my output again.

If you need me to test with a noncontig set, I can probably crash my machine on purpose and guard a core out. (I'd rather not if it's not needed though)

Another thing about the guard behavior -- it works a core at a time, so all 4 threads (for POWER9) will get removed at the same time.

Also, this has a dependency on bd94c8ab29 -- without that commit, powernv was not properly propagating the associativity information for the second domain.

kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="144" mask="ffffffffffffffff,ffffffffffffffff,ffff,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143</cpu>
  <children>
   <group level="2" cache-level="3">
    <cpu count="72" mask="ffffffffffffffff,ff,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f,0,0,0">0, 1, 2, 3</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0,0,0,0">4, 5, 6, 7</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00,0,0,0">8, 9, 10, 11</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000,0,0,0">12, 13, 14, 15</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000,0,0,0">16, 17, 18, 19</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000,0,0,0">20, 21, 22, 23</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000,0,0,0">24, 25, 26, 27</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000,0,0,0">28, 29, 30, 31</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000,0,0,0">32, 33, 34, 35</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000,0,0,0">36, 37, 38, 39</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000000,0,0,0">40, 41, 42, 43</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000000,0,0,0">44, 45, 46, 47</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000000,0,0,0">48, 49, 50, 51</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f0000000000000,0,0,0">52, 53, 54, 55</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f00000000000000,0,0,0">56, 57, 58, 59</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="f000000000000000,0,0,0">60, 61, 62, 63</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f,0,0">64, 65, 66, 67</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0,0,0">68, 69, 70, 71</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
    </children>
   </group>
   <group level="2" cache-level="3">
    <cpu count="72" mask="0,ffffffffffffff00,ffff,0">72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00,0,0">72, 73, 74, 75</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000,0,0">76, 77, 78, 79</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000,0,0">80, 81, 82, 83</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000,0,0">84, 85, 86, 87</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000,0,0">88, 89, 90, 91</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000,0,0">92, 93, 94, 95</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000,0,0">96, 97, 98, 99</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000,0,0">100, 101, 102, 103</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000000,0,0">104, 105, 106, 107</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000000,0,0">108, 109, 110, 111</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000000,0,0">112, 113, 114, 115</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f0000000000000,0,0">116, 117, 118, 119</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f00000000000000,0,0">120, 121, 122, 123</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,f000000000000000,0,0">124, 125, 126, 127</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f,0">128, 129, 130, 131</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f0,0">132, 133, 134, 135</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f00,0">136, 137, 138, 139</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="4" mask="0,0,f000,0">140, 141, 142, 143</cpu>
      <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
     </group>
    </children>
   </group>
  </children>
 </group>
</groups>

This revision is now accepted and ready to land.Mar 31 2021, 4:22 AM

Thanks for re-testing. I had tested it on my blackbird but it has only one socket populated.

Closed by commit rGcb5f8694a522: powernv: Include NUMA locality information in the CPU topology (authored by markj). · Explain WhyApr 1 2021, 2:32 PM

This revision was automatically updated to reflect the committed changes.

markj added a commit: rGcb5f8694a522: powernv: Include NUMA locality information in the CPU topology.

markj added a commit: rG6f19077eaf50: powernv: Include NUMA locality information in the CPU topology.Apr 15 2021, 1:48 PM