Fix mpr device mapper fallback logic for busses with multiple expanders.

[asomers]

Summary

Fix mpr device mapper fallback logic for busses with multiple expanders.

r308217 added fallback logic for mpr's device mapper, to be used in case the
card's nonvolatile mapping table is corrupt. The fallback logic simply uses a
device's phy number as its SCSI target id. But this doesn't work on SAS busses
with multiple expanders, because phy numbers are only unique per-expander.

Change the fallback logic to use a combination of the Enclosure Handle and
Slot.

Test Plan

Tested on several systems with corrupt drive mapping tables, each with a bus
that has two enclosures and 36 drives, plus another bus with one enclosure,
three expanders, and 97 drives.

[asomers]

This is the change I mentioned at lunch on Wednesday. Would this fail for your setup? Is there something that could make it work for both of us? Perhaps different behavior based on whether use_phynum is 1 or 2?

[asomers]

Forgot to hit save

[slm]

sys/dev/mpr/mpr_sas_lsi.c

846–850

'fallible'

863

Does this work for directly attached devices too, with no enclosures?

1295

Do you need to account for NVMe?

[asomers]

sys/dev/mpr/mpr_sas_lsi.c

863

Probably not.

1295

I don't know. Do I? I don't have any tri-mode hardware.

[slm]

sys/dev/mpr/mpr_sas_lsi.c

863

It might work, but I guess it should be tested to be sure. I think the EnclosureHandle will be 1 for direct-attached devices. Can you explain why you're subtracting 1 here?

1295

I'm thinking we should handle them the same way.

[asomers]

sys/dev/mpr/mpr_sas_lsi.c

863

I'm subtracting one just to make the best use out of my 1024 available IDs. The first enclosure always has enclosure handle 1, not 0. The problem with directly-attached devices is that their Slot is probably 0. But I don't have any to test with.

1295

Do I need to make any changes for them?

[slm]

sys/dev/mpr/mpr_sas_lsi.c

863

I was just concerned about an EnclosureHandle of 0, but that's an invalid value that shouldn't be returned and it's probably OK. The way it works is that the EnclosureHandle will be 1 for direct-attached devices and it represents a virtual Enclosure to the FW. I believe the Slot or Phy number should be valid, even for direct-attached devices. It's probably OK to leave it as is, and it will work. You can't attach any drives directly to test?

1295

You should be able to make the exact same changes here just to cover both device types. If you don't there might be confusion later if things are working for SAS but not NVMe.

[asomers]

I no longer have access to any hardware that can exercise this change. @slm do you want to commandeer the revision? Otherwise I'll abandon it.

sys/dev/mpr/mpr_sas_lsi.c

863

I don't think I have the right type of cables for direct attaching a drive. Everything I do uses expanders.

[asomers]

Abandoning the revision because I no longer have relevant hardware to develop on. @slm can commandeer the revision if he wants to.