gmultipath: make physpath distinct from the underlying providers'

[asomers]

Summary

zfsd uses a device's physical path attribute to automatically replace a
missing ZFS disk when a blank disk is inserted into the same physical
slot. Currently gmultipath passes through its underlying providers'
physical path attribute. That may cause zfsd to replace a missing
gmultipath provider with a newly arrived, single-path disk. That would
be bad.

This commit fixes that problem by simply appending "/mp" to the
underlying providers' physical path, in a manner similar to what geli
already does.

Sponsored by: Axcient

[mav]

I'm sorry for missing it earlier, but if there are actually several paths through the different HBAs and enclosures, what of the paths do you report? Random currently active one? Wouldn't it be better to not pass it through at all, unless we can somehow map all paths into something shared?

[asomers]

In D29941#676861, @mav wrote:

I'm sorry for missing it earlier, but if there are actually several paths through the different HBAs and enclosures, what of the paths do you report? Random currently active one? Wouldn't it be better to not pass it through at all, unless we can somehow map all paths into something shared?

It reports the currently active path's physical path, yes. That works with dual-path SAS backplanes, where both paths by design have the same physical path string. I don't know about Fibrechannel, iSCSI, or FCoE multipathing.

[mav]

Excuse me if I am wrong, I don't have any multipath environment now to make sure, but I remember that SES on different expanders of dual-port backplane report different IDs.

[asomers]

In D29941#676901, @mav wrote:

Excuse me if I am wrong, I don't have any multipath environment now to make sure, but I remember that SES on different expanders of dual-port backplane report different IDs.

Hm. I thought that CAM used the Enclosure Logical Identifier to generate the physpath. It *should* do that. But now that I double-check; you're right. CAM is using the Addressed Logical Unit SAS ID instead. Maybe it was a Spectra-specific patch to use the ELI that never got upstreamed. @ken could you please check how SpectraBSD generates the physical path? You can use commands like

sudo diskinfo -p /dev/da0
sudo sg_ses -p1 /dev/ses0 | grep -i 'enclosure.*logical'
sudo sg_inq -p di /dev/ses0

[ken]

In D29941#676906, @asomers wrote:

In D29941#676901, @mav wrote:

Excuse me if I am wrong, I don't have any multipath environment now to make sure, but I remember that SES on different expanders of dual-port backplane report different IDs.

Hm. I thought that CAM used the Enclosure Logical Identifier to generate the physpath. It *should* do that. But now that I double-check; you're right. CAM is using the Addressed Logical Unit SAS ID instead. Maybe it was a Spectra-specific patch to use the ELI that never got upstreamed. @ken could you please check how SpectraBSD generates the physical path? You can use commands like

sudo diskinfo -p /dev/da0
sudo sg_ses -p1 /dev/ses0 | grep -i 'enclosure.*logical'
sudo sg_inq -p di /dev/ses0

{blackpearl:/usr/home/kenm:!:0} diskinfo -p /dev/da8
enc@5003048001e6057d/elmtype@array_device/slot@14
{blackpearl:/usr/home/kenm:!:0} sg_ses -p1 /dev/enc0 | grep -i 'enclosure.*logical'

enclosure logical identifier (hex): 5003048001e6057f

{blackpearl:/usr/home/kenm:!:0} sg_ses -p1 /dev/enc1 | grep -i 'enclosure.*logical'

enclosure logical identifier (hex): 5003048001e942ff

{blackpearl:/usr/home/kenm:!:0} sg_inq -p di /dev/enc0
VPD INQUIRY: Device Identification page

Designation descriptor number 1, descriptor length: 12
  transport: Serial Attached SCSI Protocol (SPL-4)
  designator_type: NAA,  code_set: Binary
  associated with the Target port
    NAA 5, IEEE Company_id: 0x3048
    Vendor Specific Identifier: 0x1e6057d
    [0x5003048001e6057d]
Designation descriptor number 2, descriptor length: 12
  designator_type: NAA,  code_set: Binary
  associated with the Addressed logical unit
    NAA 5, IEEE Company_id: 0x3048
    Vendor Specific Identifier: 0x1e6057d
    [0x5003048001e6057d]

{blackpearl:/usr/home/kenm:!:0} sg_inq -p di /dev/enc1
VPD INQUIRY: Device Identification page

Designation descriptor number 1, descriptor length: 12
  transport: Serial Attached SCSI Protocol (SPL-4)
  designator_type: NAA,  code_set: Binary
  associated with the Target port
    NAA 5, IEEE Company_id: 0x3048
    Vendor Specific Identifier: 0x1e942fd
    [0x5003048001e942fd]
Designation descriptor number 2, descriptor length: 12
  designator_type: NAA,  code_set: Binary
  associated with the Addressed logical unit
    NAA 5, IEEE Company_id: 0x3048
    Vendor Specific Identifier: 0x1e942fd
    [0x5003048001e942fd]

It is entirely possible that your memory is correct, and I messed it up in a subsequent merge.

If you could refresh my memory about where the physical path is getting set (I haven't been in there in quite a while), I can look back through the history and see if it has changed. I'm also happy to give you guys a copy of the current ses/enc driver. Merging it is a pain, and it would be great if we could upstream the changes.

[asomers]

Yep. That's wrong. In 14.0-CURRENT, the function that sets physpath is ses_set_physpath, in cam/scsi/scsi_enc_ses.c.

[ken]

In D29941#676918, @asomers wrote:

Yep. That's wrong. In 14.0-CURRENT, the function that sets physpath is ses_set_physpath, in cam/scsi/scsi_enc_ses.c.

Ok, it looks like it has been that way (using the device ID from page 0x83) since Justin wrote it in 2013.

I'm still happy to share the code we have, if you guys are going to do changes to SES, it would be nice to get some of our changes upstream. :)