Changeset View
Changeset View
Standalone View
Standalone View
head/lib/geom/multipath/gmultipath.8
Property | Old Value | New Value |
---|---|---|
svn:keywords | null | FreeBSD=%H \ No newline at end of property |
.\" Copyright (c) 2007 Matthew Jacob | |||||
.\" All rights reserved. | |||||
.\" | |||||
.\" Redistribution and use in source and binary forms, with or without | |||||
.\" modification, are permitted provided that the following conditions | |||||
.\" are met: | |||||
.\" 1. Redistributions of source code must retain the above copyright | |||||
.\" notice, this list of conditions and the following disclaimer. | |||||
.\" 2. Redistributions in binary form must reproduce the above copyright | |||||
.\" notice, this list of conditions and the following disclaimer in the | |||||
.\" documentation and/or other materials provided with the distribution. | |||||
.\" | |||||
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND | |||||
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE | |||||
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE | |||||
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE | |||||
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | |||||
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS | |||||
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) | |||||
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | |||||
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY | |||||
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF | |||||
.\" SUCH DAMAGE. | |||||
.\" | |||||
.\" $FreeBSD$ | |||||
.\" | |||||
.Dd September 8, 2016 | |||||
.Dt GMULTIPATH 8 | |||||
.Os | |||||
.Sh NAME | |||||
.Nm gmultipath | |||||
.Nd "disk multipath control utility" | |||||
.Sh SYNOPSIS | |||||
.Nm | |||||
.Cm create | |||||
.Op Fl ARv | |||||
.Ar name | |||||
.Ar prov ... | |||||
.Nm | |||||
.Cm label | |||||
.Op Fl ARv | |||||
.Ar name | |||||
.Ar prov ... | |||||
.Nm | |||||
.Cm configure | |||||
.Op Fl APRv | |||||
.Ar name | |||||
.Nm | |||||
.Cm add | |||||
.Op Fl v | |||||
.Ar name prov | |||||
.Nm | |||||
.Cm remove | |||||
.Op Fl v | |||||
.Ar name prov | |||||
.Nm | |||||
.Cm fail | |||||
.Op Fl v | |||||
.Ar name prov | |||||
.Nm | |||||
.Cm restore | |||||
.Op Fl v | |||||
.Ar name prov | |||||
.Nm | |||||
.Cm rotate | |||||
.Op Fl v | |||||
.Ar name | |||||
.Nm | |||||
.Cm prefer | |||||
.Op Fl v | |||||
.Ar name | |||||
.Ar prov | |||||
.Nm | |||||
.Cm getactive | |||||
.Op Fl v | |||||
.Ar name | |||||
.Nm | |||||
.Cm destroy | |||||
.Op Fl v | |||||
.Ar name | |||||
.Nm | |||||
.Cm stop | |||||
.Op Fl v | |||||
.Ar name | |||||
.Nm | |||||
.Cm clear | |||||
.Op Fl v | |||||
.Ar prov ... | |||||
.Nm | |||||
.Cm list | |||||
.Nm | |||||
.Cm status | |||||
.Nm | |||||
.Cm load | |||||
.Nm | |||||
.Cm unload | |||||
.Sh DESCRIPTION | |||||
The | |||||
.Nm | |||||
utility is used for device multipath configuration. | |||||
.Pp | |||||
The multipath device can be configured using two different methods: | |||||
.Dq manual | |||||
or | |||||
.Dq automatic . | |||||
When using the | |||||
.Dq manual | |||||
method, no metadata are stored on the devices, so the multipath | |||||
device has to be configured by hand every time it is needed. | |||||
Additional device paths also will not be detected automatically. | |||||
The | |||||
.Dq automatic | |||||
method uses on-disk metadata to detect device and all its paths. | |||||
Metadata use the last sector of the underlying disk device and | |||||
include device name and UUID. | |||||
The UUID guarantees uniqueness in a shared storage environment | |||||
but is in general too cumbersome to use. | |||||
The name is what is exported via the device interface. | |||||
.Pp | |||||
The first argument to | |||||
.Nm | |||||
indicates an action to be performed: | |||||
.Bl -tag -width ".Cm destroy" | |||||
.It Cm create | |||||
Create multipath device with | |||||
.Dq manual | |||||
method without writing any on-disk metadata. | |||||
It is up to administrator, how to properly identify device paths. | |||||
Kernel will only check that all given providers have same media and | |||||
sector sizes. | |||||
.Pp | |||||
.Fl A | |||||
option enables Active/Active mode, | |||||
.Fl R | |||||
option enables Active/Read mode, otherwise Active/Passive mode is used | |||||
by default. | |||||
.It Cm label | |||||
Create multipath device with | |||||
.Dq automatic | |||||
method. | |||||
Label the first given provider with on-disk metadata using the specified | |||||
.Ar name . | |||||
The rest of given providers will be retasted to detect these metadata. | |||||
It reliably protects against specifying unrelated providers. | |||||
Providers with no matching metadata detected will not be added to the device. | |||||
.Pp | |||||
.Fl A | |||||
option enables Active/Active mode, | |||||
.Fl R | |||||
option enables Active/Read mode, otherwise Active/Passive mode is used | |||||
by default. | |||||
.It Cm configure | |||||
Configure the given multipath device. | |||||
.Pp | |||||
.Fl A | |||||
option enables Active/Active mode, | |||||
.Fl P | |||||
option enables Active/Passive mode, | |||||
.Fl R | |||||
option enables Active/Read mode. | |||||
.It Cm add | |||||
Add the given provider as a path to the given multipath device. | |||||
Should normally be used only for devices created with | |||||
.Dq manual | |||||
method, unless you know what you are doing (you are sure that it is another | |||||
device path, but tasting its metadata in regular | |||||
.Dq automatic | |||||
way is not possible). | |||||
.It Cm remove | |||||
Remove the given provider as a path from the given multipath device. | |||||
If the last path removed, the multipath device will be destroyed. | |||||
.It Cm fail | |||||
Mark specified provider as a path of the specified multipath device as failed. | |||||
If there are other paths present, new requests will be forwarded there. | |||||
.It Cm restore | |||||
Mark specified provider as a path of the specified multipath device as | |||||
operational, allowing it to handle requests. | |||||
.It Cm rotate | |||||
Change the active provider/path to the next available provider in Active/Passive mode. | |||||
.It Cm prefer | |||||
Change the active provider/path to the specified provider in Active/Passive mode. | |||||
.It Cm getactive | |||||
Get the currently active provider(s)/path(s). | |||||
.It Cm destroy | |||||
Destroy the given multipath device clearing metadata. | |||||
.It Cm stop | |||||
Stop the given multipath device without clearing metadata. | |||||
.It Cm clear | |||||
Clear metadata on the given provider. | |||||
.It Cm list | |||||
See | |||||
.Xr geom 8 . | |||||
.It Cm status | |||||
See | |||||
.Xr geom 8 . | |||||
.It Cm load | |||||
See | |||||
.Xr geom 8 . | |||||
.It Cm unload | |||||
See | |||||
.Xr geom 8 . | |||||
.El | |||||
.Sh SYSCTL VARIABLES | |||||
The following | |||||
.Xr sysctl 8 | |||||
variable can be used to control the behavior of the | |||||
.Nm MULTIPATH | |||||
GEOM class. | |||||
.Bl -tag -width indent | |||||
.It Va kern.geom.multipath.debug : No 0 | |||||
Debug level of the | |||||
.Nm MULTIPATH | |||||
GEOM class. | |||||
This can be set to 0 (default) or 1 to disable or enable various | |||||
forms of chattiness. | |||||
.It Va kern.geom.multipath.exclusive : No 1 | |||||
Open underlying providers exclusively, preventing individual paths access. | |||||
.El | |||||
.Sh EXIT STATUS | |||||
Exit status is 0 on success, and 1 if the command fails. | |||||
.Sh MULTIPATH ARCHITECTURE | |||||
This is a multiple path architecture with no device knowledge or | |||||
presumptions other than size matching built in. | |||||
Therefore the user must exercise some care | |||||
in selecting providers that do indeed represent multiple paths to the | |||||
same underlying disk device. | |||||
The reason for this is that there are several | |||||
criteria across multiple underlying transport types that can | |||||
.Ar indicate | |||||
identity, but in all respects such identity can rarely be considered | |||||
.Ar definitive . | |||||
.Pp | |||||
For example, if you use the World Word Port Name of a Fibre Channel | |||||
disk object you might believe that two disks that have the same WWPN | |||||
on different paths (or even disjoint fabrics) might be considered | |||||
the same disk. | |||||
Nearly always this would be a safe assumption, until | |||||
you realize that a WWPN, like an Ethernet MAC address, is a soft | |||||
programmable entity, and that a misconfigured Director Class switch | |||||
could lead you to believe incorrectly that you have found multiple | |||||
paths to the same device. | |||||
This is an extreme and theoretical case, but | |||||
it is possible enough to indicate that the policy for deciding which | |||||
of multiple pathnames refer to the same device should be left to the | |||||
system operator who will use tools and knowledge of their own storage | |||||
subsystem to make the correct configuration selection. | |||||
.Pp | |||||
There are Active/Passive, Active/Read and Active/Active operation modes | |||||
supported. | |||||
In Active/Passive mode only one path has I/O moving on it | |||||
at any point in time. | |||||
This I/O continues until an I/O is returned with | |||||
a generic I/O error or a "Nonexistent Device" error. | |||||
When this occurs, that path is marked FAIL, the next path | |||||
in a list is selected as active and the failed I/O reissued. | |||||
In Active/Active mode all paths not marked FAIL may handle I/O at the same time. | |||||
Requests are distributed between paths to equalize load. | |||||
For capable devices it allows the utilisation of the bandwidth available on all paths. | |||||
In Active/Read mode all paths not marked FAIL may handle reads at the same time, | |||||
but unlike in Active/Active mode only one path handles write requests at any | |||||
point in time; closely following the original write request order if the layer | |||||
above needs it for data consistency (not waiting for requisite write completion | |||||
before sending dependent write). | |||||
.Pp | |||||
When new devices are added to the system the | |||||
.Nm MULTIPATH | |||||
GEOM class is given an opportunity to taste these new devices. | |||||
If a new | |||||
device has a | |||||
.Nm MULTIPATH | |||||
on-disk metadata label, the device is either used to create a new | |||||
.Nm MULTIPATH | |||||
GEOM, or added to the list of paths for an existing | |||||
.Nm MULTIPATH | |||||
GEOM. | |||||
.Pp | |||||
It is this mechanism that works reasonably with | |||||
.Xr isp 4 | |||||
and | |||||
.Xr mpt 4 | |||||
based Fibre Channel disk devices. | |||||
For these devices, when a device disappears | |||||
(due to e.g., a cable pull or power failure to a switch), the device is | |||||
proactively marked as gone and I/O to it failed. | |||||
This causes the | |||||
.Nm MULTIPATH | |||||
failure event just described. | |||||
.Pp | |||||
When Fibre Channel events inform either | |||||
.Xr isp 4 | |||||
or | |||||
.Xr mpt 4 | |||||
host bus adapters that new devices may have arrived (e.g., the arrival | |||||
of an RSCN event from the Fabric Domain Controller), they can cause | |||||
a rescan to occur and cause the attachment and configuration of any | |||||
(now) new devices to occur, causing the taste event described above. | |||||
.Pp | |||||
This means that this multipath architecture is not a one-shot path | |||||
failover, but can be considered to be steady state as long as failed | |||||
paths are repaired (automatically or otherwise). | |||||
.Pp | |||||
Automatic rescanning is not a requirement. | |||||
Nor is Fibre Channel. | |||||
The | |||||
same failover mechanisms work equally well for traditional "Parallel" | |||||
SCSI but may require manual intervention with | |||||
.Xr camcontrol 8 | |||||
to cause the reattachment of repaired device links. | |||||
.Sh EXAMPLES | |||||
The following example shows how to use | |||||
.Xr camcontrol 8 | |||||
to find possible multiple path devices and to create a | |||||
.Nm MULTIPATH | |||||
GEOM class for them. | |||||
.Bd -literal -offset indent | |||||
mysys# camcontrol devlist | |||||
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 0 (da0,pass0) | |||||
<ECNCTX @WESTVILLE > at scbus0 target 0 lun 1 (da1,pass1) | |||||
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 0 (da2,pass2) | |||||
<ECNCTX @WESTVILLE > at scbus1 target 0 lun 1 (da3,pass3) | |||||
mysys# camcontrol inquiry da0 -S | |||||
ECNTX0LUN000000SER10ac0d01 | |||||
mysys# camcontrol inquiry da2 -S | |||||
ECNTX0LUN000000SER10ac0d01 | |||||
.Ed | |||||
.Pp | |||||
Now that you have used the Serial Number to compare two disk paths | |||||
it is not entirely unreasonable to conclude that these are multiple | |||||
paths to the same device. | |||||
However, only the user who is familiar | |||||
with their storage is qualified to make this judgement. | |||||
.Pp | |||||
You can then use the | |||||
.Nm | |||||
command to label and create a | |||||
.Nm MULTIPATH | |||||
GEOM provider named | |||||
.Ar FRED . | |||||
.Bd -literal -offset indent | |||||
gmultipath label -v FRED /dev/da0 /dev/da2 | |||||
disklabel -Brw /dev/multipath/FRED auto | |||||
newfs /dev/multipath/FREDa | |||||
mount /dev/multipath/FREDa /mnt.... | |||||
.Ed | |||||
.Pp | |||||
The resultant console output looks something like: | |||||
.Bd -literal -offset indent | |||||
GEOM_MULTIPATH: da0 added to FRED | |||||
GEOM_MULTIPATH: da0 is now active path in FRED | |||||
GEOM_MULTIPATH: da2 added to FRED | |||||
.Ed | |||||
.Pp | |||||
To load the | |||||
.Nm | |||||
module at boot time, add this entry to | |||||
.Pa /boot/loader.conf : | |||||
.Bd -literal -offset ident | |||||
geom_multipath_load="YES" | |||||
.Ed | |||||
.Sh SEE ALSO | |||||
.Xr geom 4 , | |||||
.Xr isp 4 , | |||||
.Xr mpt 4 , | |||||
.Xr loader.conf 5 , | |||||
.Xr camcontrol 8 , | |||||
.Xr geom 8 , | |||||
.Xr mount 8 , | |||||
.Xr newfs 8 , | |||||
.Xr sysctl 8 | |||||
.Sh HISTORY | |||||
The | |||||
.Nm | |||||
utility first appeared in | |||||
.Fx 7.0 | |||||
.Sh AUTHORS | |||||
.An Matthew Jacob Aq Mt mjacob@FreeBSD.org | |||||
.An Alexander Motin Aq Mt mav@FreeBSD.org |