Index: stable/11/share/man/man4/mlx5io.4 =================================================================== --- stable/11/share/man/man4/mlx5io.4 (revision 347868) +++ stable/11/share/man/man4/mlx5io.4 (revision 347869) @@ -1,163 +1,168 @@ .\" .\" Copyright (c) 2018, 2019 Mellanox Technologies .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd May 7, 2019 .Dt mlx5io 4 .Os .Sh NAME .Nm mlx5io .Nd IOCTL interface to manage Connect-X 4/5 Mellanox network adapters .Sh SYNOPSIS .In dev/mlx5/mlx5io.h .Sh DESCRIPTION The .Nm interface is provided for management of the Connect-X 4 and 5 network adapters in the aspects not covered by the generic network configuration, mostly related to the PCIe attachment and internal card working. Interface consists of the commands, which are passed by means of .Xr ioctl 2 on the file descriptor, opened from the .Pa /dev/mlx5ctl device node. .Pp The following commands are implemented: .Bl -tag -width indent .It Dv MLX5_FWDUMP_FORCE Take the snapshot of the firmware registers state and store it in the kernel buffer. The buffer must be empty, in other words, no dumps should be written so far, or existing dump cleared with the .Dv MLX5_FWDUMP_RESET command for the specified device. The argument for the command should point to the .Vt struct mlx5_tool_addr structure, containing the PCIe bus address of the device. .Bd -literal struct mlx5_tool_addr { uint32_t domain; uint8_t bus; uint8_t slot; uint8_t func; }; .Ed .It Dv MLX5_FWDUMP_RESET Clear the stored firmware dump, preparing the kernel buffer for the next dump. The argument for the command should point to the .Vt struct mlx5_tool_addr structure, containing the PCIe bus address of the device. .It Dv MLX5_FWDUMP_GET Fetch the stored firmware dump into the user memory. The argument to the command should point to the input/output .Vt struct mlx5_fwdump_get structure. Its .Dv devaddr field specifies the address of the device, the .Dv buf fields points to the array of .Vt struct mlx5_fwdump_reg of records of the registers values, the size of the array is specified in the .Dv reg_cnt field. .Bd -literal struct mlx5_fwdump_get { struct mlx5_tool_addr devaddr; struct mlx5_fwdump_reg *buf; size_t reg_cnt; size_t reg_filled; /* out */ }; .Ed .Pp On successfull return, the .Dv reg_filled field reports the number of the .Dv buf array elements actually filled with the registers values. If .Dv buf contains the .Dv NULL pointer, no registers are filled, but .Dv reg_filled still contains the number of registers that should be passed for the complete dump. .Pp The .Vt struct mlx5_fwdump_reg element contains the address of the register in the field .Dv addr , and its value in the field .Dv val . .Bd -literal struct mlx5_fwdump_reg { uint32_t addr; uint32_t val; }; .Ed .It Dv MLX5_FW_UPDATE Requests firmware update (flash) on the adapter specified by the .Dv devaddr using the firmware image in .Dv MFA2 format. The argument for the ioctl command is the .Vt struct mlx5_fw_update with the following definition. .Bd -literal struct mlx5_fw_update { struct mlx5_tool_addr devaddr; void *img_fw_data; size_t img_fw_data_len; }; .Ed Image address in memory is passed in .Dv img_fw_data , the length of the image is specified in .Dv img_fw_data_len field. +.It Dv MLX5_FW_RESET +Requests PCIe link-level reset on the device. +The address of the device is specified by the +.Vt struct mlx5_tool_addr +structure, which should be passed as an argument. .El .Sh FILES The .Pa /dev/mlx5ctl .Xr devfs 5 node is used to pass commands to the driver. .Sh RETURN VALUES If successful, the IOCTL returns zero. Otherwise, -1 is returned and the global variable .Va errno is set to indicate the error. .Sh SEE ALSO .Xr errno 2 , .Xr ioctl 2 , .Xr mlx5en 4 , .Xr mlx5ib 4 , .Xr mlx5tool 8 and .Xr pci 9 . Index: stable/11/sys/dev/mlx5/mlx5_core/mlx5_fwdump.c =================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_fwdump.c (revision 347868) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_fwdump.c (revision 347869) @@ -1,347 +1,376 @@ /*- * Copyright (c) 2018, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include extern const struct mlx5_crspace_regmap mlx5_crspace_regmap_mt4117[]; extern const struct mlx5_crspace_regmap mlx5_crspace_regmap_mt4115[]; extern const struct mlx5_crspace_regmap mlx5_crspace_regmap_connectx5[]; struct mlx5_dump_data { const struct mlx5_crspace_regmap *rege; uint32_t *dump; unsigned dump_size; int dump_valid; struct mtx dump_lock; }; static MALLOC_DEFINE(M_MLX5_DUMP, "MLX5DUMP", "MLX5 Firmware dump"); static unsigned mlx5_fwdump_getsize(const struct mlx5_crspace_regmap *rege) { const struct mlx5_crspace_regmap *r; unsigned sz; for (sz = 0, r = rege; r->cnt != 0; r++) sz += r->cnt; return (sz); } static void mlx5_fwdump_destroy_dd(struct mlx5_dump_data *dd) { mtx_destroy(&dd->dump_lock); free(dd->dump, M_MLX5_DUMP); free(dd, M_MLX5_DUMP); } void mlx5_fwdump_prep(struct mlx5_core_dev *mdev) { struct mlx5_dump_data *dd; int error; error = mlx5_vsc_find_cap(mdev); if (error != 0) { /* Inability to create a firmware dump is not fatal. */ device_printf((&mdev->pdev->dev)->bsddev, "WARN: " "mlx5_fwdump_prep failed %d\n", error); return; } dd = malloc(sizeof(struct mlx5_dump_data), M_MLX5_DUMP, M_WAITOK); switch (pci_get_device(mdev->pdev->dev.bsddev)) { case 0x1013: dd->rege = mlx5_crspace_regmap_mt4115; break; case 0x1015: dd->rege = mlx5_crspace_regmap_mt4117; break; case 0x1017: case 0x1019: dd->rege = mlx5_crspace_regmap_connectx5; break; default: free(dd, M_MLX5_DUMP); return; /* silently fail, do not prevent driver attach */ } dd->dump_size = mlx5_fwdump_getsize(dd->rege); dd->dump = malloc(dd->dump_size * sizeof(uint32_t), M_MLX5_DUMP, M_WAITOK | M_ZERO); dd->dump_valid = 0; mtx_init(&dd->dump_lock, "mlx5dmp", NULL, MTX_DEF | MTX_NEW); if (atomic_cmpset_rel_ptr((uintptr_t *)&mdev->dump_data, 0, (uintptr_t)dd) == 0) mlx5_fwdump_destroy_dd(dd); } void mlx5_fwdump(struct mlx5_core_dev *mdev) { struct mlx5_dump_data *dd; const struct mlx5_crspace_regmap *r; uint32_t i, ri; int error; dev_info(&mdev->pdev->dev, "Issuing FW dump\n"); dd = (struct mlx5_dump_data *)atomic_load_acq_ptr((uintptr_t *) &mdev->dump_data); if (dd == NULL) return; mtx_lock(&dd->dump_lock); if (dd->dump_valid) { /* only one dump */ dev_warn(&mdev->pdev->dev, "Only one FW dump can be captured aborting FW dump\n"); goto failed; } /* mlx5_vsc already warns, be silent. */ error = mlx5_vsc_lock(mdev); if (error != 0) goto failed; error = mlx5_vsc_set_space(mdev, MLX5_VSC_DOMAIN_PROTECTED_CRSPACE); if (error != 0) goto unlock_vsc; for (i = 0, r = dd->rege; r->cnt != 0; r++) { for (ri = 0; ri < r->cnt; ri++) { error = mlx5_vsc_read(mdev, r->addr + ri * 4, &dd->dump[i]); if (error != 0) goto unlock_vsc; i++; } } atomic_store_rel_int(&dd->dump_valid, 1); unlock_vsc: mlx5_vsc_unlock(mdev); failed: mtx_unlock(&dd->dump_lock); } void mlx5_fwdump_clean(struct mlx5_core_dev *mdev) { struct mlx5_dump_data *dd; for (;;) { dd = mdev->dump_data; if (dd == NULL) return; if (atomic_cmpset_ptr((uintptr_t *)&mdev->dump_data, (uintptr_t)dd, 0) == 1) { mlx5_fwdump_destroy_dd(dd); return; } } } static int mlx5_dbsf_to_core(const struct mlx5_tool_addr *devaddr, struct mlx5_core_dev **mdev) { device_t dev; struct pci_dev *pdev; dev = pci_find_dbsf(devaddr->domain, devaddr->bus, devaddr->slot, devaddr->func); if (dev == NULL) return (ENOENT); if (device_get_devclass(dev) != mlx5_core_driver.bsdclass) return (EINVAL); pdev = device_get_softc(dev); *mdev = pci_get_drvdata(pdev); if (*mdev == NULL) return (ENOENT); return (0); } static int mlx5_fwdump_copyout(struct mlx5_dump_data *dd, struct mlx5_fwdump_get *fwg) { const struct mlx5_crspace_regmap *r; struct mlx5_fwdump_reg rv, *urv; uint32_t i, ri; int error; if (dd == NULL) return (ENOENT); if (fwg->buf == NULL) { fwg->reg_filled = dd->dump_size; return (0); } if (atomic_load_acq_int(&dd->dump_valid) == 0) return (ENOENT); urv = fwg->buf; for (i = 0, r = dd->rege; r->cnt != 0; r++) { for (ri = 0; ri < r->cnt; ri++) { if (i >= fwg->reg_cnt) goto out; rv.addr = r->addr + ri * 4; rv.val = dd->dump[i]; error = copyout(&rv, urv, sizeof(rv)); if (error != 0) return (error); urv++; i++; } } out: fwg->reg_filled = i; return (0); } static int +mlx5_fw_reset(struct mlx5_core_dev *mdev) +{ + device_t dev, bus; + int error; + + error = -mlx5_set_mfrl_reg(mdev, MLX5_FRL_LEVEL3); + if (error == 0) { + dev = mdev->pdev->dev.bsddev; + mtx_lock(&Giant); + bus = device_get_parent(dev); + error = BUS_RESET_CHILD(device_get_parent(bus), bus, + DEVF_RESET_DETACH); + mtx_unlock(&Giant); + } + return (error); +} + +static int mlx5_fwdump_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, struct thread *td) { struct mlx5_core_dev *mdev; struct mlx5_fwdump_get *fwg; struct mlx5_tool_addr *devaddr; struct mlx5_dump_data *dd; struct mlx5_fw_update *fu; struct firmware fake_fw; int error; error = 0; switch (cmd) { case MLX5_FWDUMP_GET: if ((fflag & FREAD) == 0) { error = EBADF; break; } fwg = (struct mlx5_fwdump_get *)data; devaddr = &fwg->devaddr; error = mlx5_dbsf_to_core(devaddr, &mdev); if (error != 0) break; error = mlx5_fwdump_copyout(mdev->dump_data, fwg); break; case MLX5_FWDUMP_RESET: if ((fflag & FWRITE) == 0) { error = EBADF; break; } devaddr = (struct mlx5_tool_addr *)data; error = mlx5_dbsf_to_core(devaddr, &mdev); if (error != 0) break; dd = mdev->dump_data; if (dd != NULL) atomic_store_rel_int(&dd->dump_valid, 0); else error = ENOENT; break; case MLX5_FWDUMP_FORCE: if ((fflag & FWRITE) == 0) { error = EBADF; break; } devaddr = (struct mlx5_tool_addr *)data; error = mlx5_dbsf_to_core(devaddr, &mdev); if (error != 0) break; mlx5_fwdump(mdev); break; case MLX5_FW_UPDATE: if ((fflag & FWRITE) == 0) { error = EBADF; break; } fu = (struct mlx5_fw_update *)data; if (fu->img_fw_data_len > 10 * 1024 * 1024) { error = EINVAL; break; } devaddr = &fu->devaddr; error = mlx5_dbsf_to_core(devaddr, &mdev); if (error != 0) break; bzero(&fake_fw, sizeof(fake_fw)); fake_fw.name = "umlx_fw_up"; fake_fw.datasize = fu->img_fw_data_len; fake_fw.version = 1; fake_fw.data = (void *)kmem_malloc(kmem_arena, fu->img_fw_data_len, M_WAITOK); if (fake_fw.data == NULL) { error = ENOMEM; break; } error = copyin(fu->img_fw_data, __DECONST(void *, fake_fw.data), fu->img_fw_data_len); if (error == 0) error = -mlx5_firmware_flash(mdev, &fake_fw); kmem_free(kmem_arena, (vm_offset_t)fake_fw.data, fu->img_fw_data_len); + break; + case MLX5_FW_RESET: + if ((fflag & FWRITE) == 0) { + error = EBADF; + break; + } + devaddr = (struct mlx5_tool_addr *)data; + error = mlx5_dbsf_to_core(devaddr, &mdev); + if (error != 0) + break; + error = mlx5_fw_reset(mdev); break; default: error = ENOTTY; break; } return (error); } static struct cdevsw mlx5_fwdump_devsw = { .d_version = D_VERSION, .d_ioctl = mlx5_fwdump_ioctl, }; static struct cdev *mlx5_fwdump_dev; int mlx5_fwdump_init(void) { struct make_dev_args mda; int error; make_dev_args_init(&mda); mda.mda_flags = MAKEDEV_WAITOK | MAKEDEV_CHECKNAME; mda.mda_devsw = &mlx5_fwdump_devsw; mda.mda_uid = UID_ROOT; mda.mda_gid = GID_OPERATOR; mda.mda_mode = 0640; error = make_dev_s(&mda, &mlx5_fwdump_dev, "mlx5ctl"); return (-error); } void mlx5_fwdump_fini(void) { if (mlx5_fwdump_dev != NULL) destroy_dev(mlx5_fwdump_dev); } Index: stable/11/sys/dev/mlx5/mlx5io.h =================================================================== --- stable/11/sys/dev/mlx5/mlx5io.h (revision 347868) +++ stable/11/sys/dev/mlx5/mlx5io.h (revision 347869) @@ -1,150 +1,151 @@ /*- * Copyright (c) 2018, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _DEV_MLX5_MLX5IO_H_ #define _DEV_MLX5_MLX5IO_H_ #include struct mlx5_fwdump_reg { uint32_t addr; uint32_t val; }; struct mlx5_tool_addr { uint32_t domain; uint8_t bus; uint8_t slot; uint8_t func; }; struct mlx5_fwdump_get { struct mlx5_tool_addr devaddr; struct mlx5_fwdump_reg *buf; size_t reg_cnt; size_t reg_filled; /* out */ }; struct mlx5_fw_update { struct mlx5_tool_addr devaddr; void *img_fw_data; size_t img_fw_data_len; }; #define MLX5_FWDUMP_GET _IOWR('m', 1, struct mlx5_fwdump_get) #define MLX5_FWDUMP_RESET _IOW('m', 2, struct mlx5_tool_addr) #define MLX5_FWDUMP_FORCE _IOW('m', 3, struct mlx5_tool_addr) #define MLX5_FW_UPDATE _IOW('m', 4, struct mlx5_fw_update) +#define MLX5_FW_RESET _IOW('m', 5, struct mlx5_tool_addr) #ifndef _KERNEL #define MLX5_DEV_PATH _PATH_DEV"mlx5ctl" #endif enum mlx5_fpga_id { MLX5_FPGA_NEWTON = 0, MLX5_FPGA_EDISON = 1, MLX5_FPGA_MORSE = 2, MLX5_FPGA_MORSEQ = 3, }; enum mlx5_fpga_image { MLX5_FPGA_IMAGE_USER = 0, MLX5_FPGA_IMAGE_FACTORY = 1, MLX5_FPGA_IMAGE_FACTORY_FAILOVER = 2, MLX5_FPGA_IMAGE_RESET = 17, MLX5_FPGA_IMAGE_RELOAD = 18, }; enum mlx5_fpga_status { MLX5_FPGA_STATUS_SUCCESS = 0, MLX5_FPGA_STATUS_FAILURE = 1, MLX5_FPGA_STATUS_IN_PROGRESS = 2, MLX5_FPGA_STATUS_DISCONNECTED = 3, }; struct mlx5_fpga_query { enum mlx5_fpga_image admin_image; enum mlx5_fpga_image oper_image; enum mlx5_fpga_status image_status; }; enum mlx5_fpga_tee { MLX5_FPGA_TEE_DISABLE = 0, MLX5_FPGA_TEE_GENERATE_EVENT = 1, MLX5_FPGA_TEE_GENERATE_SINGLE_EVENT = 2, }; enum mlx5_fpga_connect { MLX5_FPGA_CONNECT_QUERY = 0, MLX5_FPGA_CONNECT_DISCONNECT = 0x9, MLX5_FPGA_CONNECT_CONNECT = 0xA, }; /** * enum mlx5_fpga_access_type - Enumerated the different methods possible for * accessing the device memory address space */ enum mlx5_fpga_access_type { /** Use the slow CX-FPGA I2C bus*/ MLX5_FPGA_ACCESS_TYPE_I2C = 0x0, /** Use the fast 'shell QP' */ MLX5_FPGA_ACCESS_TYPE_RDMA, /** Use the fastest available method */ MLX5_FPGA_ACCESS_TYPE_DONTCARE, MLX5_FPGA_ACCESS_TYPE_MAX = MLX5_FPGA_ACCESS_TYPE_DONTCARE, }; #define MLX5_FPGA_INTERNAL_SENSORS_LOW 63 #define MLX5_FPGA_INTERNAL_SENSORS_HIGH 63 struct mlx5_fpga_temperature { uint32_t temperature; uint32_t index; uint32_t tee; uint32_t max_temperature; uint32_t temperature_threshold_hi; uint32_t temperature_threshold_lo; uint32_t mte; uint32_t mtr; char sensor_name[16]; }; #define MLX5_FPGA_CAP_ARR_SZ 0x40 #define MLX5_FPGA_ACCESS_TYPE _IOWINT('m', 0x80) #define MLX5_FPGA_LOAD _IOWINT('m', 0x81) #define MLX5_FPGA_RESET _IO('m', 0x82) #define MLX5_FPGA_IMAGE_SEL _IOWINT('m', 0x83) #define MLX5_FPGA_QUERY _IOR('m', 0x84, struct mlx5_fpga_query) #define MLX5_FPGA_CAP _IOR('m', 0x85, uint32_t[MLX5_FPGA_CAP_ARR_SZ]) #define MLX5_FPGA_TEMPERATURE _IOWR('m', 0x86, struct mlx5_fpga_temperature) #define MLX5_FPGA_CONNECT _IOWR('m', 0x87, enum mlx5_fpga_connect) #define MLX5_FPGA_RELOAD _IO('m', 0x88) #define MLX5_FPGA_TOOLS_NAME_SUFFIX "_mlx5_fpga_tools" #endif Index: stable/11/usr.sbin/mlx5tool/mlx5tool.8 =================================================================== --- stable/11/usr.sbin/mlx5tool/mlx5tool.8 (revision 347868) +++ stable/11/usr.sbin/mlx5tool/mlx5tool.8 (revision 347869) @@ -1,106 +1,117 @@ .\" .\" Copyright (c) 2018, 2019 Mellanox Technologies .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd May 7, 2019 .Dt mlx5tool 8 .Os .Sh NAME .Nm mlx5tool .Nd Utility for managing Connect-X 4/5 Mellanox network adapters .Sh SYNOPSIS .Nm .Fl d Ar domain:bus:slot:func .Fl e .Nm .Fl d Ar domain:bus:slot:func .Fl rn .Nm .Fl d Ar domain:bus:slot:func .Fl o Ar file .Fl w .Nm .Fl d Ar domain:bus:slot:func .Fl f Ar file.mfa2 +.Nm +.Fl d Ar domain:bus:slot:func +.Fl z .Sh DESCRIPTION The .Nm utility is provided for management of the Connect-X 4 and 5 network adapters in the aspects not covered by the generic .Xr ifconfig 8 command, mostly related to the PCIe attachment and internal card working. The utility executes commands on specific adapter, which is addressed using .Em device:bus:slot:function conventions of the PCIe buses. You can match adapters ethernet name and addresses using the .X pciconf 8 utility. The address is passed as an argument of the .Fl d option, which must be specified for each invocation. .Pp When the driver detects the malfunctioning of the hardware, or by user request, it is possible to create .Em firmware dump , which contains debugging information about internal device state, for analysis of the failure by the Mellanox support team. .Pp The following commands are currently implemented: .Bl -tag -width indent .It Fl e Take the snapshot of the firmware registers state and store it in the kernel buffer. The buffer must be empty, in other words, no dumps should be written so far, or existing dump cleared with the .Fl r command for the specified device. .It Fl r Clear the stored firmware dump, preparing the kernel buffer for the next dump. .It Fl w Fetches the stored firmware dump and writes it into the file specified by the .Fl o option argument. .It Fl f Flashes the firmware image .Fa file.mfa2 to the specified adapter. Image must be in MFA2 pack format and contain a component suitable for the adapter hardware. +.Pp +Typically, PCIe link-level reset is required to activate the +newly flashed image, which can be performed by the system reboot +or using the +.Fl z +option. +.It Fl z +Performs PCIe link-level reset on the specified device. .El .Sh FILES The .Pa /dev/mlx5ctl .Xr devfs 5 node is used to pass commands to the driver. .Sh SEE ALSO .Xr mlx5en 4 , .Xr mlx5ib 4 , .Xr mlx5io 4 , .Xr ifconfig 8 and .Xr pciconf 8 . Index: stable/11/usr.sbin/mlx5tool/mlx5tool.c =================================================================== --- stable/11/usr.sbin/mlx5tool/mlx5tool.c (revision 347868) +++ stable/11/usr.sbin/mlx5tool/mlx5tool.c (revision 347869) @@ -1,283 +1,302 @@ /*- * Copyright (c) 2018, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* stolen from pciconf.c: parsesel() */ static int parse_pci_addr(const char *addrstr, struct mlx5_tool_addr *addr) { char *eppos; unsigned long selarr[4]; int i; if (addrstr == NULL) { warnx("no pci address specified"); return (1); } if (strncmp(addrstr, "pci", 3) == 0) { addrstr += 3; i = 0; while (isdigit(*addrstr) && i < 4) { selarr[i++] = strtoul(addrstr, &eppos, 10); addrstr = eppos; if (*addrstr == ':') addrstr++; } if (i > 0 && *addrstr == '\0') { addr->func = (i > 2) ? selarr[--i] : 0; addr->slot = (i > 0) ? selarr[--i] : 0; addr->bus = (i > 0) ? selarr[--i] : 0; addr->domain = (i > 0) ? selarr[--i] : 0; return (0); } } warnx("invalid pci address %s", addrstr); return (1); } static int mlx5tool_save_dump(int ctldev, const struct mlx5_tool_addr *addr, const char *dumpname) { struct mlx5_fwdump_get fdg; struct mlx5_fwdump_reg *rege; FILE *dump; size_t cnt; int error, res; if (dumpname == NULL) dump = stdout; else dump = fopen(dumpname, "w"); if (dump == NULL) { warn("open %s", dumpname); return (1); } res = 1; memset(&fdg, 0, sizeof(fdg)); fdg.devaddr = *addr; error = ioctl(ctldev, MLX5_FWDUMP_GET, &fdg); if (error != 0) { warn("MLX5_FWDUMP_GET dumpsize"); goto out; } rege = calloc(fdg.reg_filled, sizeof(*rege)); if (rege == NULL) { warn("alloc rege"); goto out; } fdg.buf = rege; fdg.reg_cnt = fdg.reg_filled; error = ioctl(ctldev, MLX5_FWDUMP_GET, &fdg); if (error != 0) { if (errno == ENOENT) warnx("no dump recorded"); else warn("MLX5_FWDUMP_GET dump fetch"); goto out; } for (cnt = 0; cnt < fdg.reg_cnt; cnt++, rege++) fprintf(dump, "0x%08x\t0x%08x\n", rege->addr, rege->val); res = 0; out: if (dump != stdout) fclose(dump); return (res); } static int mlx5tool_dump_reset(int ctldev, const struct mlx5_tool_addr *addr) { if (ioctl(ctldev, MLX5_FWDUMP_RESET, addr) == -1) { warn("MLX5_FWDUMP_RESET"); return (1); } return (0); } static int mlx5tool_dump_force(int ctldev, const struct mlx5_tool_addr *addr) { if (ioctl(ctldev, MLX5_FWDUMP_FORCE, addr) == -1) { warn("MLX5_FWDUMP_FORCE"); return (1); } return (0); } static int mlx5tool_fw_update(int ctldev, const struct mlx5_tool_addr *addr, const char *img_fw_path) { struct stat st; struct mlx5_fw_update fwup; int error, fd, res; res = 0; fd = open(img_fw_path, O_RDONLY); if (fd == -1) { warn("Unable to open %s", img_fw_path); res = 1; goto close_fd; } error = fstat(fd, &st); if (error != 0) { warn("Unable to stat %s", img_fw_path); res = 1; goto close_fd; } memset(&fwup, 0, sizeof(fwup)); memcpy(&fwup.devaddr, addr, sizeof(fwup.devaddr)); fwup.img_fw_data = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0); if (fwup.img_fw_data == MAP_FAILED) { warn("Unable to mmap %s", img_fw_path); res = 1; goto close_fd; } fwup.img_fw_data_len = st.st_size; error = ioctl(ctldev, MLX5_FW_UPDATE, &fwup); if (error == -1) { warn("MLX5_FW_UPDATE"); } munmap(fwup.img_fw_data, st.st_size); close_fd: close(fd); return (res); } +static int +mlx5tool_fw_reset(int ctldev, const struct mlx5_tool_addr *addr) +{ + + if (ioctl(ctldev, MLX5_FW_RESET, addr) == -1) { + warn("MLX5_FW_RESET"); + return (1); + } + return (0); +} + static void usage(void) { fprintf(stderr, "Usage: mlx5tool -d pci [-w -o dump.file | -r |" - " -e | -f fw.mfa2]\n"); + " -e | -f fw.mfa2 | -z]\n"); fprintf(stderr, "\t-w - write firmware dump to the specified file\n"); fprintf(stderr, "\t-r - reset dump\n"); fprintf(stderr, "\t-e - force dump\n"); fprintf(stderr, "\t-f fw.img - flash firmware from fw.img\n"); + fprintf(stderr, "\t-z - initiate firmware reset\n"); exit(1); } enum mlx5_action { ACTION_DUMP_GET, ACTION_DUMP_RESET, ACTION_DUMP_FORCE, ACTION_FW_UPDATE, + ACTION_FW_RESET, ACTION_NONE, }; int main(int argc, char *argv[]) { struct mlx5_tool_addr addr; char *dumpname; char *addrstr; char *img_fw_path; int c, ctldev, res; enum mlx5_action act; act = ACTION_NONE; addrstr = NULL; dumpname = NULL; img_fw_path = NULL; - while ((c = getopt(argc, argv, "d:ef:ho:rw")) != -1) { + while ((c = getopt(argc, argv, "d:ef:ho:rwz")) != -1) { switch (c) { case 'd': addrstr = optarg; break; case 'w': act = ACTION_DUMP_GET; break; case 'e': act = ACTION_DUMP_FORCE; break; case 'o': dumpname = optarg; break; case 'r': act = ACTION_DUMP_RESET; break; case 'f': act = ACTION_FW_UPDATE; img_fw_path = optarg; break; + case 'z': + act = ACTION_FW_RESET; + break; case 'h': default: usage(); } } if (act == ACTION_NONE || (dumpname != NULL && act != ACTION_DUMP_GET) || (img_fw_path != NULL && act != ACTION_FW_UPDATE)) usage(); if (parse_pci_addr(addrstr, &addr) != 0) exit(1); ctldev = open(MLX5_DEV_PATH, O_RDWR); if (ctldev == -1) err(1, "open "MLX5_DEV_PATH); switch (act) { case ACTION_DUMP_GET: res = mlx5tool_save_dump(ctldev, &addr, dumpname); break; case ACTION_DUMP_RESET: res = mlx5tool_dump_reset(ctldev, &addr); break; case ACTION_DUMP_FORCE: res = mlx5tool_dump_force(ctldev, &addr); break; case ACTION_FW_UPDATE: res = mlx5tool_fw_update(ctldev, &addr, img_fw_path); + break; + case ACTION_FW_RESET: + res = mlx5tool_fw_reset(ctldev, &addr); break; default: res = 0; break; } close(ctldev); exit(res); } Index: stable/11 =================================================================== --- stable/11 (revision 347868) +++ stable/11 (revision 347869) Property changes on: stable/11 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r347313