Index: share/man/man4/ntb.4 =================================================================== --- share/man/man4/ntb.4 +++ share/man/man4/ntb.4 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd September 2, 2017 +.Dd August 29, 2019 .Dt NTB 4 .Os .Sh NAME @@ -66,7 +66,7 @@ with all available resources, allowing any driver to attach. .El .Sh DESCRIPTION -Non-Transparent Bridges allow to connect two computer systems with PCIe +Non-Transparent Bridges connect two computer systems with PCIe link(s), providing each of them limited access to others memory space, scratchpad registers and interrupts. The @@ -76,9 +76,10 @@ configuration. .Sh SEE ALSO .Xr if_ntb 4 , -.Xr ntb_transport 4 , +.Xr ntb_hw_amd 4 , .Xr ntb_hw_intel 4 , -.Xr ntb_hw_plx 4 +.Xr ntb_hw_plx 4 , +.Xr ntb_transport 4 .Sh AUTHORS .An -nosplit The Index: share/man/man4/ntb_hw_amd.4 =================================================================== --- share/man/man4/ntb_hw_amd.4 +++ share/man/man4/ntb_hw_amd.4 @@ -1,5 +1,5 @@ .\" -.\" Copyright (c) 2016-2019 Alexander Motin +.\" Copyright (c) 2019-2020 Rajesh Kumar .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,82 +25,70 @@ .\" .\" $FreeBSD$ .\" -.Dd August 15, 2019 -.Dt NTB_TRANSPORT 4 +.Dd August 29, 2019 +.Dt NTB_HW_AMD 4 .Os .Sh NAME -.Nm ntb_transport -.Nd Packet-oriented transport for Non-Transparent Bridges +.Nm ntb_hw_amd +.Nd AMD Non-Transparent Bridge driver .Sh SYNOPSIS To compile this driver into your kernel, place the following lines in your kernel configuration file: .Bd -ragged -offset indent .Cd "device ntb" -.Cd "device ntb_transport" +.Cd "device ntb_hw_amd" .Ed .Pp Or, to load the driver as a module at boot, place the following line in .Xr loader.conf 5 : .Bd -literal -offset indent -ntb_transport_load="YES" +ntb_hw_amd_load="YES" .Ed .Pp -The following tunables are settable from the -.Xr loader 8 : +The following sysctls are supported in this driver .Bl -ohang -.It Va hw.ntb_transport.debug_level -Driver debug level. -The default value is 0, higher means more verbose. -.It Va hw.ntb_transport.max_mw_size -Limits maximum memory window usage. -Allocation of big physically contiguous memory buffer may be a problem, -while too big buffers makes no much sense for low latency network interface. -.It Va hint.ntb_transport. Ns Ar X Ns Va .config -Configures a set of the transport consumers, separated by commas. -Each consumer can be configured as: "[][:]", where: -.Va name -is a name of the driver to attach (empty means any), -.Va queues -is a number of queues to allocate (empty means automatic). -The default configuration is empty string, which means single consumer -with one queue per memory window, allowing any driver to attach. +.It Va dev.ntb_hw.X.info +Reading this sysctl will give the basic details like the number of memory +windows, scratchpads and doorbells exposed by the NTB on the local host to +access the devices beyond the bridge. +It also provides details about the masked doorbells, translation address and +size limit of each exposed memory window and link status information. .El .Sh DESCRIPTION The -.Nm -driver attaches on top of the -.Nm ntb -driver to utilize its resources to create a set of bidirectional queues, -delivering packets between the systems. -The primary purpose of this driver is to be used by -.Nm if_ntb -network interface, but other consumers may also be developed using KPI. +.Nm ntb_hw_amd +driver provides support for the Non-Transparent Bridge (NTB) hardware in +AMD EPYC processor family. +The Non-Transparent Bridge does not look as a regular PCI bridge, but as PCI +endpoint device, hiding the devices behind it. +The driver hides details of hardware on the other side, but exposes memory windows, +scratchpads and doorbells to access the other side via hardware independent +KPI to +.Xr ntb 4 +subsystem. .Pp -Each -.Nm -require from underlying -.Nm ntb -instance: -.Bl -bullet -compact -.It -1 or more memory windows; -.It -6 scratchpads, plus 2 more for each additional memory window; -.It -1 doorbell for each memory window or configured queue. -.El +The hardware provides 2 (two 64-bit) or 3 (one 32-bit and two 64-bit) memory +windows to the other system's memory, up-to 16 scratchpad registers and 16 +doorbells to communicate with and interrupt the other system respectively. +.Sh CONFIGURATION +The NTB configuration should be set by BIOS. +This includes enabling NTB, choosing topology (only NTB-to-Root Port +mode is supported now), role of the host in the topology. +This needs to be done on both systems. +.Pp +The BAR size for memory windows is configured to 1 MiB by default. .Sh SEE ALSO .Xr if_ntb 4 , .Xr ntb 4 , -.Xr ntb_hw_intel 4 , -.Xr ntb_hw_plx 4 +.Xr ntb_transport 4 .Sh AUTHORS .An -nosplit The .Nm -driver was developed by Intel and originally written by -.An Carl Delsey Aq Mt carl@FreeBSD.org . -Later improvements were done by +driver was developed by AMD and originally written by +.An Rajesh Kumar Aq Mt rajesh1.kumar@amd.com . +Reviewed by +.An Alexander Motin Aq Mt mav@FreeBSD.org , .An Conrad E. Meyer Aq Mt cem@FreeBSD.org and -.An Alexander Motin Aq Mt mav@FreeBSD.org . +.An Warner Losh Aq Mt imp@FreeBSD.org . Index: share/man/man4/ntb_transport.4 =================================================================== --- share/man/man4/ntb_transport.4 +++ share/man/man4/ntb_transport.4 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd August 15, 2019 +.Dd August 29, 2019 .Dt NTB_TRANSPORT 4 .Os .Sh NAME @@ -92,6 +92,7 @@ .Sh SEE ALSO .Xr if_ntb 4 , .Xr ntb 4 , +.Xr ntb_hw_amd 4 , .Xr ntb_hw_intel 4 , .Xr ntb_hw_plx 4 .Sh AUTHORS