Index: head/share/man/man4/cc.4 =================================================================== --- head/share/man/man4/cc.4 (revision 220559) +++ head/share/man/man4/cc.4 (revision 220560) @@ -1,116 +1,117 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes and -.\" Lawrence Stewart under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes and Lawrence Stewart under sponsorship from the FreeBSD +.\" Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt cc 4 .Os .Sh NAME .Nm cc .Nd Modular congestion control .Sh DESCRIPTION The modular congestion control framework allows the TCP implementation to dynamically change the congestion control algorithm used by new and existing connections. Algorithms are identified by a unique .Xr ascii 7 name. Algorithm modules can be compiled into the kernel or loaded as kernel modules using the .Xr kld 4 facility. .Pp The default algorithm is NewReno, and all connections use the default unless explicitly overridden using the TCP_CONGESTION socket option (see .Xr tcp 4 for details). The default can be changed using a .Xr sysctl 3 MIB variable detailed in the .Sx MIB Variables section below. .Sh MIB Variables The framework exposes the following variables in the .Va net.inet.tcp.cc branch of the .Xr sysctl 3 MIB: .Bl -tag -width ".Va available" .It Va available Read-only list of currently available congestion control algorithms by name. .It Va algorithm Returns the current default congestion control algorithm when read, and changes the default when set. When attempting to change the default algorithm, this variable should be set to one of the names listed by the .Va net.inet.tcp.cc.available MIB variable. .El .Sh SEE ALSO .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr tcp 4 , .Xr cc 9 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm modular congestion control framework first appeared in .Fx 9.0 . .Pp The framework was first released in 2007 by James Healy and Lawrence Stewart -whilst working on the NewTCP research project at Swinburne University's Centre -for Advanced Internet Architectures, Melbourne, Australia, which was made -possible in part by a grant from the Cisco University Research Program Fund at -Community Foundation Silicon Valley. +whilst working on the NewTCP research project at Swinburne University of +Technology's Centre for Advanced Internet Architectures, Melbourne, Australia, +which was made possible in part by a grant from the Cisco University Research +Program Fund at Community Foundation Silicon Valley. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm facility was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org , .An James Healy Aq jimmy@deefa.com and .An David Hayes Aq david.hayes@ieee.org . .Pp This manual page was written by .An David Hayes Aq david.hayes@ieee.org and .An Lawrence Stewart Aq lstewart@FreeBSD.org . Index: head/share/man/man4/cc_chd.4 =================================================================== --- head/share/man/man4/cc_chd.4 (revision 220559) +++ head/share/man/man4/cc_chd.4 (revision 220560) @@ -1,128 +1,128 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes -.\" under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_CHD 4 .Os .Sh NAME .Nm cc_chd .Nd CHD Congestion Control Algorithm .Sh DESCRIPTION CHD enhances the HD algorithm implemented in .Xr cc_hd 4 . It provides tolerance to non-congestion related packet loss and improvements to coexistence with traditional loss-based TCP flows, especially when the bottleneck link is lightly multiplexed. .Pp Like HD, the algorithm aims to keep network queuing delays below a particular threshold (queue_threshold) and decides to reduce the congestion window (cwnd) probabilistically based on its estimate of the network queuing delay. .Pp It differs from HD in three key aspects: .Bl -bullet .It The probability of cwnd reduction due to congestion is calculated once per round trip time instead of each time an acknowledgement is received as done by .Xr cc_hd 4 . .It Packet losses that occur while the queuing delay is less than queue_threshold do not cause cwnd to be reduced. .It CHD uses a shadow window to help regain lost transmission opportunities when competing with loss-based TCP flows. .El .Sh MIB Variables The algorithm exposes the following tunable variables in the .Va net.inet.tcp.cc.chd branch of the .Xr sysctl 3 MIB: .Bl -tag -width ".Va queue_threshold" .It Va queue_threshold Queueing congestion threshold (qth) in ticks. Default is 20. .It Va pmax Per RTT maximum backoff probability as a percentage. Default is 50. .It Va qmin Minimum queuing delay threshold (qmin) in ticks. Default is 5. .It Va loss_fair If 1, cwnd is adjusted using the shadow window when a congestion related loss is detected. Default is 1. .It Va use_max If 1, the maximum RTT seen within the measurement period is used as the basic delay measurement for the algorithm, otherwise a sampled RTT measurement is used. Default is 1. .El .Sh SEE ALSO .Xr cc 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr h_ertt 4 , .Xr tcp 4 , .Xr cc 9 , .Xr khelp 9 .Rs .%A "D. A. Hayes" .%A "G. Armitage" .%T "Improved coexistence and loss tolerance for delay based TCP congestion control" .%J "in 35th Annual IEEE Conference on Local Computer Networks" .%D "October 2010" .%P "24-31" .Re .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm congestion control module first appeared in .Fx 9.0 . .Pp The module was first released in 2010 by David Hayes whilst working on the -NewTCP research project at Swinburne University's Centre for Advanced Internet -Architectures, Melbourne, Australia. +NewTCP research project at Swinburne University of Technology's Centre for +Advanced Internet Architectures, Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module and this manual page were written by .An David Hayes Aq david.hayes@ieee.org . Index: head/share/man/man4/cc_cubic.4 =================================================================== --- head/share/man/man4/cc_cubic.4 (revision 220559) +++ head/share/man/man4/cc_cubic.4 (revision 220560) @@ -1,114 +1,114 @@ .\" .\" Copyright (c) 2009 Lawrence Stewart .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" Portions of this documentation were written at the Centre for Advanced -.\" Internet Architectures, Swinburne University, Melbourne, Australia by -.\" David Hayes under sponsorship from the FreeBSD Foundation. +.\" Internet Architectures, Swinburne University of Technology, Melbourne, +.\" Australia by David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_CUBIC 4 .Os .Sh NAME .Nm cc_cubic .Nd CUBIC Congestion Control Algorithm .Sh DESCRIPTION The CUBIC congestion control algorithm was designed to provide increased throughput in fast and long-distance networks. It attempts to maintain fairness when competing with legacy NewReno TCP in lower speed scenarios where NewReno is able to operate adequately. .Pp The congestion window is increased as a function of the time elapsed since the last congestion event. During regular operation, the window increase function follows a cubic function, with the inflection point set to be the congestion window value reached at the last congestion event. CUBIC also calculates an estimate of the congestion window that NewReno would have achieved at a given time after a congestion event. When updating the congestion window, the algorithm will choose the larger of the calculated CUBIC and estimated NewReno windows. .Pp CUBIC also backs off less on congestion by changing the multiplicative decrease factor from 1/2 (used by standard NewReno TCP) to 4/5. .Pp The implementation was done in a clean-room fashion, and is based on the Internet Draft and paper referenced in the .Sx SEE ALSO section below. .Sh MIB Variables There are currently no tunable MIB variables. .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr tcp 4 , .Xr cc 9 .Rs .%A "Sangtae Ha" .%A "Injong Rhee" .%A "Lisong Xu" .%T "CUBIC for Fast Long-Distance Networks" .%U "http://tools.ietf.org/id/draft-rhee-tcpm-cubic-02.txt" .Re .Rs .%A "Sangtae Ha" .%A "Injong Rhee" .%A "Lisong Xu" .%T "CUBIC: a new TCP-friendly high-speed TCP variant" .%J "SIGOPS Oper. Syst. Rev." .%V "42" .%N "5" .%D "July 2008" .%P "64-74" .Re .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm congestion control module first appeared in .Fx 9.0 . .Pp The module was first released in 2009 by Lawrence Stewart whilst studying at -Swinburne University's Centre for Advanced Internet Architectures, Melbourne, -Australia. +Swinburne University of Technology's Centre for Advanced Internet Architectures, +Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module and this manual page were written by .An Lawrence Stewart Aq lstewart@FreeBSD.org and .An David Hayes Aq david.hayes@ieee.org . Index: head/share/man/man4/cc_hd.4 =================================================================== --- head/share/man/man4/cc_hd.4 (revision 220559) +++ head/share/man/man4/cc_hd.4 (revision 220560) @@ -1,120 +1,120 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes -.\" under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_HD 4 .Os .Sh NAME .Nm cc_hd .Nd HD Congestion Control Algorithm .Sh DESCRIPTION The HD congestion control algorithm is an implementation of the Hamilton Institute's delay-based congestion control which aims to keep network queuing delays below a particular threshold (queue_threshold). .Pp HD probabilistically reduces the congestion window (cwnd) based on its estimate of the network queuing delay. The probability of reducing cwnd is zero at hd_qmin or less, rising to a maximum at queue_threshold, and then back to zero at the maximum queuing delay. .Pp Loss-based congestion control algorithms such as NewReno probe for network capacity by filling queues until there is a packet loss. HD competes with loss-based congestion control algorithms by allowing its probability of reducing cwnd to drop from a maximum at queue_threshold to be zero at the maximum queuing delay. This has been shown to work well when the bottleneck link is highly multiplexed. .Sh MIB Variables The algorithm exposes the following tunable variables in the .Va net.inet.tcp.cc.hd branch of the .Xr sysctl 3 MIB: .Bl -tag -width ".Va queue_threshold" .It Va queue_threshold Queueing congestion threshold (qth) in ticks. Default is 20. .It Va pmax Per packet maximum backoff probability as a percentage. Default is 5. .It Va qmin Minimum queuing delay threshold (qmin) in ticks. Default is 5. .El .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr h_ertt 4 , .Xr tcp 4 , .Xr cc 9 , .Xr khelp 9 .Rs .%A "L. Budzisz" .%A "R. Stanojevic" .%A "R. Shorten" .%A "F. Baker" .%T "A strategy for fair coexistence of loss and delay-based congestion control algorithms" .%J "IEEE Commun. Lett." .%D "Jul 2009" .%V "13" .%N "7" .%P "555-557" .Re .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh FUTURE WORK The Hamilton Institute have recently made some improvements to the algorithm implemented by this module and have called it Coexistent-TCP (C-TCP). The improvments should be evaluated and potentially incorporated into this module. .Sh HISTORY The .Nm congestion control module first appeared in .Fx 9.0 . .Pp The module was first released in 2010 by David Hayes whilst working on the -NewTCP research project at Swinburne University's Centre for Advanced Internet -Architectures, Melbourne, Australia. +NewTCP research project at Swinburne University of Technology's Centre for +Advanced Internet Architectures, Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module and this manual page were written by .An David Hayes Aq david.hayes@ieee.org . Index: head/share/man/man4/cc_htcp.4 =================================================================== --- head/share/man/man4/cc_htcp.4 (revision 220559) +++ head/share/man/man4/cc_htcp.4 (revision 220560) @@ -1,136 +1,136 @@ .\" .\" Copyright (c) 2008 Lawrence Stewart .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" Portions of this documentation were written at the Centre for Advanced -.\" Internet Architectures, Swinburne University, Melbourne, Australia by -.\" David Hayes under sponsorship from the FreeBSD Foundation. +.\" Internet Architectures, Swinburne University of Technology, Melbourne, +.\" Australia by David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_HTCP 4 .Os .Sh NAME .Nm cc_htcp .Nd H-TCP Congestion Control Algorithm .Sh DESCRIPTION The H-TCP congestion control algorithm was designed to provide increased throughput in fast and long-distance networks. It attempts to maintain fairness when competing with legacy NewReno TCP in lower speed scenarios where NewReno is able to operate adequately. .Pp The congestion window is increased as a function of the time elapsed since the last congestion event. The window increase algorithm operates like NewReno for the first second after a congestion event, and then switches to a high-speed mode based on a quadratic increase function. .Pp The implementation was done in a clean-room fashion, and is based on the Internet Draft and other documents referenced in the .Sx SEE ALSO section below. .Sh MIB Variables The algorithm exposes the following tunable variables in the .Va net.inet.tcp.cc.htcp branch of the .Xr sysctl 3 MIB: .Bl -tag -width ".Va adaptive_backoff" .It Va adaptive_backoff Controls use of the adaptive backoff algorithm, which is designed to keep network queues non-empty during congestion recovery episodes. Default is 0 (disabled). .It Va rtt_scaling Controls use of the RTT scaling algorithm, which is designed to make congestion window increase during congestion avoidance mode invariant with respect to RTT. Default is 0 (disabled). .El .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr tcp 4 , .Xr cc 9 .Rs .%A "D. Leith" .%A "R. Shorten" .%T "H-TCP: TCP Congestion Control for High Bandwidth-Delay Product Paths" .%U "http://tools.ietf.org/id/draft-leith-tcp-htcp-06.txt" .Re .Rs .%A "D. Leith" .%A "R. Shorten" .%A "T. Yee" .%T "H-TCP: A framework for congestion control in high-speed and long-distance networks" .%B "Proc. PFLDnet" .%D "2005" .Re .Rs .%A "G. Armitage" .%A "L. Stewart" .%A "M. Welzl" .%A "J. Healy" .%T "An independent H-TCP implementation under FreeBSD 7.0: description and observed behaviour" .%J "SIGCOMM Comput. Commun. Rev." .%V "38" .%N "3" .%D "July 2008" .%P "27-38" .Re .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm congestion control module first appeared in .Fx 9.0 . .Pp The module was first released in 2007 by James Healy and Lawrence Stewart whilst -working on the NewTCP research project at Swinburne University's Centre for -Advanced Internet Architectures, Melbourne, Australia, which was made possible -in part by a grant from the Cisco University Research Program Fund at Community -Foundation Silicon Valley. +working on the NewTCP research project at Swinburne University of Technology's +Centre for Advanced Internet Architectures, Melbourne, Australia, which was made +possible in part by a grant from the Cisco University Research Program Fund at +Community Foundation Silicon Valley. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module was written by .An James Healy Aq jimmy@deefa.com and .An Lawrence Stewart Aq lstewart@FreeBSD.org . .Pp This manual page was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org and .An David Hayes Aq david.hayes@ieee.org . Index: head/share/man/man4/cc_newreno.4 =================================================================== --- head/share/man/man4/cc_newreno.4 (revision 220559) +++ head/share/man/man4/cc_newreno.4 (revision 220560) @@ -1,82 +1,82 @@ .\" .\" Copyright (c) 2009 Lawrence Stewart .\" Copyright (c) 2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" Portions of this documentation were written at the Centre for Advanced -.\" Internet Architectures, Swinburne University, Melbourne, Australia by -.\" Lawrence Stewart under sponsorship from the FreeBSD Foundation. +.\" Internet Architectures, Swinburne University of Technology, Melbourne, +.\" Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_NEWRENO 4 .Os .Sh NAME .Nm cc_newreno .Nd NewReno Congestion Control Algorithm .Sh DESCRIPTION The NewReno congestion control algorithm is the default for TCP. Details about the algorithm can be found in RFC5681. .Sh MIB Variables There are currently no tunable MIB variables. .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_vegas 4 , .Xr tcp 4 , .Xr cc 9 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm congestion control algorithm first appeared in its modular form in .Fx 9.0 . .Pp The module was first released in 2007 by James Healy and Lawrence Stewart whilst -working on the NewTCP research project at Swinburne University's Centre for -Advanced Internet Architectures, Melbourne, Australia, which was made possible -in part by a grant from the Cisco University Research Program Fund at Community -Foundation Silicon Valley. +working on the NewTCP research project at Swinburne University of Technology's +Centre for Advanced Internet Architectures, Melbourne, Australia, which was made +possible in part by a grant from the Cisco University Research Program Fund at +Community Foundation Silicon Valley. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module was written by .An James Healy Aq jimmy@deefa.com , .An Lawrence Stewart Aq lstewart@FreeBSD.org and .An David Hayes Aq david.hayes@ieee.org . .Pp This manual page was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org . Index: head/share/man/man4/cc_vegas.4 =================================================================== --- head/share/man/man4/cc_vegas.4 (revision 220559) +++ head/share/man/man4/cc_vegas.4 (revision 220560) @@ -1,138 +1,138 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes -.\" under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC_VEGAS 4 .Os .Sh NAME .Nm cc_vegas .Nd Vegas Congestion Control Algorithm .Sh DESCRIPTION The Vegas congestion control algorithm uses what the authors term the actual and expected transmission rates to determine whether there is congestion along the network path i.e. .Pp .Bl -item -offset indent .It actual rate = (total data sent in a RTT) / RTT .It expected rate = cwnd / RTTmin .It diff = expected - actual .El .Pp where RTT is the measured instantaneous round trip time and RTTmin is the smallest round trip time observed during the connection. .Pp The algorithm aims to keep diff between two parameters alpha and beta, such that: .Pp .Bl -item -offset indent .It alpha < diff < beta .El .Pp If diff > beta, congestion is inferred and cwnd is decremented by one packet (or the maximum TCP segment size). If diff < alpha, then cwnd is incremented by one packet. Alpha and beta govern the amount of buffering along the path. .Pp The implementation was done in a clean-room fashion, and is based on the paper referenced in the .Sx SEE ALSO section below. .Sh IMPLEMENTATION NOTES The time from the transmission of a marked packet until the receipt of an acknowledgement for that packet is measured once per RTT. This implementation does not implement Brakmo's and Peterson's original duplicate ACK policy since clock ticks in today's machines are not as coarse as they were (i.e. 500ms) when Vegas was originally designed. Note that modern TCP recovery processes such as fast retransmit and SACK are enabled by default in the TCP stack. .Sh MIB Variables The algorithm exposes the following tunable variables in the .Va net.inet.tcp.cc.vegas branch of the .Xr sysctl 3 MIB: .Bl -tag -width ".Va alpha" .It Va alpha Query or set the Vegas alpha parameter as a number of buffers on the path. When setting alpha, the value must satisfy: 0 < alpha < beta. Default is 1. .It Va beta Query or set the Vegas beta parameter as a number of buffers on the path. When setting beta, the value must satisfy: 0 < alpha < beta. Default is 3. .El .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr h_ertt 4 , .Xr tcp 4 , .Xr cc 9 , .Xr khelp 9 .Rs .%A "L. S. Brakmo" .%A "L. L. Peterson" .%T "TCP Vegas: end to end congestion avoidance on a global internet" .%J "IEEE J. Sel. Areas Commun." .%D "October 1995" .%V "13" .%N "8" .%P "1465-1480" .Re .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm congestion control module first appeared in .Fx 9.0 . .Pp The module was first released in 2010 by David Hayes whilst working on the -NewTCP research project at Swinburne University's Centre for Advanced Internet -Architectures, Melbourne, Australia. +NewTCP research project at Swinburne University of Technology's Centre for +Advanced Internet Architectures, Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm congestion control module and this manual page were written by .An David Hayes Aq david.hayes@ieee.org . Index: head/share/man/man4/h_ertt.4 =================================================================== --- head/share/man/man4/h_ertt.4 (revision 220559) +++ head/share/man/man4/h_ertt.4 (revision 220560) @@ -1,143 +1,143 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes -.\" under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes under sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt h_ertt 9 .Os .Sh NAME .Nm h_ertt .Nd Enhanced Round Trip Time Khelp module .Sh SYNOPSIS .In netinet/khelp/h_ertt.h .Sh DESCRIPTION The .Nm Khelp module works within the .Xr khelp 9 framework to provide TCP with a per-connection, low noise estimate of the instantaneous RTT. The implementation attempts to be robust in the face of delayed acknowledgements, TCP Segmentation Offload (TSO), receivers who manipulate TCP timestamps and lack of the TCP timestamp option altogether. .Pp TCP receivers using delayed acknowledgements either acknowledge every second packet (reflecting the time stamp of the first) or use a timeout to trigger the acknowledgement if no second packet arrives. If the heuristic used by .Nm determines that the receiver is using delayed acknowledgements, it measures the RTT using the second packet (the one that triggers the acknowledgement). It does not measure the RTT if the acknowledgement is for the first packet, since it cannot be accurately determined. .Pp When TSO is in use, .Nm will momentarily disable TSO whilst marking a packet to use for a new measurement. The process has negligible impact on the connection. .Pp .Nm associates the following struct with each connection's TCP control block: .Bd -literal struct ertt { TAILQ_HEAD(txseginfo_head, txseginfo) txsegi_q; /* Private. */ long bytes_tx_in_rtt; /* Private. */ long bytes_tx_in_marked_rtt; unsigned long marked_snd_cwnd; int rtt; int maxrtt; int minrtt; int dlyack_rx; /* Private. */ int timestamp_errors; /* Private. */ int markedpkt_rtt; /* Private. */ uint32_t flags; }; .Ed .Pp The fields marked as private should not be manipulated by any code outside of the .Nm implementation. The non-private fields provide the following data: .Bl -tag -width ".Va bytes_tx_in_marked_rtt" -offset indent .It Va bytes_tx_in_marked_rtt The number of bytes transmitted in the .Va markedpkt_rtt . .It Va marked_snd_cwnd The value of cwnd for the marked rtt measurement. .It Va rtt The most recent RTT measurement. .It Va maxrtt The longest RTT measurement that has been taken. .It Va minrtt The shortest RTT measurement that has been taken. .It Va flags The ERTT_NEW_MEASUREMENT flag will be set by the implementation when a new measurement is available. It is the responsibility of .Nm consumers to unset the flag if they wish to use it as a notification method for new measurements. .El .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_hd 4 , .Xr cc_vegas 4 , .Xr hhook 9 , .Xr khelp 9 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm module first appeared in .Fx 9.0 . .Pp The module was first released in 2010 by David Hayes whilst working on the -NewTCP research project at Swinburne University's Centre for Advanced Internet -Architectures, Melbourne, Australia. +NewTCP research project at Swinburne University of Technology's Centre for +Advanced Internet Architectures, Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm Khelp module and this manual page were written by .An David Hayes Aq david.hayes@ieee.org . .Sh BUGS The module maintains enhanced RTT estimates for all new TCP connections created after the time at which the module was loaded. It might be beneficial to see if it is possible to have the module only affect connections which actually care about ERTT estimates. Index: head/share/man/man4/siftr.4 =================================================================== --- head/share/man/man4/siftr.4 (revision 220559) +++ head/share/man/man4/siftr.4 (revision 220560) @@ -1,758 +1,758 @@ .\" .\" Copyright (c) 2010 The FreeBSD Foundation .\" All rights reserved. .\" .\" Portions of this software were developed at the Centre for Advanced .\" Internet Architectures, Swinburne University of Technology, Melbourne, .\" Australia by Lawrence Stewart under sponsorship from the FreeBSD .\" Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions, and the following disclaimer, .\" without modification, immediately at the beginning of the file. .\" 2. The name of the author may not be used to endorse or promote products .\" derived from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd November 12, 2010 .Dt SIFTR 4 .Os .Sh NAME .Nm SIFTR .Nd Statistical Information For TCP Research .Sh SYNOPSIS To load .Ns Nm as a module at run-time, run the following command as root: .Bd -literal -offset indent kldload siftr .Ed .Pp Alternatively, to load .Ns Nm as a module at boot time, add the following line into the .Xr loader.conf 5 file: .Bd -literal -offset indent siftr_load="YES" .Ed .Sh DESCRIPTION .Nm .Ns ( Em S Ns tatistical .Em I Ns nformation .Em F Ns or .Em T Ns CP .Em R Ns esearch ) is a kernel module that logs a range of statistics on active TCP connections to a log file. It provides the ability to make highly granular measurements of TCP connection state, aimed at system administrators, developers and researchers. .Ss Compile-time Configuration The default operation of .Nm is to capture IPv4 TCP/IP packets. .Nm can be configured to support IPv4 and IPv6 by uncommenting: .Bd -literal -offset indent CFLAGS+=-DSIFTR_IPV6 .Ed .Pp in .Aq sys/modules/siftr/Makefile and recompiling. .Pp In the IPv4-only (default) mode, standard dotted decimal notation (e.g. "136.186.229.95") is used to format IPv4 addresses for logging. In IPv6 mode, standard dotted decimal notation is used to format IPv4 addresses, and standard colon-separated hex notation (see RFC 4291) is used to format IPv6 addresses for logging. Note that SIFTR uses uncompressed notation to format IPv6 addresses. For example, the address "fe80::20f:feff:fea2:531b" would be logged as "fe80:0:0:0:20f:feff:fea2:531b". .Ss Run-time Configuration .Nm utilises the .Xr sysctl 8 interface to export its configuration variables to user-space. The following variables are available: .Bl -tag -offset indent .It Va net.inet.siftr.enabled controls whether the module performs its measurements or not. By default, the value is set to 0, which means the module will not be taking any measurements. Having the module loaded with .Va net.inet.siftr.enabled set to 0 will have no impact on the performance of the network stack, as the packet filtering hooks are only inserted when .Va net.inet.siftr.enabled is set to 1. .El .Bl -tag -offset indent .It Va net.inet.siftr.ppl controls how many inbound/outbound packets for a given TCP connection will cause a log message to be generated for the connection. By default, the value is set to 1, which means the module will log a message for every packet of every TCP connection. The value can be set to any integer in the range [1,2^32], and can be changed at any time, even while the module is enabled. .El .Bl -tag -offset indent .It Va net.inet.siftr.logfile controls the path to the file that the module writes its log messages to. By default, the file /var/log/siftr.log is used. The path can be changed at any time, even while the module is enabled. .El .Bl -tag -offset indent .It Va net.inet.siftr.genhashes controls whether a hash is generated for each TCP packet seen by .Nm . By default, the value is set to 0, which means no hashes are generated. The hashes are useful to correlate which TCP packet triggered the generation of a particular log message, but calculating them adds additional computational overhead into the fast path. .El .Ss Log Format A typical .Nm log file will contain 3 different types of log message. All messages are written in plain ASCII text. .Pp Note: The .Qq \e present in the example log messages in this section indicates a line continuation and is not part of the actual log message .Pp The first type of log message is written to the file when the module is enabled and starts collecting data from the running kernel. The text below shows an example module enable log. The fields are tab delimited key-value pairs which describe some basic information about the system. .Bd -literal -offset indent enable_time_secs=1238556193 enable_time_usecs=462104 \\ siftrver=1.2.2 hz=1000 tcp_rtt_scale=32 \\ sysname=FreeBSD sysver=604000 ipmode=4 .Ed .Pp Field descriptions are as follows: .Bl -tag -offset indent .It Va enable_time_secs time at which the module was enabled, in seconds since the UNIX epoch. .El .Bl -tag -offset indent .It Va enable_time_usecs time at which the module was enabled, in microseconds since enable_time_secs. .El .Bl -tag -offset indent .It Va siftrver version of .Nm . .El .Bl -tag -offset indent .It Va hz tick rate of the kernel in ticks per second. .El .Bl -tag -offset indent .It Va tcp_rtt_scale smoothed RTT estimate scaling factor .El .Bl -tag -offset indent .It Va sysname operating system name .El .Bl -tag -offset indent .It Va sysver operating system version .El .Bl -tag -offset indent .It Va ipmode IP mode as defined at compile time. An ipmode of "4" means IPv6 is not supported and IP addresses are logged in regular dotted quad format. An ipmode of "6" means IPv6 is supported, and IP addresses are logged in dotted quad or hex format, as described in the .Qq Compile-time Configuration subsection. .El .Pp The second type of log message is written to the file when a data log message is generated. The text below shows an example data log triggered by an IPv4 TCP/IP packet. The data is CSV formatted. .Bd -literal -offset indent o,0xbec491a5,1238556193.463551,172.16.7.28,22,172.16.2.5,55931, \\ 1073725440,172312,6144,66560,66608,8,1,4,1448,936,1,996,255, \\ 33304,208,66608,0,208,0 .Ed .Pp Field descriptions are as follows: .Bl -tag -offset indent .It Va 1 Direction of packet that triggered the log message. Either .Qq i for in, or .Qq o for out. .El .Bl -tag -offset indent .It Va 2 Hash of the packet that triggered the log message. .El .Bl -tag -offset indent .It Va 3 Time at which the packet that triggered the log message was processed by the .Xr pfil 9 hook function, in seconds and microseconds since the UNIX epoch. .El .Bl -tag -offset indent .It Va 4 The IPv4 or IPv6 address of the local host, in dotted quad (IPv4 packet) or colon-separated hex (IPv6 packet) notation. .El .Bl -tag -offset indent .It Va 5 The TCP port that the local host is communicating via. .El .Bl -tag -offset indent .It Va 6 The IPv4 or IPv6 address of the foreign host, in dotted quad (IPv4 packet) or colon-separated hex (IPv6 packet) notation. .El .Bl -tag -offset indent .It Va 7 The TCP port that the foreign host is communicating via. .El .Bl -tag -offset indent .It Va 8 The slow start threshold for the flow, in bytes. .El .Bl -tag -offset indent .It Va 9 The current congestion window for the flow, in bytes. .El .Bl -tag -offset indent .It Va 10 The current bandwidth-controlled window for the flow, in bytes. .El .Bl -tag -offset indent .It Va 11 The current sending window for the flow, in bytes. The post scaled value is reported, except during the initial handshake (first few packets), during which time the unscaled value is reported. .El .Bl -tag -offset indent .It Va 12 The current receive window for the flow, in bytes. The post scaled value is always reported. .El .Bl -tag -offset indent .It Va 13 The current window scaling factor for the sending window. .El .Bl -tag -offset indent .It Va 14 The current window scaling factor for the receiving window. .El .Bl -tag -offset indent .It Va 15 The current state of the TCP finite state machine, as defined in .Aq Pa netinet/tcp_fsm.h . .El .Bl -tag -offset indent .It Va 16 The maximum segment size for the flow, in bytes. .El .Bl -tag -offset indent .It Va 17 The current smoothed RTT estimate for the flow, in units of TCP_RTT_SCALE * HZ, where TCP_RTT_SCALE is a define found in tcp_var.h, and HZ is the kernel's tick timer. Divide by TCP_RTT_SCALE * HZ to get the RTT in secs. TCP_RTT_SCALE and HZ are reported in the enable log message. .El .Bl -tag -offset indent .It Va 18 SACK enabled indicator. 1 if SACK enabled, 0 otherwise. .El .Bl -tag -offset indent .It Va 19 The current state of the TCP flags for the flow. See .Aq Pa netinet/tcp_var.h for information about the various flags. .El .Bl -tag -offset indent .It Va 20 The current retransmission timeout length for the flow, in units of HZ, where HZ is the kernel's tick timer. Divide by HZ to get the timeout length in seconds. HZ is reported in the enable log message. .El .Bl -tag -offset indent .It Va 21 The current size of the socket send buffer in bytes. .El .Bl -tag -offset indent .It Va 22 The current number of bytes in the socket send buffer. .El .Bl -tag -offset indent .It Va 23 The current size of the socket receive buffer in bytes. .El .Bl -tag -offset indent .It Va 24 The current number of bytes in the socket receive buffer. .El .Bl -tag -offset indent .It Va 25 The current number of unacknowledged bytes in-flight. Bytes acknowledged via SACK are not excluded from this count. .El .Bl -tag -offset indent .It Va 26 The current number of segments in the reassembly queue. .El .Pp The third type of log message is written to the file when the module is disabled and ceases collecting data from the running kernel. The text below shows an example module disable log. The fields are tab delimited key-value pairs which provide statistics about operations since the module was most recently enabled. .Bd -literal -offset indent disable_time_secs=1238556197 disable_time_usecs=933607 \\ num_inbound_tcp_pkts=356 num_outbound_tcp_pkts=627 \\ total_tcp_pkts=983 num_inbound_skipped_pkts_malloc=0 \\ num_outbound_skipped_pkts_malloc=0 num_inbound_skipped_pkts_mtx=0 \\ num_outbound_skipped_pkts_mtx=0 num_inbound_skipped_pkts_tcb=0 \\ num_outbound_skipped_pkts_tcb=0 num_inbound_skipped_pkts_icb=0 \\ num_outbound_skipped_pkts_icb=0 total_skipped_tcp_pkts=0 \\ flow_list=172.16.7.28;22-172.16.2.5;55931, .Ed .Pp Field descriptions are as follows: .Bl -tag -offset indent .It Va disable_time_secs Time at which the module was disabled, in seconds since the UNIX epoch. .El .Bl -tag -offset indent .It Va disable_time_usecs Time at which the module was disabled, in microseconds since disable_time_secs. .El .Bl -tag -offset indent .It Va num_inbound_tcp_pkts Number of TCP packets that traversed up the network stack. This only includes inbound TCP packets during the periods when .Nm was enabled. .El .Bl -tag -offset indent .It Va num_outbound_tcp_pkts Number of TCP packets that traversed down the network stack. This only includes outbound TCP packets during the periods when .Nm was enabled. .El .Bl -tag -offset indent .It Va total_tcp_pkts The summation of num_inbound_tcp_pkts and num_outbound_tcp_pkts. .El .Bl -tag -offset indent .It Va num_inbound_skipped_pkts_malloc Number of inbound packets that were not processed because of failed malloc() calls. .El .Bl -tag -offset indent .It Va num_outbound_skipped_pkts_malloc Number of outbound packets that were not processed because of failed malloc() calls. .El .Bl -tag -offset indent .It Va num_inbound_skipped_pkts_mtx Number of inbound packets that were not processed because of failure to add the packet to the packet processing queue. .El .Bl -tag -offset indent .It Va num_outbound_skipped_pkts_mtx Number of outbound packets that were not processed because of failure to add the packet to the packet processing queue. .El .Bl -tag -offset indent .It Va num_inbound_skipped_pkts_tcb Number of inbound packets that were not processed because of failure to find the TCP control block associated with the packet. .El .Bl -tag -offset indent .It Va num_outbound_skipped_pkts_tcb Number of outbound packets that were not processed because of failure to find the TCP control block associated with the packet. .El .Bl -tag -offset indent .It Va num_inbound_skipped_pkts_icb Number of inbound packets that were not processed because of failure to find the IP control block associated with the packet. .El .Bl -tag -offset indent .It Va num_outbound_skipped_pkts_icb Number of outbound packets that were not processed because of failure to find the IP control block associated with the packet. .El .Bl -tag -offset indent .It Va total_skipped_tcp_pkts The summation of all skipped packet counters. .El .Bl -tag -offset indent .It Va flow_list A CSV list of TCP flows that triggered data log messages to be generated since the module was loaded. Each flow entry in the CSV list is formatted as .Qq local_ip;local_port-foreign_ip;foreign_port . If there are no entries in the list (i.e. no data log messages were generated), the value will be blank. If there is at least one entry in the list, a trailing comma will always be present. .El .Pp The total number of data log messages found in the log file for a module enable/disable cycle should equate to total_tcp_pkts - total_skipped_tcp_pkts. .Sh IMPLEMENTATION NOTES .Nm hooks into the network stack using the .Xr pfil 9 interface. In its current incarnation, it hooks into the AF_INET/AF_INET6 (IPv4/IPv6) .Xr pfil 9 filtering points, which means it sees packets at the IP layer of the network stack. This means that TCP packets inbound to the stack are intercepted before they have been processed by the TCP layer. Packets outbound from the stack are intercepted after they have been processed by the TCP layer. .Pp The diagram below illustrates how .Nm inserts itself into the stack. .Bd -literal -offset indent ---------------------------------- Upper Layers ---------------------------------- ^ | | | | | | v TCP in TCP out ---------------------------------- ^ | |________ _________| | | | v --------- | SIFTR | --------- ^ | ________| |__________ | | | v IPv{4/6} in IPv{4/6} out ---------------------------------- ^ | | | | v Layer 2 in Layer 2 out ---------------------------------- Physical Layer ---------------------------------- .Ed .Pp .Nm uses the .Xr alq 9 interface to manage writing data to disk. .Pp At first glance, you might mistakenly think that .Nm extracts information from individual TCP packets. This is not the case. .Nm uses TCP packet events (inbound and outbound) for each TCP flow originating from the system to trigger a dump of the state of the TCP control block for that flow. With the PPL set to 1, we are in effect sampling each TCP flow's control block state as frequently as flow packets enter/leave the system. For example, setting PPL to 2 halves the sampling rate i.e. every second flow packet (inbound OR outbound) causes a dump of the control block state. .Pp The distinction between interrogating individual packets vs interrogating the control block is important, because .Nm does not remove the need for packet capturing tools like .Xr tcpdump 1 . .Nm allows you to correlate and observe the cause-and-affect relationship between what you see on the wire (captured using a tool like .Xr tcpdump 1 Ns ) and changes in the TCP control block corresponding to the flow of interest. It is therefore useful to use .Nm and a tool like .Xr tcpdump 1 to gather the necessary data to piece together the complete picture. Use of either tool on its own will not be able to provide all of the necessary data. .Pp As a result of needing to interrogate the TCP control block, certain packets during the lifecycle of a connection are unable to trigger a .Nm log message. The initial handshake takes place without the existence of a control block and the final ACK is exchanged when the connection is in the TIMEWAIT state. .Pp .Nm was designed to minimise the delay introduced to packets traversing the network stack. This design called for a highly optimised and minimal hook function that extracted the minimal details necessary whilst holding the packet up, and passing these details to another thread for actual processing and logging. .Pp This multithreaded design does introduce some contention issues when accessing the data structure shared between the threads of operation. When the hook function tries to place details in the structure, it must first acquire an exclusive lock. Likewise, when the processing thread tries to read details from the structure, it must also acquire an exclusive lock to do so. If one thread holds the lock, the other must wait before it can obtain it. This does introduce some additional bounded delay into the kernel's packet processing code path. .Pp In some cases (e.g. low memory, connection termination), TCP packets that enter the .Nm .Xr pfil 9 hook function will not trigger a log message to be generated. .Nm refers to this outcome as a .Qq skipped packet . Note that .Nm always ensures that packets are allowed to continue through the stack, even if they could not successfully trigger a data log message. .Nm will therefore not introduce any packet loss for TCP/IP packets traversing the network stack. .Ss Important Behaviours The behaviour of a log file path change whilst the module is enabled is as follows: .Bl -enum .It Attempt to open the new file path for writing. If this fails, the path change will fail and the existing path will continue to be used. .It Assuming the new path is valid and opened successfully: .Bl -dash .It Flush all pending log messages to the old file path. .It Close the old file path. .It Switch the active log file pointer to point at the new file path. .It Commence logging to the new file. .El .El .Pp During the time between the flush of pending log messages to the old file and commencing logging to the new file, new log messages will still be generated and buffered. As soon as the new file path is ready for writing, the accumulated log messages will be written out to the file. .Sh EXAMPLES To enable the module's operations, run the following command as root: sysctl net.inet.siftr.enabled=1 .Pp To change the granularity of log messages such that 1 log message is generated for every 10 TCP packets per connection, run the following command as root: sysctl net.inet.siftr.ppl=10 .Pp To change the log file location to /tmp/siftr.log, run the following command as root: sysctl net.inet.siftr.logfile=/tmp/siftr.log .Sh SEE ALSO .Xr alq 9 , .Xr pfil 9 .Xr sysctl 8 , .Xr tcp 4 , .Xr tcpdump 1 , .Sh ACKNOWLEDGEMENTS Development of this software was made possible in part by grants from the Cisco University Research Program Fund at Community Foundation Silicon Valley, and the FreeBSD Foundation. .Sh HISTORY .Nm first appeared in .Fx 7.4 and .Fx 8.2 . .Pp .Nm was first released in 2007 by Lawrence Stewart and James Healy whilst working on -the NewTCP research project at Swinburne University's Centre for Advanced -Internet Architectures, Melbourne, Australia, which was made possible in part by -a grant from the Cisco University Research Program Fund at Community Foundation -Silicon Valley. +the NewTCP research project at Swinburne University of Technology's Centre for +Advanced Internet Architectures, Melbourne, Australia, which was made possible +in part by a grant from the Cisco University Research Program Fund at Community +Foundation Silicon Valley. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Pp Work on .Nm v1.2.x was sponsored by the FreeBSD Foundation as part of the .Qq Enhancing the FreeBSD TCP Implementation project 2008-2009. More details are available at: .Pp http://www.freebsdfoundation.org/ .Pp http://caia.swin.edu.au/freebsd/etcp09/ .Sh AUTHORS .An -nosplit .Nm was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org and .An James Healy Aq jimmy@deefa.com . .Pp This manual page was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org . .Sh BUGS Current known limitations and any relevant workarounds are outlined below: .Bl -dash .It The internal queue used to pass information between the threads of operation is currently unbounded. This allows .Nm to cope with bursty network traffic, but sustained high packet-per-second traffic can cause exhaustion of kernel memory if the processing thread cannot keep up with the packet rate. .It If using .Nm on a machine that is also running other modules utilising the .Xr pfil 9 framework e.g. .Xr dummynet 4 , .Xr ipfw 8 , .Xr pf 4 Ns , the order in which you load the modules is important. You should kldload the other modules first, as this will ensure TCP packets undergo any necessary manipulations before .Nm .Qq sees and processes them. .It There is a known, harmless lock order reversal warning between the .Xr pfil 9 mutex and tcbinfo TCP lock reported by .Xr witness 4 when .Nm is enabled in a kernel compiled with .Xr witness 4 support. .It There is no way to filter which TCP flows you wish to capture data for. Post processing is required to separate out data belonging to particular flows of interest. .It The module does not detect deletion of the log file path. New log messages will simply be lost if the log file being used by .Nm is deleted whilst the module is set to use the file. Switching to a new log file using the .Em net.inet.siftr.logfile variable will create the new file and allow log messages to begin being written to disk again. The new log file path must differ from the path to the deleted file. .It The hash table used within the code is sized to hold 65536 flows. This is not a hard limit, because chaining is used to handle collisions within the hash table structure. However, we suspect (based on analogies with other hash table performance data) that the hash table look up performance (and therefore the module's packet processing performance) will degrade in an exponential manner as the number of unique flows handled in a module enable/disable cycle approaches and surpasses 65536. .It There is no garbage collection performed on the flow hash table. The only way currently to flush it is to disable .Nm . .It The PPL variable applies to packets that make it into the processing thread, not total packets received in the hook function. Packets are skipped before the PPL variable is applied, which means there may be a slight discrepancy in the triggering of log messages. For example, if PPL was set to 10, and the 8th packet since the last log message is skipped, the 11th packet will actually trigger the log message to be generated. This is discussed in greater depth in CAIA technical report 070824A. .It At the time of writing, there was no simple way to hook into the TCP layer to intercept packets. .Nm Ap s use of IP layer hook points means all IP traffic will be processed by the .Nm .Xr pfil 9 hook function, which introduces minor, but nonetheless unnecessary packet delay and processing overhead on the system for non-TCP packets as well. Hooking in at the IP layer is also not ideal from the data gathering point of view. Packets traversing up the stack will be intercepted and cause a log message generation BEFORE they have been processed by the TCP layer, which means we cannot observe the cause-and-affect relationship between inbound events and the corresponding TCP control block as precisely as could be. Ideally, .Nm should intercept packets after they have been processed by the TCP layer i.e. intercept packets coming up the stack after they have been processed by tcp_input(), and intercept packets coming down the stack after they have been processed by tcp_output(). The current code still gives satisfactory granularity though, as inbound events tend to trigger outbound events, allowing the cause-and-effect to be observed indirectly by capturing the state on outbound events as well. .It The .Qq inflight bytes value logged by .Nm does not take into account bytes that have been .No SACK Ap ed by the receiving host. .It Packet hash generation does not currently work for IPv6 based TCP packets. .It Compressed notation is not used for IPv6 address representation. This consumes more bytes than is necessary in log output. .El Index: head/share/man/man9/cc.9 =================================================================== --- head/share/man/man9/cc.9 (revision 220559) +++ head/share/man/man9/cc.9 (revision 220560) @@ -1,333 +1,333 @@ .\" .\" Copyright (c) 2008-2009 Lawrence Stewart .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" Portions of this documentation were written at the Centre for Advanced -.\" Internet Architectures, Swinburne University, Melbourne, Australia by -.\" David Hayes and Lawrence Stewart under sponsorship from the +.\" Internet Architectures, Swinburne University of Technology, Melbourne, +.\" Australia by David Hayes and Lawrence Stewart under sponsorship from the .\" FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt CC 9 .Os .Sh NAME .Nm cc , .Nm DECLARE_CC_MODULE , .Nm CC_VAR .Nd Modular Congestion Control .Sh SYNOPSIS .In netinet/cc.h .In netinet/cc/cc_module.h .Fn DECLARE_CC_MODULE "ccname" "ccalgo" .Fn CC_VAR "ccv" "what" .Sh DESCRIPTION The .Nm framework allows congestion control algorithms to be implemented as dynamically loadable kernel modules via the .Xr kld 4 facility. Transport protocols can select from the list of available algorithms on a connection-by-connection basis, or use the system default (see .Xr cc 4 for more details). .Pp .Nm modules are identified by an .Xr ascii 7 name and set of hook functions encapsulated in a .Vt "struct cc_algo" , which has the following members: .Bd -literal -offset indent struct cc_algo { char name[TCP_CA_NAME_MAX]; int (*mod_init) (void); int (*mod_destroy) (void); int (*cb_init) (struct cc_var *ccv); void (*cb_destroy) (struct cc_var *ccv); void (*conn_init) (struct cc_var *ccv); void (*ack_received) (struct cc_var *ccv, uint16_t type); void (*cong_signal) (struct cc_var *ccv, uint32_t type); void (*post_recovery) (struct cc_var *ccv); void (*after_idle) (struct cc_var *ccv); }; .Ed .Pp The .Va name field identifies the unique name of the algorithm, and should be no longer than TCP_CA_NAME_MAX-1 characters in length (the TCP_CA_NAME_MAX define lives in .In netinet/tcp.h for compatibility reasons). .Pp The .Va mod_init function is called when a new module is loaded into the system but before the registration process is complete. It should be implemented if a module needs to set up some global state prior to being available for use by new connections. Returning a non-zero value from .Va mod_init will cause the loading of the module to fail. .Pp The .Va mod_destroy function is called prior to unloading an existing module from the kernel. It should be implemented if a module needs to clean up any global state before being removed from the kernel. The return value is currently ignored. .Pp The .Va cb_init function is called when a TCP control block .Vt struct tcpcb is created. It should be implemented if a module needs to allocate memory for storing private per-connection state. Returning a non-zero value from .Va cb_init will cause the connection set up to be aborted, terminating the connection as a result. .Pp The .Va cb_destroy function is called when a TCP control block .Vt struct tcpcb is destroyed. It should be implemented if a module needs to free memory allocated in .Va cb_init . .Pp The .Va conn_init function is called when a new connection has been established and variables are being initialised. It should be implemented to initialise congestion control algorithm variables for the newly established connection. .Pp The .Va ack_received function is called when a TCP acknowledgement (ACK) packet is received. Modules use the .Fa type argument as an input to their congestion management algorithms. The ACK types currently reported by the stack are CC_ACK and CC_DUPACK. CC_ACK indicates the received ACK acknowledges previously unacknowledged data. CC_DUPACK indicates the received ACK acknowledges data we have already received an ACK for. .Pp The .Va cong_signal function is called when a congestion event is detected by the TCP stack. Modules use the .Fa type argument as an input to their congestion management algorithms. The congestion event types currently reported by the stack are CC_ECN, CC_RTO, CC_RTO_ERR and CC_NDUPACK. CC_ECN is reported when the TCP stack receives an explicit congestion notification (RFC3168). CC_RTO is reported when the retransmission time out timer fires. CC_RTO_ERR is reported if the retransmission time out timer fired in error. CC_NDUPACK is reported if N duplicate ACKs have been received back-to-back, where N is the fast retransmit duplicate ack threshold (N=3 currently as per RFC5681). .Pp The .Va post_recovery function is called after the TCP connection has recovered from a congestion event. It should be implemented to adjust state as required. .Pp The .Va after_idle function is called when data transfer resumes after an idle period. It should be implemented to adjust state as required. .Pp The .Fn DECLARE_CC_MODULE macro provides a convenient wrapper around the .Xr DECLARE_MODULE 9 macro, and is used to register a .Nm module with the .Nm framework. The .Fa ccname argument specifies the module's name. The .Fa ccalgo argument points to the module's .Vt struct cc_algo . .Pp .Nm modules must instantiate a .Vt struct cc_algo , but are only required to set the name field, and optionally any of the function pointers. The stack will skip calling any function pointer which is NULL, so there is no requirement to implement any of the function pointers. Using the C99 designated initialiser feature to set fields is encouraged. .Pp Each function pointer which deals with congestion control state is passed a pointer to a .Vt struct cc_var , which has the following members: .Bd -literal -offset indent struct cc_var { void *cc_data; int bytes_this_ack; tcp_seq curack; uint32_t flags; int type; union ccv_container { struct tcpcb *tcp; struct sctp_nets *sctp; } ccvc; }; .Ed .Pp .Vt struct cc_var groups congestion control related variables into a single, embeddable structure and adds a layer of indirection to accessing transport protocol control blocks. The eventual goal is to allow a single set of .Nm modules to be shared between all congestion aware transport protocols, though currently only .Xr tcp 4 is supported. .Pp To aid the eventual transition towards this goal, direct use of variables from the transport protocol's data structures is strongly discouraged. However, it is inevitable at the current time to require access to some of these variables, and so the .Fn CC_VAR macro exists as a convenience accessor. The .Fa ccv argument points to the .Vt struct cc_var passed into the function by the .Nm framework. The .Fa what argument specifies the name of the variable to access. .Pp Apart from the .Va type and .Va ccv_container fields, the remaining fields in .Vt struct cc_var are for use by .Nm modules. .Pp The .Va cc_data field is available for algorithms requiring additional per-connection state to attach a dynamic memory pointer to. The memory should be allocated and attached in the module's .Va cb_init hook function. .Pp The .Va bytes_this_ack field specifies the number of new bytes acknowledged by the most recently received ACK packet. It is only valid in the .Va ack_received hook function. .Pp The .Va curack field specifies the sequence number of the most recently received ACK packet. It is only valid in the .Va ack_received , .Va cong_signal and .Va post_recovery hook functions. .Pp The .Va flags field is used to pass useful information from the stack to a .Nm module. The CCF_ABC_SENTAWND flag is relevant in .Va ack_received and is set when appropriate byte counting (RFC3465) has counted a window's worth of bytes has been sent. It is the module's responsibility to clear the flag after it has processed the signal. The CCF_CWND_LIMITED flag is relevant in .Va ack_received and is set when the connection's ability to send data is currently constrained by the value of the congestion window. Algorithms should use the abscence of this flag being set to avoid accumulating a large difference between the congestion window and send window. .Sh SEE ALSO .Xr cc 4 , .Xr cc_chd 4 , .Xr cc_cubic 4 , .Xr cc_hd 4 , .Xr cc_htcp 4 , .Xr cc_newreno 4 , .Xr cc_vegas 4 , .Xr tcp 4 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh FUTURE WORK Integrate with .Xr sctp 4 . .Sh HISTORY The modular Congestion Control (CC) framework first appeared in .Fx 9.0 . .Pp The framework was first released in 2007 by James Healy and Lawrence Stewart -whilst working on the NewTCP research project at Swinburne University's Centre -for Advanced Internet Architectures, Melbourne, Australia, which was made -possible in part by a grant from the Cisco University Research Program Fund at -Community Foundation Silicon Valley. +whilst working on the NewTCP research project at Swinburne University of +Technology's Centre for Advanced Internet Architectures, Melbourne, Australia, +which was made possible in part by a grant from the Cisco University Research +Program Fund at Community Foundation Silicon Valley. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm framework was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org , .An James Healy Aq jimmy@deefa.com and .An David Hayes Aq david.hayes@ieee.org . .Pp This manual page was written by .An David Hayes Aq david.hayes@ieee.org and .An Lawrence Stewart Aq lstewart@FreeBSD.org . Index: head/share/man/man9/hhook.9 =================================================================== --- head/share/man/man9/hhook.9 (revision 220559) +++ head/share/man/man9/hhook.9 (revision 220560) @@ -1,387 +1,388 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes and -.\" Lawrence Stewart under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes and Lawrence Stewart under sponsorship from the FreeBSD +.\" Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt hhook 9 .Os .Sh NAME .Nm hhook , .Nm hhook_head_register , .Nm hhook_head_deregister , .Nm hhook_head_deregister_lookup , .Nm hhook_run_hooks , .Nm HHOOKS_RUN_IF , .Nm HHOOKS_RUN_LOOKUP_IF .Nd Helper Hook Framework .Sh SYNOPSIS .In sys/hhook.h .Ft typedef int .Fn "\*(lp*hhook_func_t\*(rp" "int32_t hhook_type" "int32_t hhook_id" \ "void *udata" "void *ctx_data" "void *hdata" "struct osd *hosd" .Fn "int hhook_head_register" "int32_t hhook_type" "int32_t hhook_id" \ "struct hhook_head **hhh" "uint32_t flags" .Fn "int hhook_head_deregister" "struct hhook_head *hhh" .Fn "int hhook_head_deregister_lookup" "int32_t hhook_type" "int32_t hhook_id" .Fn "void hhook_run_hooks" "struct hhook_head *hhh" "void *ctx_data" \ "struct osd *hosd" .Fn HHOOKS_RUN_IF "hhh" "ctx_data" "hosd" .Fn HHOOKS_RUN_LOOKUP_IF "hhook_type" "hhook_id" "ctx_data" "hosd" .Sh DESCRIPTION .Nm provides a framework for managing and running arbitrary hook functions at defined hook points within the kernel. The KPI was inspired by .Xr pfil 9 , and in many respects can be thought of as a more generic superset of pfil. .Pp The .Xr khelp 9 and .Nm frameworks are tightly integrated. Khelp is responsible for registering and deregistering Khelp module hook functions with .Nm points. The KPI functions used by .Xr khelp 9 to do this are not documented here as they are not relevant to consumers wishing to instantiate hook points. .Ss Information for Khelp Module Implementors Khelp modules indirectly interact with .Nm by defining appropriate hook functions for insertion into hook points. Hook functions must conform to the .Ft hhook_func_t function pointer declaration outlined in the .Sx SYNOPSIS . .Pp The .Fa hhook_type and .Fa hhook_id arguments identify the hook point which has called into the hook function. These are useful when a single hook function is registered for multiple hook points and wants to know which hook point has called into it. .In sys/hhook.h lists available .Fa hhook_type defines and subsystems which export hook points are responsible for defining the .Fa hhook_id value in appropriate header files. .Pp The .Fa udata argument will be passed to the hook function if it was specified in the .Vt struct hookinfo at hook registration time. .Pp The .Fa ctx_data argument contains context specific data from the hook point call site. The data type passed is subsystem dependent. .Pp The .Fa hdata argument is a pointer to the persistent per-object storage allocated for use by the module if required. The pointer will only ever be NULL if the module did not request per-object storage. .Pp The .Fa hosd argument can be used with the .Xr khelp 9 framework's .Fn khelp_get_osd function to access data belonging to a different Khelp module. .Pp Khelp modules instruct the Khelp framework to register their hook functions with .Nm points by creating a .Vt "struct hookinfo" per hook point, which contains the following members: .Bd -literal -offset indent struct hookinfo { hhook_func_t hook_func; struct helper *hook_helper; void *hook_udata; int32_t hook_id; int32_t hook_type; }; .Ed .Pp Khelp modules are responsible for setting all members of the struct except .Va hook_helper which is handled by the Khelp framework. .Ss Creating and Managing Hook Points Kernel subsystems that wish to provide .Nm points typically need to make four and possibly five key changes to their implementation: .Bl -bullet .It Define a list of .Va hhook_id mappings in an appropriate subsystem header. .It Register each hook point with the .Fn hhook_head_register function during initialisation of the subsystem. .It Select or create a standardised data type to pass to hook functions as contextual data. .It Add a call to .Fn HHOOKS_RUN_IF or .Fn HHOOKS_RUN_IF_LOOKUP at the point in the subsystem's code where the hook point should be executed. .It If the subsystem can be dynamically added/removed at runtime, each hook point registered with the .Fn hhook_head_register function when the subsystem was initialised needs to be deregistered with the .Fn hhook_head_deregister or .Fn hhook_head_deregister_lookup functions when the subsystem is being deinitialised prior to removal. .El .Pp The .Fn hhook_head_register function registers a hook point with the .Nm framework. The .Fa hook_type argument defines the high level type for the hook point. Valid types are defined in .In sys/hhook.h and new types should be added as required. The .Fa hook_id argument specifies a unique, subsystem specific identifier for the hook point. The .Fa hhh argument will, if not NULL, be used to store a reference to the .Vt struct hhook_head created as part of the registration process. Subsystems will generally want to store a local copy of the .Vt struct hhook_head so that they can use the .Fn HHOOKS_RUN_IF macro to instantiate hook points. The HHOOK_WAITOK flag may be passed in via the .Fa flags argument if .Xr malloc 9 is allowed to sleep waiting for memory to become available. If the hook point is within a virtualised subsystem (e.g. the network stack), the HHOOK_HEADISINVNET flag should be passed in via the .Fa flags argument so that the .Vt struct hhook_head created during the registration process will be added to a virtualised list. .Pp The .Fn hhook_head_deregister function deregisters a previously registered hook point from the .Nm framework. The .Fa hhh argument is the pointer to the .Vt struct hhook_head returned by .Fn hhoook_head_register when the hook point was registered. .Pp The .Fn hhook_head_deregister_lookup function can be used instead of .Fn hhook_head_deregister in situations where the caller does not have a cached copy of the .Vt struct hhook_head and wants to deregister a hook point using the appropriate .Fa hook_type and .Fa hook_id identifiers instead. .Pp The .Fn hhook_run_hooks function should normally not be called directly and should instead be called indirectly via the .Fn HHOOKS_RUN_IF macro. However, there may be circumstances where it is preferable to call the function directly, and so it is documented here for completeness. The .Fa hhh argument references the .Nm point to call all registered hook functions for. The .Fa ctx_data argument specifies a pointer to the contextual hook point data to pass into the hook functions. The .Fa hosd argument should be the pointer to the appropriate object's .Vt struct osd if the subsystem provides the ability for Khelp modules to associate per-object data. Subsystems which do not should pass NULL. .Pp The .Fn HHOOKS_RUN_IF macro is the preferred way to implement hook points. It only calls the .Fn hhook_run_hooks function if at least one hook function is registered for the hook point. By checking for registered hook functions, the macro minimises the cost associated with adding hook points to frequently used code paths by reducing to a simple if test in the common case where no hook functions are registered. The arguments are as described for the .Fn hhook_run_hooks function. .Pp The .Fn HHOOKS_RUN_IF_LOOKUP macro performs the same function as the .Fn HHOOKS_RUN_IF macro, but performs an additional step to look up the .Vt struct hhook_head for the specified .Fa hook_type and .Fa hook_id identifiers. It should not be used except in code paths which are infrequently executed because of the reference counting overhead associated with the look up. .Sh IMPLEMENTATION NOTES Each .Vt struct hhook_head protects its internal list of hook functions with a .Xr rmlock 9 . Therefore, anytime .Fn hhook_run_hooks is called directly or indirectly via the .Fn HHOOKS_RUN_IF or .Fn HHOOKS_RUN_IF_LOOKUP macros, a non-sleepable read lock will be acquired and held across the calls to all registered hook functions. .Sh RETURN VALUES .Fn hhook_head_register returns 0 if no errors occurred. It returns EEXIST if a hook point with the same .Fa hook_type and .Fa hook_id is already registered. It returns EINVAL if the HHOOK_HEADISINVNET flag is not set in .Fa flags because the implementation does not yet support hook points in non-virtualised subsystems (see the .Sx BUGS section for details). It returns ENOMEM if .Xr malloc 9 failed to allocate memory for the new .Vt struct hhook_head . .Pp .Fn hhook_head_deregister and .Fn hhook_head_deregister_lookup return 0 if no errors occurred. They return ENOENT if .Fa hhh is NULL. They return EBUSY if the reference count of .Fa hhh is greater than one. .Sh EXAMPLES A well commented example Khelp module can be found at: .Pa /usr/share/examples/kld/khelp/h_example.c .Pp The .Xr tcp 4 implementation provides two .Nm points which are called for packets sent/received when a connection is in the established phase. Search for HHOOK in the following files: .Pa sys/netinet/tcp_var.h , .Pa sys/netinet/tcp_input.c , .Pa sys/netinet/tcp_output.c and .Pa sys/netinet/tcp_subr.c . .Sh SEE ALSO .Xr khelp 9 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm framework first appeared in .Fx 9.0 . .Pp The .Nm framework was first released in 2010 by Lawrence Stewart whilst studying at -Swinburne University's Centre for Advanced Internet Architectures, Melbourne, -Australia. +Swinburne University of Technology's Centre for Advanced Internet Architectures, +Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm framework was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org . .Pp This manual page was written by .An David Hayes Aq david.hayes@ieee.org and .An Lawrence Stewart Aq lstewart@FreeBSD.org . .Sh BUGS The framework does not currently support registering hook points in subsystems which have not been virtualised with VIMAGE. Fairly minimal internal changes to the .Nm implementation are required to address this. Index: head/share/man/man9/khelp.9 =================================================================== --- head/share/man/man9/khelp.9 (revision 220559) +++ head/share/man/man9/khelp.9 (revision 220560) @@ -1,437 +1,438 @@ .\" .\" Copyright (c) 2010-2011 The FreeBSD Foundation .\" All rights reserved. .\" .\" This documentation was written at the Centre for Advanced Internet -.\" Architectures, Swinburne University, Melbourne, Australia by David Hayes and -.\" Lawrence Stewart under sponsorship from the FreeBSD Foundation. +.\" Architectures, Swinburne University of Technology, Melbourne, Australia by +.\" David Hayes and Lawrence Stewart under sponsorship from the FreeBSD +.\" Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 15, 2011 .Dt khelp 9 .Os .Sh NAME .Nm khelp , .Nm khelp_init_osd , .Nm khelp_destroy_osd , .Nm khelp_get_id , .Nm khelp_get_osd , .Nm khelp_add_hhook , .Nm khelp_remove_hhook , .Nm KHELP_DECLARE_MOD , .Nm KHELP_DECLARE_MOD_UMA .Nd Kernel Helper Framework .Sh SYNOPSIS .In sys/khelp.h .In sys/module_khelp.h .Fn "int khelp_init_osd" "uint32_t classes" "struct osd *hosd" .Fn "int khelp_destroy_osd" "struct osd *hosd" .Fn "int32_t khelp_get_id" "char *hname" .Fn "void * khelp_get_osd" "struct osd *hosd" "int32_t id" .Fn "int khelp_add_hhook" "struct hookinfo *hki" "uint32_t flags" .Fn "int khelp_remove_hhook" "struct hookinfo *hki" .Fn KHELP_DECLARE_MOD "hname" "hdata" "hhooks" "version" .Fn KHELP_DECLARE_MOD_UMA "hname" "hdata" "hhooks" "version" "ctor" "dtor" .Sh DESCRIPTION .Nm provides a framework for managing .Nm modules, which indirectly use the .Xr hhook 9 KPI to register their hook functions with hook points of interest within the kernel. Khelp modules aim to provide a structured way to dynamically extend the kernel at runtime in an ABI preserving manner. Depending on the subsystem providing hook points, a .Nm module may be able to associate per-object data for maintaining relevant state between hook calls. The .Xr hhook 9 and .Nm frameworks are tightly integrated and anyone interested in .Nm should also read the .Xr hhook 9 manual page thoroughly. .Ss Information for Khelp Module Implementors .Nm modules are represented within the .Nm framework by a .Vt struct helper which has the following members: .Bd -literal -offset indent struct helper { int (*mod_init) (void); int (*mod_destroy) (void); #define HELPER_NAME_MAXLEN 16 char h_name[HELPER_NAME_MAXLEN]; uma_zone_t h_zone; struct hookinfo *h_hooks; uint32_t h_nhooks; uint32_t h_classes; int32_t h_id; volatile uint32_t h_refcount; uint16_t h_flags; TAILQ_ENTRY(helper) h_next; }; .Ed .Pp Modules must instantiate a .Vt struct helper , but are only required to set the .Va h_classes field, and may optionally set the .Va h_flags , .Va mod_init and .Va mod_destroy fields where required. The framework takes care of all other fields and modules should refrain from manipulating them. Using the C99 designated initialiser feature to set fields is encouraged. .Pp If specified, the .Va mod_init function will be run by the .Nm framework prior to completing the registration process. Returning a non-zero value from the .Va mod_init function will abort the registration process and fail to load the module. If specified, the .Va mod_destroy function will be run by the .Nm framework during the deregistration process, after the module has been deregistered by the .Nm framework. The return value is currently ignored. Valid .Nm classes are defined in .In sys/khelp.h . Valid flags are defined in .In sys/module_khelp.h . The HELPER_NEEDS_OSD flag should be set in the .Va h_flags field if the .Nm module requires persistent per-object data storage. There is no programmatic way (yet) to check if a .Nm class provides the ability for .Nm modules to associate persistent per-object data, so a manual check is required. .Pp The .Fn KHELP_DECLARE_MOD and .Fn KHELP_DECLARE_MOD_UMA macros provide convenient wrappers around the .Xr DECLARE_MODULE 9 macro, and are used to register a .Nm module with the .Nm framework. .Fn KHELP_DECLARE_MOD_UMA should only be used by modules which require the use of persistent per-object storage i.e. modules which set the HELPER_NEEDS_OSD flag in their .Vt struct helper Ns 's .Va h_flags field. .Pp The first four arguments common to both macros are as follows. The .Fa hname argument specifies the unique .Xr ascii 7 name for the .Nm module. It should be no longer than HELPER_NAME_MAXLEN-1 characters in length. The .Fa hdata argument is a pointer to the module's .Vt struct helper . The .Fa hhooks argument points to a static array of .Vt struct hookinfo structures. The array should contain a .Vt struct hookinfo for each .Xr hhook 9 point the module wishes to hook, even when using the same hook function multiple times for different .Xr hhook 9 points. The .Fa version argument specifies a version number for the module which will be passed to .Xr MODULE_VERSION 9 . The .Fn KHELP_DECLARE_MOD_UMA macro takes the additional .Fa ctor and .Fa dtor arguments, which specify optional .Xr uma 9 constructor and destructor functions. NULL should be passed where the functionality is not required. .Pp The .Fn khelp_get_id function returns the numeric identifier for the .Nm module with name .Fa hname . .Pp The .Fn khelp_get_osd function is used to obtain the per-object data pointer for a specified .Nm module. The .Fa hosd argument is a pointer to the underlying subsystem object's .Vt struct osd . This is provided by the .Xr hhook 9 framework when calling into a .Nm module's hook function. The .Fa id argument specifies the numeric identifier for the .Nm module to extract the data pointer from .Fa hosd for. The .Fa id is obtained using the .Fn khelp_get_id function. .Pp The .Fn khelp_add_hhook and .Fn khelp_remove_hhook functions allow a .Nm module to dynamically hook/unhook .Xr hhook 9 points at run time. The .Fa hki argument specifies a pointer to a .Vt struct hookinfo which encapsulates the required information about the .Xr hhook 9 point and hook function being manipulated. The HHOOK_WAITOK flag may be passed in via the .Fa flags argument of .Fn khelp_add_hhook if .Xr malloc 9 is allowed to sleep waiting for memory to become available. .Ss Integrating Khelp Into a Kernel Subsystem Most of the work required to allow .Nm modules to do useful things relates to defining and instantiating suitable .Xr hhook 9 points for .Nm modules to hook into. The only additional decision a subsystem needs to make is whether it wants to allow .Nm modules to associate persistent per-object data. Providing support for persistent data storage can allow .Nm modules to perform more complex functionality which may be desirable. Subsystems which want to allow Khelp modules to associate persistent per-object data with one of the subsystem's data structures need to make the following two key changes: .Bl -bullet .It Embed a .Vt struct osd pointer in the structure definition for the object. .It Add calls to .Fn khelp_init_osd and .Fn khelp_destroy_osd to the subsystem code paths which are responsible for respectively initialising and destroying the object. .El .Pp The .Fn khelp_init_osd function initialises the per-object data storage for all currently loaded .Nm modules of appropriate classes which have set the HELPER_NEEDS_OSD flag in their .Va h_flags field. The .Fa classes argument specifies a bitmask of .Nm classes which this subsystem associates with. If a .Nm module matches any of the classes in the bitmask, that module will be associated with the object. The .Fa hosd argument specifies the pointer to the object's .Vt struct osd which will be used to provide the persistent storage for use by .Nm modules. .Pp The .Fn khelp_destroy_osd function frees all memory that was associated with an object's .Vt struct osd by a previous call to .Fn khelp_init_osd . The .Fa hosd argument specifies the pointer to the object's .Vt struct osd which will be purged in preparation for destruction. .Sh IMPLEMENTATION NOTES .Nm modules are protected from being prematurely unloaded by a reference count. The count is incremented each time a subsystem calls .Fn khelp_init_osd causing persistent storage to be allocated for the module, and decremented for each corresponding call to .Fn khelp_destroy_osd . Only when a module's reference count has dropped to zero can the module be unloaded. .Sh RETURN VALUES The .Fn khelp_init_osd function returns zero if no errors occurred. It returns ENOMEM if a .Nm module which requires per-object storage fails to allocate the necessary memory. .Pp The .Fn khelp_destroy_osd function only returns zero to indicate that no errors occurred. .Pp The .Fn khelp_get_id function returns the unique numeric identifier for the registered .Nm module with name .Fa hname . It return -1 if no module with the specified name is currently registered. .Pp The .Fn khelp_get_osd function returns the pointer to the .Nm module's persistent object storage memory. If the module identified by .Fa id does not have persistent object storage registered with the object's .Fa hosd .Vt struct osd , NULL is returned. .Pp The .Fn khelp_add_hhook function returns zero if no errors occurred. It returns ENOENT if it could not find the requested .Xr hhook 9 point. It returns ENOMEM if .Xr malloc 9 failed to allocate memory. It returns EEXIST if attempting to register the same hook function more than once for the same .Xr hhook 9 point. .Pp The .Fn khelp_remove_hhook function returns zero if no errors occurred. It returns ENOENT if it could not find the requested .Xr hhook 9 point. .Sh EXAMPLES A well commented example Khelp module can be found at: .Pa /usr/share/examples/kld/khelp/h_example.c .Pp The Enhanced Round Trip Time (ERTT) .Xr h_ertt 4 .Nm module provides a more complex example of what is possible. .Sh SEE ALSO .Xr h_ertt 4 , .Xr hhook 9 , .Xr osd 9 .Sh ACKNOWLEDGEMENTS Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley. .Sh HISTORY The .Nm kernel helper framework first appeared in .Fx 9.0 . .Pp The .Nm framework was first released in 2010 by Lawrence Stewart whilst studying at -Swinburne University's Centre for Advanced Internet Architectures, Melbourne, -Australia. +Swinburne University of Technology's Centre for Advanced Internet Architectures, +Melbourne, Australia. More details are available at: .Pp http://caia.swin.edu.au/urp/newtcp/ .Sh AUTHORS .An -nosplit The .Nm framework was written by .An Lawrence Stewart Aq lstewart@FreeBSD.org . .Pp This manual page was written by .An David Hayes Aq david.hayes@ieee.org and .An Lawrence Stewart Aq lstewart@FreeBSD.org . Index: head/sys/kern/kern_hhook.c =================================================================== --- head/sys/kern/kern_hhook.c (revision 220559) +++ head/sys/kern/kern_hhook.c (revision 220560) @@ -1,455 +1,455 @@ /*- * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by grants from the FreeBSD Foundation and Cisco University Research - * Program Fund at Community Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, + * made possible in part by grants from the FreeBSD Foundation and Cisco + * University Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include struct hhook { hhook_func_t hhk_func; struct helper *hhk_helper; void *hhk_udata; STAILQ_ENTRY(hhook) hhk_next; }; MALLOC_DECLARE(M_HHOOK); MALLOC_DEFINE(M_HHOOK, "hhook", "Helper hooks are linked off hhook_head lists"); LIST_HEAD(hhookheadhead, hhook_head); VNET_DEFINE(struct hhookheadhead, hhook_head_list); #define V_hhook_head_list VNET(hhook_head_list) static struct mtx hhook_head_list_lock; MTX_SYSINIT(hhookheadlistlock, &hhook_head_list_lock, "hhook_head list lock", MTX_DEF); /* Private function prototypes. */ static void hhook_head_destroy(struct hhook_head *hhh); #define HHHLIST_LOCK() mtx_lock(&hhook_head_list_lock) #define HHHLIST_UNLOCK() mtx_unlock(&hhook_head_list_lock) #define HHHLIST_LOCK_ASSERT() mtx_assert(&hhook_head_list_lock, MA_OWNED) #define HHH_LOCK_INIT(hhh) rm_init(&(hhh)->hhh_lock, "hhook_head rm lock") #define HHH_LOCK_DESTROY(hhh) rm_destroy(&(hhh)->hhh_lock) #define HHH_WLOCK(hhh) rm_wlock(&(hhh)->hhh_lock) #define HHH_WUNLOCK(hhh) rm_wunlock(&(hhh)->hhh_lock) #define HHH_RLOCK(hhh, rmpt) rm_rlock(&(hhh)->hhh_lock, (rmpt)) #define HHH_RUNLOCK(hhh, rmpt) rm_runlock(&(hhh)->hhh_lock, (rmpt)) /* * Run all helper hook functions for a given hook point. */ void hhook_run_hooks(struct hhook_head *hhh, void *ctx_data, struct osd *hosd) { struct hhook *hhk; void *hdata; struct rm_priotracker rmpt; KASSERT(hhh->hhh_refcount > 0, ("hhook_head %p refcount is 0", hhh)); HHH_RLOCK(hhh, &rmpt); STAILQ_FOREACH(hhk, &hhh->hhh_hooks, hhk_next) { if (hhk->hhk_helper->h_flags & HELPER_NEEDS_OSD) { hdata = osd_get(OSD_KHELP, hosd, hhk->hhk_helper->h_id); if (hdata == NULL) continue; } else hdata = NULL; /* * XXXLAS: We currently ignore the int returned by the hook, * but will likely want to handle it in future to allow hhook to * be used like pfil and effect changes at the hhook calling * site e.g. we could define a new hook type of HHOOK_TYPE_PFIL * and standardise what particular return values mean and set * the context data to pass exactly the same information as pfil * hooks currently receive, thus replicating pfil with hhook. */ hhk->hhk_func(hhh->hhh_type, hhh->hhh_id, hhk->hhk_udata, ctx_data, hdata, hosd); } HHH_RUNLOCK(hhh, &rmpt); } /* * Register a new helper hook function with a helper hook point. */ int hhook_add_hook(struct hhook_head *hhh, struct hookinfo *hki, uint32_t flags) { struct hhook *hhk, *tmp; int error; error = 0; if (hhh == NULL) return (ENOENT); hhk = malloc(sizeof(struct hhook), M_HHOOK, M_ZERO | ((flags & HHOOK_WAITOK) ? M_WAITOK : M_NOWAIT)); if (hhk == NULL) return (ENOMEM); hhk->hhk_helper = hki->hook_helper; hhk->hhk_func = hki->hook_func; hhk->hhk_udata = hki->hook_udata; HHH_WLOCK(hhh); STAILQ_FOREACH(tmp, &hhh->hhh_hooks, hhk_next) { if (tmp->hhk_func == hki->hook_func && tmp->hhk_udata == hki->hook_udata) { /* The helper hook function is already registered. */ error = EEXIST; break; } } if (!error) { STAILQ_INSERT_TAIL(&hhh->hhh_hooks, hhk, hhk_next); hhh->hhh_nhooks++; } else free(hhk, M_HHOOK); HHH_WUNLOCK(hhh); return (error); } /* * Lookup a helper hook point and register a new helper hook function with it. */ int hhook_add_hook_lookup(struct hookinfo *hki, uint32_t flags) { struct hhook_head *hhh; int error; hhh = hhook_head_get(hki->hook_type, hki->hook_id); if (hhh == NULL) return (ENOENT); error = hhook_add_hook(hhh, hki, flags); hhook_head_release(hhh); return (error); } /* * Remove a helper hook function from a helper hook point. */ int hhook_remove_hook(struct hhook_head *hhh, struct hookinfo *hki) { struct hhook *tmp; if (hhh == NULL) return (ENOENT); HHH_WLOCK(hhh); STAILQ_FOREACH(tmp, &hhh->hhh_hooks, hhk_next) { if (tmp->hhk_func == hki->hook_func && tmp->hhk_udata == hki->hook_udata) { STAILQ_REMOVE(&hhh->hhh_hooks, tmp, hhook, hhk_next); free(tmp, M_HHOOK); hhh->hhh_nhooks--; break; } } HHH_WUNLOCK(hhh); return (0); } /* * Lookup a helper hook point and remove a helper hook function from it. */ int hhook_remove_hook_lookup(struct hookinfo *hki) { struct hhook_head *hhh; hhh = hhook_head_get(hki->hook_type, hki->hook_id); if (hhh == NULL) return (ENOENT); hhook_remove_hook(hhh, hki); hhook_head_release(hhh); return (0); } /* * Register a new helper hook point. */ int hhook_head_register(int32_t hhook_type, int32_t hhook_id, struct hhook_head **hhh, uint32_t flags) { struct hhook_head *tmphhh; tmphhh = hhook_head_get(hhook_type, hhook_id); if (tmphhh != NULL) { /* Hook point previously registered. */ hhook_head_release(tmphhh); return (EEXIST); } /* XXXLAS: Need to implement support for non-virtualised hooks. */ if ((flags & HHOOK_HEADISINVNET) == 0) { printf("%s: only vnet-style virtualised hooks can be used\n", __func__); return (EINVAL); } tmphhh = malloc(sizeof(struct hhook_head), M_HHOOK, M_ZERO | ((flags & HHOOK_WAITOK) ? M_WAITOK : M_NOWAIT)); if (tmphhh == NULL) return (ENOMEM); tmphhh->hhh_type = hhook_type; tmphhh->hhh_id = hhook_id; tmphhh->hhh_nhooks = 0; STAILQ_INIT(&tmphhh->hhh_hooks); HHH_LOCK_INIT(tmphhh); if (hhh != NULL) refcount_init(&tmphhh->hhh_refcount, 1); else refcount_init(&tmphhh->hhh_refcount, 0); if (flags & HHOOK_HEADISINVNET) { tmphhh->hhh_flags |= HHH_ISINVNET; HHHLIST_LOCK(); LIST_INSERT_HEAD(&V_hhook_head_list, tmphhh, hhh_next); HHHLIST_UNLOCK(); } else { /* XXXLAS: Add tmphhh to the non-virtualised list. */ } *hhh = tmphhh; return (0); } static void hhook_head_destroy(struct hhook_head *hhh) { struct hhook *tmp, *tmp2; HHHLIST_LOCK_ASSERT(); LIST_REMOVE(hhh, hhh_next); HHH_WLOCK(hhh); STAILQ_FOREACH_SAFE(tmp, &hhh->hhh_hooks, hhk_next, tmp2) free(tmp, M_HHOOK); HHH_WUNLOCK(hhh); HHH_LOCK_DESTROY(hhh); free(hhh, M_HHOOK); } /* * Remove a helper hook point. */ int hhook_head_deregister(struct hhook_head *hhh) { int error; error = 0; HHHLIST_LOCK(); if (hhh == NULL) error = ENOENT; else if (hhh->hhh_refcount > 1) error = EBUSY; else hhook_head_destroy(hhh); HHHLIST_UNLOCK(); return (error); } /* * Remove a helper hook point via a hhook_head lookup. */ int hhook_head_deregister_lookup(int32_t hhook_type, int32_t hhook_id) { struct hhook_head *hhh; int error; hhh = hhook_head_get(hhook_type, hhook_id); error = hhook_head_deregister(hhh); if (error == EBUSY) hhook_head_release(hhh); return (error); } /* * Lookup and return the hhook_head struct associated with the specified type * and id, or NULL if not found. If found, the hhook_head's refcount is bumped. */ struct hhook_head * hhook_head_get(int32_t hhook_type, int32_t hhook_id) { struct hhook_head *hhh; /* XXXLAS: Pick hhook_head_list based on hhook_head flags. */ HHHLIST_LOCK(); LIST_FOREACH(hhh, &V_hhook_head_list, hhh_next) { if (hhh->hhh_type == hhook_type && hhh->hhh_id == hhook_id) { refcount_acquire(&hhh->hhh_refcount); break; } } HHHLIST_UNLOCK(); return (hhh); } void hhook_head_release(struct hhook_head *hhh) { refcount_release(&hhh->hhh_refcount); } /* * Check the hhook_head private flags and return the appropriate public * representation of the flag to the caller. The function is implemented in a * way that allows us to cope with other subsystems becoming virtualised in the * future. */ uint32_t hhook_head_is_virtualised(struct hhook_head *hhh) { uint32_t ret; ret = 0; if (hhh != NULL) { if (hhh->hhh_flags & HHH_ISINVNET) ret = HHOOK_HEADISINVNET; } return (ret); } uint32_t hhook_head_is_virtualised_lookup(int32_t hook_type, int32_t hook_id) { struct hhook_head *hhh; uint32_t ret; hhh = hhook_head_get(hook_type, hook_id); if (hhh == NULL) return (0); ret = hhook_head_is_virtualised(hhh); hhook_head_release(hhh); return (ret); } /* * Vnet created and being initialised. */ static void hhook_vnet_init(const void *unused __unused) { LIST_INIT(&V_hhook_head_list); } /* * Vnet being torn down and destroyed. */ static void hhook_vnet_uninit(const void *unused __unused) { struct hhook_head *hhh, *tmphhh; /* * If subsystems which export helper hook points use the hhook KPI * correctly, the loop below should have no work to do because the * subsystem should have already called hhook_head_deregister(). */ HHHLIST_LOCK(); LIST_FOREACH_SAFE(hhh, &V_hhook_head_list, hhh_next, tmphhh) { printf("%s: hhook_head type=%d, id=%d cleanup required\n", __func__, hhh->hhh_type, hhh->hhh_id); hhook_head_destroy(hhh); } HHHLIST_UNLOCK(); } /* * When a vnet is created and being initialised, init the V_hhook_head_list. */ VNET_SYSINIT(hhook_vnet_init, SI_SUB_PROTO_BEGIN, SI_ORDER_FIRST, hhook_vnet_init, NULL); /* * The hhook KPI provides a mechanism for subsystems which export helper hook * points to clean up on vnet tear down, but in case the KPI is misused, * provide a function to clean up and free memory for a vnet being destroyed. */ VNET_SYSUNINIT(hhook_vnet_uninit, SI_SUB_PROTO_BEGIN, SI_ORDER_FIRST, hhook_vnet_uninit, NULL); Index: head/sys/kern/kern_khelp.c =================================================================== --- head/sys/kern/kern_khelp.c (revision 220559) +++ head/sys/kern/kern_khelp.c (revision 220560) @@ -1,471 +1,471 @@ /*- * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by grants from the FreeBSD Foundation and Cisco University Research - * Program Fund at Community Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, + * made possible in part by grants from the FreeBSD Foundation and Cisco + * University Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static struct rwlock khelp_list_lock; RW_SYSINIT(khelplistlock, &khelp_list_lock, "helper list lock"); static TAILQ_HEAD(helper_head, helper) helpers = TAILQ_HEAD_INITIALIZER(helpers); /* Private function prototypes. */ static inline void khelp_remove_osd(struct helper *h, struct osd *hosd); #define KHELP_LIST_WLOCK() rw_wlock(&khelp_list_lock) #define KHELP_LIST_WUNLOCK() rw_wunlock(&khelp_list_lock) #define KHELP_LIST_RLOCK() rw_rlock(&khelp_list_lock) #define KHELP_LIST_RUNLOCK() rw_runlock(&khelp_list_lock) #define KHELP_LIST_LOCK_ASSERT() rw_assert(&khelp_list_lock, RA_LOCKED) int khelp_register_helper(struct helper *h) { struct helper *tmph; int error, i, inserted; error = 0; inserted = 0; refcount_init(&h->h_refcount, 0); h->h_id = osd_register(OSD_KHELP, NULL, NULL); /* It's only safe to add the hooks after osd_register(). */ if (h->h_nhooks > 0) { for (i = 0; i < h->h_nhooks && !error; i++) { /* We don't require the module to assign hook_helper. */ h->h_hooks[i].hook_helper = h; error = khelp_add_hhook(&h->h_hooks[i], HHOOK_NOWAIT); } if (error) { for (i--; i >= 0; i--) khelp_remove_hhook(&h->h_hooks[i]); osd_deregister(OSD_KHELP, h->h_id); } } if (!error) { KHELP_LIST_WLOCK(); /* * Keep list of helpers sorted in descending h_id order. Due to * the way osd_set() works, a sorted list ensures * init_helper_osd() will operate with improved efficiency. */ TAILQ_FOREACH(tmph, &helpers, h_next) { if (tmph->h_id < h->h_id) { TAILQ_INSERT_BEFORE(tmph, h, h_next); inserted = 1; break; } } if (!inserted) TAILQ_INSERT_TAIL(&helpers, h, h_next); KHELP_LIST_WUNLOCK(); } return (error); } int khelp_deregister_helper(struct helper *h) { struct helper *tmph; int error, i; error = 0; KHELP_LIST_WLOCK(); if (h->h_refcount > 0) error = EBUSY; else { error = ENOENT; TAILQ_FOREACH(tmph, &helpers, h_next) { if (tmph == h) { TAILQ_REMOVE(&helpers, h, h_next); error = 0; break; } } } KHELP_LIST_WUNLOCK(); if (!error) { if (h->h_nhooks > 0) { for (i = 0; i < h->h_nhooks; i++) khelp_remove_hhook(&h->h_hooks[i]); } osd_deregister(OSD_KHELP, h->h_id); } return (error); } int khelp_init_osd(uint32_t classes, struct osd *hosd) { struct helper *h; void *hdata; int error; KASSERT(hosd != NULL, ("struct osd not initialised!")); error = 0; KHELP_LIST_RLOCK(); TAILQ_FOREACH(h, &helpers, h_next) { /* If helper is correct class and needs to store OSD... */ if (h->h_classes & classes && h->h_flags & HELPER_NEEDS_OSD) { hdata = uma_zalloc(h->h_zone, M_NOWAIT); if (hdata == NULL) { error = ENOMEM; break; } osd_set(OSD_KHELP, hosd, h->h_id, hdata); refcount_acquire(&h->h_refcount); } } if (error) { /* Delete OSD that was assigned prior to the error. */ TAILQ_FOREACH(h, &helpers, h_next) { if (h->h_classes & classes) khelp_remove_osd(h, hosd); } } KHELP_LIST_RUNLOCK(); return (error); } int khelp_destroy_osd(struct osd *hosd) { struct helper *h; int error; KASSERT(hosd != NULL, ("struct osd not initialised!")); error = 0; KHELP_LIST_RLOCK(); /* * Clean up all khelp related OSD. * * XXXLAS: Would be nice to use something like osd_exit() here but it * doesn't have the right semantics for this purpose. */ TAILQ_FOREACH(h, &helpers, h_next) khelp_remove_osd(h, hosd); KHELP_LIST_RUNLOCK(); return (error); } static inline void khelp_remove_osd(struct helper *h, struct osd *hosd) { void *hdata; if (h->h_flags & HELPER_NEEDS_OSD) { /* * If the current helper uses OSD and calling osd_get() * on the helper's h_id returns non-NULL, the helper has * OSD attached to 'hosd' which needs to be cleaned up. */ hdata = osd_get(OSD_KHELP, hosd, h->h_id); if (hdata != NULL) { uma_zfree(h->h_zone, hdata); osd_del(OSD_KHELP, hosd, h->h_id); refcount_release(&h->h_refcount); } } } void * khelp_get_osd(struct osd *hosd, int32_t id) { return (osd_get(OSD_KHELP, hosd, id)); } int32_t khelp_get_id(char *hname) { struct helper *h; int32_t id; id = -1; KHELP_LIST_RLOCK(); TAILQ_FOREACH(h, &helpers, h_next) { if (strncmp(h->h_name, hname, HELPER_NAME_MAXLEN) == 0) { id = h->h_id; break; } } KHELP_LIST_RUNLOCK(); return (id); } int khelp_add_hhook(struct hookinfo *hki, uint32_t flags) { VNET_ITERATOR_DECL(vnet_iter); int error; error = 0; /* * XXXLAS: If a helper is dynamically adding a helper hook function at * runtime using this function, we should update the helper's h_hooks * struct member to include the additional hookinfo struct. */ VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); error = hhook_add_hook_lookup(hki, flags); CURVNET_RESTORE(); #ifdef VIMAGE if (error) break; #endif } VNET_LIST_RUNLOCK_NOSLEEP(); return (error); } int khelp_remove_hhook(struct hookinfo *hki) { VNET_ITERATOR_DECL(vnet_iter); int error; error = 0; /* * XXXLAS: If a helper is dynamically removing a helper hook function at * runtime using this function, we should update the helper's h_hooks * struct member to remove the defunct hookinfo struct. */ VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); error = hhook_remove_hook_lookup(hki); CURVNET_RESTORE(); #ifdef VIMAGE if (error) break; #endif } VNET_LIST_RUNLOCK_NOSLEEP(); return (error); } int khelp_modevent(module_t mod, int event_type, void *data) { struct khelp_modevent_data *kmd; int error; kmd = (struct khelp_modevent_data *)data; error = 0; switch(event_type) { case MOD_LOAD: if (kmd->helper->h_flags & HELPER_NEEDS_OSD) { if (kmd->uma_zsize <= 0) { printf("Use KHELP_DECLARE_MOD_UMA() instead!\n"); error = EDOOFUS; break; } kmd->helper->h_zone = uma_zcreate(kmd->name, kmd->uma_zsize, kmd->umactor, kmd->umadtor, NULL, NULL, 0, 0); if (kmd->helper->h_zone == NULL) { error = ENOMEM; break; } } strlcpy(kmd->helper->h_name, kmd->name, HELPER_NAME_MAXLEN); kmd->helper->h_hooks = kmd->hooks; kmd->helper->h_nhooks = kmd->nhooks; if (kmd->helper->mod_init != NULL) error = kmd->helper->mod_init(); if (!error) error = khelp_register_helper(kmd->helper); break; case MOD_QUIESCE: case MOD_SHUTDOWN: case MOD_UNLOAD: error = khelp_deregister_helper(kmd->helper); if (!error) { if (kmd->helper->h_flags & HELPER_NEEDS_OSD) uma_zdestroy(kmd->helper->h_zone); if (kmd->helper->mod_destroy != NULL) kmd->helper->mod_destroy(); } else if (error == ENOENT) /* Do nothing and allow unload if helper not in list. */ error = 0; else if (error == EBUSY) printf("Khelp module \"%s\" can't unload until its " "refcount drops from %d to 0.\n", kmd->name, kmd->helper->h_refcount); break; default: error = EINVAL; break; } return (error); } /* * This function is called in two separate situations: * * - When the kernel is booting, it is called directly by the SYSINIT framework * to allow Khelp modules which were compiled into the kernel or loaded by the * boot loader to insert their non-virtualised hook functions into the kernel. * * - When the kernel is booting or a vnet is created, this function is also * called indirectly through khelp_vnet_init() by the vnet initialisation code. * In this situation, Khelp modules are able to insert their virtualised hook * functions into the virtualised hook points in the vnet which is being * initialised. In the case where the kernel is not compiled with "options * VIMAGE", this step is still run once at boot, but the hook functions get * transparently inserted into the standard unvirtualised network stack. */ static void khelp_init(const void *vnet) { struct helper *h; int error, i, vinit; int32_t htype, hid; error = 0; vinit = vnet != NULL; KHELP_LIST_RLOCK(); TAILQ_FOREACH(h, &helpers, h_next) { for (i = 0; i < h->h_nhooks && !error; i++) { htype = h->h_hooks[i].hook_type; hid = h->h_hooks[i].hook_id; /* * If we're doing a virtualised init (vinit != 0) and * the hook point is virtualised, or we're doing a plain * sysinit at boot and the hook point is not * virtualised, insert the hook. */ if ((hhook_head_is_virtualised_lookup(htype, hid) == HHOOK_HEADISINVNET && vinit) || (!hhook_head_is_virtualised_lookup(htype, hid) && !vinit)) { error = hhook_add_hook_lookup(&h->h_hooks[i], HHOOK_NOWAIT); } } if (error) { /* Remove any helper's hooks we successfully added. */ for (i--; i >= 0; i--) hhook_remove_hook_lookup(&h->h_hooks[i]); printf("%s: Failed to add hooks for helper \"%s\" (%p)", __func__, h->h_name, h); if (vinit) printf(" to vnet %p.\n", vnet); else printf(".\n"); error = 0; } } KHELP_LIST_RUNLOCK(); } /* * Vnet created and being initialised. */ static void khelp_vnet_init(const void *unused __unused) { khelp_init(TD_TO_VNET(curthread)); } /* * As the kernel boots, allow Khelp modules which were compiled into the kernel * or loaded by the boot loader to insert their non-virtualised hook functions * into the kernel. */ SYSINIT(khelp_init, SI_SUB_PROTO_END, SI_ORDER_FIRST, khelp_init, NULL); /* * When a vnet is created and being initialised, we need to insert the helper * hook functions for all currently registered Khelp modules into the vnet's * helper hook points. The hhook KPI provides a mechanism for subsystems which * export helper hook points to clean up on vnet shutdown, so we don't need a * VNET_SYSUNINIT for Khelp. */ VNET_SYSINIT(khelp_vnet_init, SI_SUB_PROTO_END, SI_ORDER_FIRST, khelp_vnet_init, NULL); Index: head/sys/netinet/cc/cc.c =================================================================== --- head/sys/netinet/cc/cc.c (revision 220559) +++ head/sys/netinet/cc/cc.c (revision 220560) @@ -1,325 +1,326 @@ /*- * Copyright (c) 2007-2008 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by Lawrence Stewart and James Healy, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by Lawrence Stewart and + * James Healy, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * This software was first released in 2007 by James Healy and Lawrence Stewart - * whilst working on the NewTCP research project at Swinburne University's - * Centre for Advanced Internet Architectures, Melbourne, Australia, which was - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. More details are available at: + * whilst working on the NewTCP research project at Swinburne University of + * Technology's Centre for Advanced Internet Architectures, Melbourne, + * Australia, which was made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. + * More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * List of available cc algorithms on the current system. First element * is used as the system default CC algorithm. */ struct cc_head cc_list = STAILQ_HEAD_INITIALIZER(cc_list); /* Protects the cc_list TAILQ. */ struct rwlock cc_list_lock; VNET_DEFINE(struct cc_algo *, default_cc_ptr) = &newreno_cc_algo; /* * Sysctl handler to show and change the default CC algorithm. */ static int cc_default_algo(SYSCTL_HANDLER_ARGS) { char default_cc[TCP_CA_NAME_MAX]; struct cc_algo *funcs; int err, found; err = found = 0; if (req->newptr == NULL) { /* Just print the current default. */ CC_LIST_RLOCK(); strlcpy(default_cc, CC_DEFAULT()->name, TCP_CA_NAME_MAX); CC_LIST_RUNLOCK(); err = sysctl_handle_string(oidp, default_cc, 1, req); } else { /* Find algo with specified name and set it to default. */ CC_LIST_RLOCK(); STAILQ_FOREACH(funcs, &cc_list, entries) { if (strncmp((char *)req->newptr, funcs->name, TCP_CA_NAME_MAX) == 0) { found = 1; V_default_cc_ptr = funcs; } } CC_LIST_RUNLOCK(); if (!found) err = ESRCH; } return (err); } /* * Sysctl handler to display the list of available CC algorithms. */ static int cc_list_available(SYSCTL_HANDLER_ARGS) { struct cc_algo *algo; struct sbuf *s; int err, first, nalgos; err = nalgos = 0; first = 1; CC_LIST_RLOCK(); STAILQ_FOREACH(algo, &cc_list, entries) { nalgos++; } CC_LIST_RUNLOCK(); s = sbuf_new(NULL, NULL, nalgos * TCP_CA_NAME_MAX, SBUF_FIXEDLEN); if (s == NULL) return (ENOMEM); /* * It is theoretically possible for the CC list to have grown in size * since the call to sbuf_new() and therefore for the sbuf to be too * small. If this were to happen (incredibly unlikely), the sbuf will * reach an overflow condition, sbuf_printf() will return an error and * the sysctl will fail gracefully. */ CC_LIST_RLOCK(); STAILQ_FOREACH(algo, &cc_list, entries) { err = sbuf_printf(s, first ? "%s" : ", %s", algo->name); if (err) { /* Sbuf overflow condition. */ err = EOVERFLOW; break; } first = 0; } CC_LIST_RUNLOCK(); if (!err) { sbuf_finish(s); err = sysctl_handle_string(oidp, sbuf_data(s), 1, req); } sbuf_delete(s); return (err); } /* * Reset the default CC algo to NewReno for any netstack which is using the algo * that is about to go away as its default. */ static void cc_checkreset_default(struct cc_algo *remove_cc) { VNET_ITERATOR_DECL(vnet_iter); CC_LIST_LOCK_ASSERT(); VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); if (strncmp(CC_DEFAULT()->name, remove_cc->name, TCP_CA_NAME_MAX) == 0) V_default_cc_ptr = &newreno_cc_algo; CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); } /* * Initialise CC subsystem on system boot. */ static void cc_init(void) { CC_LIST_LOCK_INIT(); STAILQ_INIT(&cc_list); } /* * Returns non-zero on success, 0 on failure. */ int cc_deregister_algo(struct cc_algo *remove_cc) { struct cc_algo *funcs, *tmpfuncs; int err; err = ENOENT; /* Never allow newreno to be deregistered. */ if (&newreno_cc_algo == remove_cc) return (EPERM); /* Remove algo from cc_list so that new connections can't use it. */ CC_LIST_WLOCK(); STAILQ_FOREACH_SAFE(funcs, &cc_list, entries, tmpfuncs) { if (funcs == remove_cc) { cc_checkreset_default(remove_cc); STAILQ_REMOVE(&cc_list, funcs, cc_algo, entries); err = 0; break; } } CC_LIST_WUNLOCK(); if (!err) /* * XXXLAS: * - We may need to handle non-zero return values in future. * - If we add CC framework support for protocols other than * TCP, we may want a more generic way to handle this step. */ tcp_ccalgounload(remove_cc); return (err); } /* * Returns 0 on success, non-zero on failure. */ int cc_register_algo(struct cc_algo *add_cc) { struct cc_algo *funcs; int err; err = 0; /* * Iterate over list of registered CC algorithms and make sure * we're not trying to add a duplicate. */ CC_LIST_WLOCK(); STAILQ_FOREACH(funcs, &cc_list, entries) { if (funcs == add_cc || strncmp(funcs->name, add_cc->name, TCP_CA_NAME_MAX) == 0) err = EEXIST; } if (!err) STAILQ_INSERT_TAIL(&cc_list, add_cc, entries); CC_LIST_WUNLOCK(); return (err); } /* * Handles kld related events. Returns 0 on success, non-zero on failure. */ int cc_modevent(module_t mod, int event_type, void *data) { struct cc_algo *algo; int err; err = 0; algo = (struct cc_algo *)data; switch(event_type) { case MOD_LOAD: if (algo->mod_init != NULL) err = algo->mod_init(); if (!err) err = cc_register_algo(algo); break; case MOD_QUIESCE: case MOD_SHUTDOWN: case MOD_UNLOAD: err = cc_deregister_algo(algo); if (!err && algo->mod_destroy != NULL) algo->mod_destroy(); if (err == ENOENT) err = 0; break; default: err = EINVAL; break; } return (err); } SYSINIT(cc, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_FIRST, cc_init, NULL); /* Declare sysctl tree and populate it. */ SYSCTL_NODE(_net_inet_tcp, OID_AUTO, cc, CTLFLAG_RW, NULL, "congestion control related settings"); SYSCTL_VNET_PROC(_net_inet_tcp_cc, OID_AUTO, algorithm, CTLTYPE_STRING|CTLFLAG_RW, NULL, 0, cc_default_algo, "A", "default congestion control algorithm"); SYSCTL_PROC(_net_inet_tcp_cc, OID_AUTO, available, CTLTYPE_STRING|CTLFLAG_RD, NULL, 0, cc_list_available, "A", "list available congestion control algorithms"); Index: head/sys/netinet/cc/cc_chd.c =================================================================== --- head/sys/netinet/cc/cc_chd.c (revision 220559) +++ head/sys/netinet/cc/cc_chd.c (revision 220560) @@ -1,497 +1,497 @@ /*- * Copyright (c) 2009-2010 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2010-2011 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by David Hayes and Lawrence Stewart, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by David Hayes and + * Lawrence Stewart, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced Internet * Architectures, Swinburne University of Technology, Melbourne, Australia by * David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the CAIA-Hamilton delay based congestion control * algorithm, based on "Improved coexistence and loss tolerance for delay based * TCP congestion control" by D. A. Hayes and G. Armitage., in 35th Annual IEEE * Conference on Local Computer Networks (LCN 2010), Denver, Colorado, USA, * 11-14 October 2010. * * Originally released as part of the NewTCP research project at Swinburne - * University's Centre for Advanced Internet Architectures, Melbourne, - * Australia, which was made possible in part by a grant from the Cisco - * University Research Program Fund at Community Foundation Silicon Valley. More - * details are available at: + * University of Technology's Centre for Advanced Internet Architectures, + * Melbourne, Australia, which was made possible in part by a grant from the + * Cisco University Research Program Fund at Community Foundation Silicon + * Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define CAST_PTR_INT(X) (*((int*)(X))) /* * Private signal type for rate based congestion signal. * See for appropriate bit-range to use for private signals. */ #define CC_CHD_DELAY 0x02000000 /* Largest possible number returned by random(). */ #define RANDOM_MAX INT_MAX static void chd_ack_received(struct cc_var *ccv, uint16_t ack_type); static void chd_cb_destroy(struct cc_var *ccv); static int chd_cb_init(struct cc_var *ccv); static void chd_cong_signal(struct cc_var *ccv, uint32_t signal_type); static void chd_conn_init(struct cc_var *ccv); static int chd_mod_init(void); struct chd { /* * Shadow window - keeps track of what the NewReno congestion window * would have been if delay-based cwnd backoffs had not been made. This * functionality aids coexistence with loss-based TCP flows which may be * sharing links along the path. */ unsigned long shadow_w; /* * Loss-based TCP compatibility flag - When set, it turns on the shadow * window functionality. */ int loss_compete; /* The maximum round trip time seen within a measured rtt period. */ int maxrtt_in_rtt; /* The previous qdly that caused cwnd to backoff. */ int prev_backoff_qdly; }; static int ertt_id; static VNET_DEFINE(uint32_t, chd_qmin) = 5; static VNET_DEFINE(uint32_t, chd_pmax) = 50; static VNET_DEFINE(uint32_t, chd_loss_fair) = 1; static VNET_DEFINE(uint32_t, chd_use_max) = 1; static VNET_DEFINE(uint32_t, chd_qthresh) = 20; #define V_chd_qthresh VNET(chd_qthresh) #define V_chd_qmin VNET(chd_qmin) #define V_chd_pmax VNET(chd_pmax) #define V_chd_loss_fair VNET(chd_loss_fair) #define V_chd_use_max VNET(chd_use_max) MALLOC_DECLARE(M_CHD); MALLOC_DEFINE(M_CHD, "chd data", "Per connection data required for the CHD congestion control algorithm"); struct cc_algo chd_cc_algo = { .name = "chd", .ack_received = chd_ack_received, .cb_destroy = chd_cb_destroy, .cb_init = chd_cb_init, .cong_signal = chd_cong_signal, .conn_init = chd_conn_init, .mod_init = chd_mod_init }; static __inline void chd_window_decrease(struct cc_var *ccv) { unsigned long win; win = min(CCV(ccv, snd_wnd), CCV(ccv, snd_cwnd)) / CCV(ccv, t_maxseg); win -= max((win / 2), 1); CCV(ccv, snd_ssthresh) = max(win, 2) * CCV(ccv, t_maxseg); } /* * Probabilistic backoff function. Returns 1 if we should backoff or 0 * otherwise. The calculation of p is similar to the calculation of p in cc_hd. */ static __inline int should_backoff(int qdly, int maxqdly, struct chd *chd_data) { unsigned long p, rand; rand = random(); if (qdly < V_chd_qthresh) { chd_data->loss_compete = 0; p = (((RANDOM_MAX / 100) * V_chd_pmax) / (V_chd_qthresh - V_chd_qmin)) * (qdly - V_chd_qmin); } else { if (qdly > V_chd_qthresh) { p = (((RANDOM_MAX / 100) * V_chd_pmax) / (maxqdly - V_chd_qthresh)) * (maxqdly - qdly); if (V_chd_loss_fair && rand < p) chd_data->loss_compete = 1; } else { p = (RANDOM_MAX / 100) * V_chd_pmax; chd_data->loss_compete = 0; } } return (rand < p); } static __inline void chd_window_increase(struct cc_var *ccv, int new_measurement) { struct chd *chd_data; int incr; chd_data = ccv->cc_data; incr = 0; if (CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh)) { /* Adapted from NewReno slow start. */ if (V_tcp_do_rfc3465) { /* In slow-start with ABC enabled. */ if (CCV(ccv, snd_nxt) == CCV(ccv, snd_max)) { /* Not due to RTO. */ incr = min(ccv->bytes_this_ack, V_tcp_abc_l_var * CCV(ccv, t_maxseg)); } else { /* Due to RTO. */ incr = min(ccv->bytes_this_ack, CCV(ccv, t_maxseg)); } } else incr = CCV(ccv, t_maxseg); } else { /* Congestion avoidance. */ if (V_tcp_do_rfc3465) { if (ccv->flags & CCF_ABC_SENTAWND) { ccv->flags &= ~CCF_ABC_SENTAWND; incr = CCV(ccv, t_maxseg); } } else if (new_measurement) incr = CCV(ccv, t_maxseg); } if (chd_data->shadow_w > 0) { /* Track NewReno window. */ chd_data->shadow_w = min(chd_data->shadow_w + incr, TCP_MAXWIN << CCV(ccv, snd_scale)); } CCV(ccv,snd_cwnd) = min(CCV(ccv, snd_cwnd) + incr, TCP_MAXWIN << CCV(ccv, snd_scale)); } /* * All ACK signals are used for timing measurements to determine delay-based * congestion. However, window increases are only performed when * ack_type == CC_ACK. */ static void chd_ack_received(struct cc_var *ccv, uint16_t ack_type) { struct chd *chd_data; struct ertt *e_t; int backoff, new_measurement, qdly, rtt; e_t = khelp_get_osd(CCV(ccv, osd), ertt_id); chd_data = ccv->cc_data; new_measurement = e_t->flags & ERTT_NEW_MEASUREMENT; backoff = qdly = 0; chd_data->maxrtt_in_rtt = imax(e_t->rtt, chd_data->maxrtt_in_rtt); if (new_measurement) { /* * There is a new per RTT measurement, so check to see if there * is delay based congestion. */ rtt = V_chd_use_max ? chd_data->maxrtt_in_rtt : e_t->rtt; chd_data->maxrtt_in_rtt = 0; if (rtt && e_t->minrtt && !IN_RECOVERY(CCV(ccv, t_flags))) { qdly = rtt - e_t->minrtt; if (qdly > V_chd_qmin) { /* * Probabilistic delay based congestion * indication. */ backoff = should_backoff(qdly, e_t->maxrtt - e_t->minrtt, chd_data); } else chd_data->loss_compete = 0; } /* Reset per RTT measurement flag to start a new measurement. */ e_t->flags &= ~ERTT_NEW_MEASUREMENT; } if (backoff) { /* * Update shadow_w before delay based backoff. */ if (chd_data->loss_compete || qdly > chd_data->prev_backoff_qdly) { /* * Delay is higher than when we backed off previously, * so it is possible that this flow is competing with * loss based flows. */ chd_data->shadow_w = max(CCV(ccv, snd_cwnd), chd_data->shadow_w); } else { /* * Reset shadow_w, as it is probable that this flow is * not competing with loss based flows at the moment. */ chd_data->shadow_w = 0; } chd_data->prev_backoff_qdly = qdly; /* * Send delay-based congestion signal to the congestion signal * handler. */ chd_cong_signal(ccv, CC_CHD_DELAY); } else if (ack_type == CC_ACK) chd_window_increase(ccv, new_measurement); } static void chd_cb_destroy(struct cc_var *ccv) { if (ccv->cc_data != NULL) free(ccv->cc_data, M_CHD); } static int chd_cb_init(struct cc_var *ccv) { struct chd *chd_data; chd_data = malloc(sizeof(struct chd), M_CHD, M_NOWAIT); if (chd_data == NULL) return (ENOMEM); chd_data->shadow_w = 0; ccv->cc_data = chd_data; return (0); } static void chd_cong_signal(struct cc_var *ccv, uint32_t signal_type) { struct ertt *e_t; struct chd *chd_data; int qdly; e_t = khelp_get_osd(CCV(ccv, osd), ertt_id); chd_data = ccv->cc_data; qdly = imax(e_t->rtt, chd_data->maxrtt_in_rtt) - e_t->minrtt; switch(signal_type) { case CC_CHD_DELAY: chd_window_decrease(ccv); /* Set new ssthresh. */ CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); CCV(ccv, snd_recover) = CCV(ccv, snd_max); ENTER_CONGRECOVERY(CCV(ccv, t_flags)); break; case CC_NDUPACK: /* Packet loss. */ /* * Only react to loss as a congestion signal if qdly > * V_chd_qthresh. If qdly is less than qthresh, presume that * this is a non congestion related loss. If qdly is greater * than qthresh, assume that we are competing with loss based * tcp flows and restore window from any unnecessary backoffs, * before the decrease. */ if (!IN_RECOVERY(CCV(ccv, t_flags)) && qdly > V_chd_qthresh) { if (chd_data->loss_compete) { CCV(ccv, snd_cwnd) = max(CCV(ccv, snd_cwnd), chd_data->shadow_w); } chd_window_decrease(ccv); } else { /* * This loss isn't congestion related, or already * recovering from congestion. */ CCV(ccv, snd_ssthresh) = CCV(ccv, snd_cwnd); CCV(ccv, snd_recover) = CCV(ccv, snd_max); } if (chd_data->shadow_w > 0) { chd_data->shadow_w = max(chd_data->shadow_w / CCV(ccv, t_maxseg) / 2, 2) * CCV(ccv, t_maxseg); } ENTER_FASTRECOVERY(CCV(ccv, t_flags)); break; default: newreno_cc_algo.cong_signal(ccv, signal_type); } } static void chd_conn_init(struct cc_var *ccv) { struct chd *chd_data; chd_data = ccv->cc_data; chd_data->prev_backoff_qdly = 0; chd_data->maxrtt_in_rtt = 0; chd_data->loss_compete = 0; /* * Initialise the shadow_cwnd to be equal to snd_cwnd in case we are * competing with loss based flows from the start. */ chd_data->shadow_w = CCV(ccv, snd_cwnd); } static int chd_mod_init(void) { ertt_id = khelp_get_id("ertt"); if (ertt_id <= 0) { printf("%s: h_ertt module not found\n", __func__); return (ENOENT); } chd_cc_algo.after_idle = newreno_cc_algo.after_idle; chd_cc_algo.post_recovery = newreno_cc_algo.post_recovery; return (0); } static int chd_loss_fair_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_chd_loss_fair; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) > 1) error = EINVAL; else V_chd_loss_fair = new; } return (error); } static int chd_pmax_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_chd_pmax; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) == 0 || CAST_PTR_INT(req->newptr) > 100) error = EINVAL; else V_chd_pmax = new; } return (error); } static int chd_qthresh_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_chd_qthresh; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) <= V_chd_qmin) error = EINVAL; else V_chd_qthresh = new; } return (error); } SYSCTL_DECL(_net_inet_tcp_cc_chd); SYSCTL_NODE(_net_inet_tcp_cc, OID_AUTO, chd, CTLFLAG_RW, NULL, "CAIA Hamilton delay-based congestion control related settings"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_chd, OID_AUTO, loss_fair, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(chd_loss_fair), 1, &chd_loss_fair_handler, "IU", "Flag to enable shadow window functionality."); SYSCTL_VNET_PROC(_net_inet_tcp_cc_chd, OID_AUTO, pmax, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(chd_pmax), 5, &chd_pmax_handler, "IU", "Per RTT maximum backoff probability as a percentage"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_chd, OID_AUTO, queue_threshold, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(chd_qthresh), 20, &chd_qthresh_handler, "IU", "Queueing congestion threshold in ticks"); SYSCTL_VNET_UINT(_net_inet_tcp_cc_chd, OID_AUTO, queue_min, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(chd_qmin), 5, "Minimum queueing delay threshold in ticks"); SYSCTL_VNET_UINT(_net_inet_tcp_cc_chd, OID_AUTO, use_max, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(chd_use_max), 1, "Use the maximum RTT seen within the measurement period (RTT) " "as the basic delay measurement for the algorithm."); DECLARE_CC_MODULE(chd, &chd_cc_algo); MODULE_DEPEND(chd, ertt, 1, 1, 1); Index: head/sys/netinet/cc/cc_cubic.c =================================================================== --- head/sys/netinet/cc/cc_cubic.c (revision 220559) +++ head/sys/netinet/cc/cc_cubic.c (revision 220560) @@ -1,407 +1,407 @@ /*- * Copyright (c) 2008-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the CUBIC congestion control algorithm for FreeBSD, * based on the Internet Draft "draft-rhee-tcpm-cubic-02" by Rhee, Xu and Ha. * Originally released as part of the NewTCP research project at Swinburne - * University's Centre for Advanced Internet Architectures, Melbourne, - * Australia, which was made possible in part by a grant from the Cisco - * University Research Program Fund at Community Foundation Silicon Valley. More - * details are available at: + * University of Technology's Centre for Advanced Internet Architectures, + * Melbourne, Australia, which was made possible in part by a grant from the + * Cisco University Research Program Fund at Community Foundation Silicon + * Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static void cubic_ack_received(struct cc_var *ccv, uint16_t type); static void cubic_cb_destroy(struct cc_var *ccv); static int cubic_cb_init(struct cc_var *ccv); static void cubic_cong_signal(struct cc_var *ccv, uint32_t type); static void cubic_conn_init(struct cc_var *ccv); static int cubic_mod_init(void); static void cubic_post_recovery(struct cc_var *ccv); static void cubic_record_rtt(struct cc_var *ccv); static void cubic_ssthresh_update(struct cc_var *ccv); struct cubic { /* Cubic K in fixed point form with CUBIC_SHIFT worth of precision. */ int64_t K; /* Sum of RTT samples across an epoch in ticks. */ int64_t sum_rtt_ticks; /* cwnd at the most recent congestion event. */ unsigned long max_cwnd; /* cwnd at the previous congestion event. */ unsigned long prev_max_cwnd; /* Number of congestion events. */ uint32_t num_cong_events; /* Minimum observed rtt in ticks. */ int min_rtt_ticks; /* Mean observed rtt between congestion epochs. */ int mean_rtt_ticks; /* ACKs since last congestion event. */ int epoch_ack_count; /* Time of last congestion event in ticks. */ int t_last_cong; }; MALLOC_DECLARE(M_CUBIC); MALLOC_DEFINE(M_CUBIC, "cubic data", "Per connection data required for the CUBIC congestion control algorithm"); struct cc_algo cubic_cc_algo = { .name = "cubic", .ack_received = cubic_ack_received, .cb_destroy = cubic_cb_destroy, .cb_init = cubic_cb_init, .cong_signal = cubic_cong_signal, .conn_init = cubic_conn_init, .mod_init = cubic_mod_init, .post_recovery = cubic_post_recovery, }; static void cubic_ack_received(struct cc_var *ccv, uint16_t type) { struct cubic *cubic_data; unsigned long w_tf, w_cubic_next; int ticks_since_cong; cubic_data = ccv->cc_data; cubic_record_rtt(ccv); /* * Regular ACK and we're not in cong/fast recovery and we're cwnd * limited and we're either not doing ABC or are slow starting or are * doing ABC and we've sent a cwnd's worth of bytes. */ if (type == CC_ACK && !IN_RECOVERY(CCV(ccv, t_flags)) && (ccv->flags & CCF_CWND_LIMITED) && (!V_tcp_do_rfc3465 || CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh) || (V_tcp_do_rfc3465 && ccv->flags & CCF_ABC_SENTAWND))) { /* Use the logic in NewReno ack_received() for slow start. */ if (CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh) || cubic_data->min_rtt_ticks == TCPTV_SRTTBASE) newreno_cc_algo.ack_received(ccv, type); else { ticks_since_cong = ticks - cubic_data->t_last_cong; /* * The mean RTT is used to best reflect the equations in * the I-D. Using min_rtt in the tf_cwnd calculation * causes w_tf to grow much faster than it should if the * RTT is dominated by network buffering rather than * propogation delay. */ w_tf = tf_cwnd(ticks_since_cong, cubic_data->mean_rtt_ticks, cubic_data->max_cwnd, CCV(ccv, t_maxseg)); w_cubic_next = cubic_cwnd(ticks_since_cong + cubic_data->mean_rtt_ticks, cubic_data->max_cwnd, CCV(ccv, t_maxseg), cubic_data->K); ccv->flags &= ~CCF_ABC_SENTAWND; if (w_cubic_next < w_tf) /* * TCP-friendly region, follow tf * cwnd growth. */ CCV(ccv, snd_cwnd) = w_tf; else if (CCV(ccv, snd_cwnd) < w_cubic_next) { /* * Concave or convex region, follow CUBIC * cwnd growth. */ if (V_tcp_do_rfc3465) CCV(ccv, snd_cwnd) = w_cubic_next; else CCV(ccv, snd_cwnd) += ((w_cubic_next - CCV(ccv, snd_cwnd)) * CCV(ccv, t_maxseg)) / CCV(ccv, snd_cwnd); } /* * If we're not in slow start and we're probing for a * new cwnd limit at the start of a connection * (happens when hostcache has a relevant entry), * keep updating our current estimate of the * max_cwnd. */ if (cubic_data->num_cong_events == 0 && cubic_data->max_cwnd < CCV(ccv, snd_cwnd)) cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } } } static void cubic_cb_destroy(struct cc_var *ccv) { if (ccv->cc_data != NULL) free(ccv->cc_data, M_CUBIC); } static int cubic_cb_init(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = malloc(sizeof(struct cubic), M_CUBIC, M_NOWAIT|M_ZERO); if (cubic_data == NULL) return (ENOMEM); /* Init some key variables with sensible defaults. */ cubic_data->t_last_cong = ticks; cubic_data->min_rtt_ticks = TCPTV_SRTTBASE; cubic_data->mean_rtt_ticks = 1; ccv->cc_data = cubic_data; return (0); } /* * Perform any necessary tasks before we enter congestion recovery. */ static void cubic_cong_signal(struct cc_var *ccv, uint32_t type) { struct cubic *cubic_data; cubic_data = ccv->cc_data; switch (type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(CCV(ccv, t_flags))) { if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { cubic_ssthresh_update(ccv); cubic_data->num_cong_events++; cubic_data->prev_max_cwnd = cubic_data->max_cwnd; cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } ENTER_RECOVERY(CCV(ccv, t_flags)); } break; case CC_ECN: if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { cubic_ssthresh_update(ccv); cubic_data->num_cong_events++; cubic_data->prev_max_cwnd = cubic_data->max_cwnd; cubic_data->max_cwnd = CCV(ccv, snd_cwnd); cubic_data->t_last_cong = ticks; CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); ENTER_CONGRECOVERY(CCV(ccv, t_flags)); } break; case CC_RTO: /* * Grab the current time and record it so we know when the * most recent congestion event was. Only record it when the * timeout has fired more than once, as there is a reasonable * chance the first one is a false alarm and may not indicate * congestion. */ if (CCV(ccv, t_rxtshift) >= 2) cubic_data->num_cong_events++; cubic_data->t_last_cong = ticks; break; } } static void cubic_conn_init(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = ccv->cc_data; /* * Ensure we have a sane initial value for max_cwnd recorded. Without * this here bad things happen when entries from the TCP hostcache * get used. */ cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } static int cubic_mod_init(void) { cubic_cc_algo.after_idle = newreno_cc_algo.after_idle; return (0); } /* * Perform any necessary tasks before we exit congestion recovery. */ static void cubic_post_recovery(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = ccv->cc_data; /* Fast convergence heuristic. */ if (cubic_data->max_cwnd < cubic_data->prev_max_cwnd) cubic_data->max_cwnd = (cubic_data->max_cwnd * CUBIC_FC_FACTOR) >> CUBIC_SHIFT; if (IN_FASTRECOVERY(CCV(ccv, t_flags))) { /* * If inflight data is less than ssthresh, set cwnd * conservatively to avoid a burst of data, as suggested in * the NewReno RFC. Otherwise, use the CUBIC method. * * XXXLAS: Find a way to do this without needing curack */ if (SEQ_GT(ccv->curack + CCV(ccv, snd_ssthresh), CCV(ccv, snd_max))) CCV(ccv, snd_cwnd) = CCV(ccv, snd_max) - ccv->curack + CCV(ccv, t_maxseg); else /* Update cwnd based on beta and adjusted max_cwnd. */ CCV(ccv, snd_cwnd) = max(1, ((CUBIC_BETA * cubic_data->max_cwnd) >> CUBIC_SHIFT)); } cubic_data->t_last_cong = ticks; /* Calculate the average RTT between congestion epochs. */ if (cubic_data->epoch_ack_count > 0 && cubic_data->sum_rtt_ticks >= cubic_data->epoch_ack_count) { cubic_data->mean_rtt_ticks = (int)(cubic_data->sum_rtt_ticks / cubic_data->epoch_ack_count); } cubic_data->epoch_ack_count = 0; cubic_data->sum_rtt_ticks = 0; cubic_data->K = cubic_k(cubic_data->max_cwnd / CCV(ccv, t_maxseg)); } /* * Record the min RTT and sum samples for the epoch average RTT calculation. */ static void cubic_record_rtt(struct cc_var *ccv) { struct cubic *cubic_data; int t_srtt_ticks; /* Ignore srtt until a min number of samples have been taken. */ if (CCV(ccv, t_rttupdated) >= CUBIC_MIN_RTT_SAMPLES) { cubic_data = ccv->cc_data; t_srtt_ticks = CCV(ccv, t_srtt) / TCP_RTT_SCALE; /* * Record the current SRTT as our minrtt if it's the smallest * we've seen or minrtt is currently equal to its initialised * value. * * XXXLAS: Should there be some hysteresis for minrtt? */ if ((t_srtt_ticks < cubic_data->min_rtt_ticks || cubic_data->min_rtt_ticks == TCPTV_SRTTBASE)) { cubic_data->min_rtt_ticks = max(1, t_srtt_ticks); /* * If the connection is within its first congestion * epoch, ensure we prime mean_rtt_ticks with a * reasonable value until the epoch average RTT is * calculated in cubic_post_recovery(). */ if (cubic_data->min_rtt_ticks > cubic_data->mean_rtt_ticks) cubic_data->mean_rtt_ticks = cubic_data->min_rtt_ticks; } /* Sum samples for epoch average RTT calculation. */ cubic_data->sum_rtt_ticks += t_srtt_ticks; cubic_data->epoch_ack_count++; } } /* * Update the ssthresh in the event of congestion. */ static void cubic_ssthresh_update(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = ccv->cc_data; /* * On the first congestion event, set ssthresh to cwnd * 0.5, on * subsequent congestion events, set it to cwnd * beta. */ if (cubic_data->num_cong_events == 0) CCV(ccv, snd_ssthresh) = CCV(ccv, snd_cwnd) >> 1; else CCV(ccv, snd_ssthresh) = (CCV(ccv, snd_cwnd) * CUBIC_BETA) >> CUBIC_SHIFT; } DECLARE_CC_MODULE(cubic, &cubic_cc_algo); Index: head/sys/netinet/cc/cc_cubic.h =================================================================== --- head/sys/netinet/cc/cc_cubic.h (revision 220559) +++ head/sys/netinet/cc/cc_cubic.h (revision 220560) @@ -1,229 +1,229 @@ /*- * Copyright (c) 2008-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _NETINET_CC_CUBIC_H_ #define _NETINET_CC_CUBIC_H_ /* Number of bits of precision for fixed point math calcs. */ #define CUBIC_SHIFT 8 #define CUBIC_SHIFT_4 32 /* 0.5 << CUBIC_SHIFT. */ #define RENO_BETA 128 /* ~0.8 << CUBIC_SHIFT. */ #define CUBIC_BETA 204 /* ~0.2 << CUBIC_SHIFT. */ #define ONE_SUB_CUBIC_BETA 51 /* 3 * ONE_SUB_CUBIC_BETA. */ #define THREE_X_PT2 153 /* (2 << CUBIC_SHIFT) - ONE_SUB_CUBIC_BETA. */ #define TWO_SUB_PT2 461 /* ~0.4 << CUBIC_SHIFT. */ #define CUBIC_C_FACTOR 102 /* CUBIC fast convergence factor: ~0.9 << CUBIC_SHIFT. */ #define CUBIC_FC_FACTOR 230 /* Don't trust s_rtt until this many rtt samples have been taken. */ #define CUBIC_MIN_RTT_SAMPLES 8 /* Userland only bits. */ #ifndef _KERNEL extern int hz; /* * Implementation based on the formulae found in the CUBIC Internet Draft * "draft-rhee-tcpm-cubic-02". * * Note BETA used in cc_cubic is equal to (1-beta) in the I-D */ static __inline float theoretical_cubic_k(double wmax_pkts) { double C; C = 0.4; return (pow((wmax_pkts * 0.2) / C, (1.0 / 3.0)) * pow(2, CUBIC_SHIFT)); } static __inline unsigned long theoretical_cubic_cwnd(int ticks_since_cong, unsigned long wmax, uint32_t smss) { double C, wmax_pkts; C = 0.4; wmax_pkts = wmax / (double)smss; return (smss * (wmax_pkts + (C * pow(ticks_since_cong / (double)hz - theoretical_cubic_k(wmax_pkts) / pow(2, CUBIC_SHIFT), 3.0)))); } static __inline unsigned long theoretical_reno_cwnd(int ticks_since_cong, int rtt_ticks, unsigned long wmax, uint32_t smss) { return ((wmax * 0.5) + ((ticks_since_cong / (float)rtt_ticks) * smss)); } static __inline unsigned long theoretical_tf_cwnd(int ticks_since_cong, int rtt_ticks, unsigned long wmax, uint32_t smss) { return ((wmax * 0.8) + ((3 * 0.2) / (2 - 0.2) * (ticks_since_cong / (float)rtt_ticks) * smss)); } #endif /* !_KERNEL */ /* * Compute the CUBIC K value used in the cwnd calculation, using an * implementation of eqn 2 in the I-D. The method used * here is adapted from Apple Computer Technical Report #KT-32. */ static __inline int64_t cubic_k(unsigned long wmax_pkts) { int64_t s, K; uint16_t p; K = s = 0; p = 0; /* (wmax * beta)/C with CUBIC_SHIFT worth of precision. */ s = ((wmax_pkts * ONE_SUB_CUBIC_BETA) << CUBIC_SHIFT) / CUBIC_C_FACTOR; /* Rebase s to be between 1 and 1/8 with a shift of CUBIC_SHIFT. */ while (s >= 256) { s >>= 3; p++; } /* * Some magic constants taken from the Apple TR with appropriate * shifts: 275 == 1.072302 << CUBIC_SHIFT, 98 == 0.3812513 << * CUBIC_SHIFT, 120 == 0.46946116 << CUBIC_SHIFT. */ K = (((s * 275) >> CUBIC_SHIFT) + 98) - (((s * s * 120) >> CUBIC_SHIFT) >> CUBIC_SHIFT); /* Multiply by 2^p to undo the rebasing of s from above. */ return (K <<= p); } /* * Compute the new cwnd value using an implementation of eqn 1 from the I-D. * Thanks to Kip Macy for help debugging this function. * * XXXLAS: Characterise bounds for overflow. */ static __inline unsigned long cubic_cwnd(int ticks_since_cong, unsigned long wmax, uint32_t smss, int64_t K) { int64_t cwnd; /* K is in fixed point form with CUBIC_SHIFT worth of precision. */ /* t - K, with CUBIC_SHIFT worth of precision. */ cwnd = ((int64_t)(ticks_since_cong << CUBIC_SHIFT) - (K * hz)) / hz; /* (t - K)^3, with CUBIC_SHIFT^3 worth of precision. */ cwnd *= (cwnd * cwnd); /* * C(t - K)^3 + wmax * The down shift by CUBIC_SHIFT_4 is because cwnd has 4 lots of * CUBIC_SHIFT included in the value. 3 from the cubing of cwnd above, * and an extra from multiplying through by CUBIC_C_FACTOR. */ cwnd = ((cwnd * CUBIC_C_FACTOR * smss) >> CUBIC_SHIFT_4) + wmax; return ((unsigned long)cwnd); } /* * Compute an approximation of the NewReno cwnd some number of ticks after a * congestion event. RTT should be the average RTT estimate for the path * measured over the previous congestion epoch and wmax is the value of cwnd at * the last congestion event. The "TCP friendly" concept in the CUBIC I-D is * rather tricky to understand and it turns out this function is not required. * It is left here for reference. */ static __inline unsigned long reno_cwnd(int ticks_since_cong, int rtt_ticks, unsigned long wmax, uint32_t smss) { /* * For NewReno, beta = 0.5, therefore: W_tcp(t) = wmax*0.5 + t/RTT * W_tcp(t) deals with cwnd/wmax in pkts, so because our cwnd is in * bytes, we have to multiply by smss. */ return (((wmax * RENO_BETA) + (((ticks_since_cong * smss) << CUBIC_SHIFT) / rtt_ticks)) >> CUBIC_SHIFT); } /* * Compute an approximation of the "TCP friendly" cwnd some number of ticks * after a congestion event that is designed to yield the same average cwnd as * NewReno while using CUBIC's beta of 0.8. RTT should be the average RTT * estimate for the path measured over the previous congestion epoch and wmax is * the value of cwnd at the last congestion event. */ static __inline unsigned long tf_cwnd(int ticks_since_cong, int rtt_ticks, unsigned long wmax, uint32_t smss) { /* Equation 4 of I-D. */ return (((wmax * CUBIC_BETA) + (((THREE_X_PT2 * ticks_since_cong * smss) << CUBIC_SHIFT) / TWO_SUB_PT2 / rtt_ticks)) >> CUBIC_SHIFT); } #endif /* _NETINET_CC_CUBIC_H_ */ Index: head/sys/netinet/cc/cc_hd.c =================================================================== --- head/sys/netinet/cc/cc_hd.c (revision 220559) +++ head/sys/netinet/cc/cc_hd.c (revision 220560) @@ -1,254 +1,254 @@ /*- * Copyright (c) 2009-2010 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010-2011 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by David Hayes and Lawrence Stewart, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by David Hayes and + * Lawrence Stewart, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced Internet * Architectures, Swinburne University of Technology, Melbourne, Australia by * David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the Hamilton Institute's delay-based congestion control * algorithm for FreeBSD, based on "A strategy for fair coexistence of loss and * delay-based congestion control algorithms," by L. Budzisz, R. Stanojevic, R. * Shorten, and F. Baker, IEEE Commun. Lett., vol. 13, no. 7, pp. 555--557, Jul. * 2009. * * Originally released as part of the NewTCP research project at Swinburne - * University's Centre for Advanced Internet Architectures, Melbourne, - * Australia, which was made possible in part by a grant from the Cisco - * University Research Program Fund at Community Foundation Silicon Valley. More - * details are available at: + * University of Technology's Centre for Advanced Internet Architectures, + * Melbourne, Australia, which was made possible in part by a grant from the + * Cisco University Research Program Fund at Community Foundation Silicon + * Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define CAST_PTR_INT(X) (*((int*)(X))) /* Largest possible number returned by random(). */ #define RANDOM_MAX INT_MAX static void hd_ack_received(struct cc_var *ccv, uint16_t ack_type); static int hd_mod_init(void); static int ertt_id; static VNET_DEFINE(uint32_t, hd_qthresh) = 20; static VNET_DEFINE(uint32_t, hd_qmin) = 5; static VNET_DEFINE(uint32_t, hd_pmax) = 5; #define V_hd_qthresh VNET(hd_qthresh) #define V_hd_qmin VNET(hd_qmin) #define V_hd_pmax VNET(hd_pmax) struct cc_algo hd_cc_algo = { .name = "hd", .ack_received = hd_ack_received, .mod_init = hd_mod_init }; /* * Hamilton backoff function. Returns 1 if we should backoff or 0 otherwise. */ static __inline int should_backoff(int qdly, int maxqdly) { unsigned long p; if (qdly < V_hd_qthresh) { p = (((RANDOM_MAX / 100) * V_hd_pmax) / (V_hd_qthresh - V_hd_qmin)) * (qdly - V_hd_qmin); } else { if (qdly > V_hd_qthresh) p = (((RANDOM_MAX / 100) * V_hd_pmax) / (maxqdly - V_hd_qthresh)) * (maxqdly - qdly); else p = (RANDOM_MAX / 100) * V_hd_pmax; } return (random() < p); } /* * If the ack type is CC_ACK, and the inferred queueing delay is greater than * the Qmin threshold, cwnd is reduced probabilistically. When backing off due * to delay, HD behaves like NewReno when an ECN signal is received. HD behaves * as NewReno in all other circumstances. */ static void hd_ack_received(struct cc_var *ccv, uint16_t ack_type) { struct ertt *e_t; int qdly; if (ack_type == CC_ACK) { e_t = khelp_get_osd(CCV(ccv, osd), ertt_id); if (e_t->rtt && e_t->minrtt && V_hd_qthresh > 0) { qdly = e_t->rtt - e_t->minrtt; if (qdly > V_hd_qmin && !IN_RECOVERY(CCV(ccv, t_flags))) { /* Probabilistic backoff of cwnd. */ if (should_backoff(qdly, e_t->maxrtt - e_t->minrtt)) { /* * Update cwnd and ssthresh update to * half cwnd and behave like an ECN (ie * not a packet loss). */ newreno_cc_algo.cong_signal(ccv, CC_ECN); return; } } } } newreno_cc_algo.ack_received(ccv, ack_type); /* As for NewReno. */ } static int hd_mod_init(void) { ertt_id = khelp_get_id("ertt"); if (ertt_id <= 0) { printf("%s: h_ertt module not found\n", __func__); return (ENOENT); } hd_cc_algo.after_idle = newreno_cc_algo.after_idle; hd_cc_algo.cong_signal = newreno_cc_algo.cong_signal; hd_cc_algo.post_recovery = newreno_cc_algo.post_recovery; return (0); } static int hd_pmax_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_hd_pmax; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) == 0 || CAST_PTR_INT(req->newptr) > 100) error = EINVAL; else V_hd_pmax = new; } return (error); } static int hd_qmin_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_hd_qmin; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) > V_hd_qthresh) error = EINVAL; else V_hd_qmin = new; } return (error); } static int hd_qthresh_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_hd_qthresh; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) < 1 || CAST_PTR_INT(req->newptr) < V_hd_qmin) error = EINVAL; else V_hd_qthresh = new; } return (error); } SYSCTL_DECL(_net_inet_tcp_cc_hd); SYSCTL_NODE(_net_inet_tcp_cc, OID_AUTO, hd, CTLFLAG_RW, NULL, "Hamilton delay-based congestion control related settings"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_hd, OID_AUTO, queue_threshold, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(hd_qthresh), 20, &hd_qthresh_handler, "IU", "queueing congestion threshold (qth) in ticks"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_hd, OID_AUTO, pmax, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(hd_pmax), 5, &hd_pmax_handler, "IU", "per packet maximum backoff probability as a percentage"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_hd, OID_AUTO, queue_min, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(hd_qmin), 5, &hd_qmin_handler, "IU", "minimum queueing delay threshold (qmin) in ticks"); DECLARE_CC_MODULE(hd, &hd_cc_algo); MODULE_DEPEND(hd, ertt, 1, 1, 1); Index: head/sys/netinet/cc/cc_htcp.c =================================================================== --- head/sys/netinet/cc/cc_htcp.c (revision 220559) +++ head/sys/netinet/cc/cc_htcp.c (revision 220560) @@ -1,521 +1,521 @@ /*- * Copyright (c) 2007-2008 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by Lawrence Stewart and James Healy, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by Lawrence Stewart and + * James Healy, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the H-TCP congestion control algorithm for FreeBSD, * based on the Internet Draft "draft-leith-tcp-htcp-06.txt" by Leith and * Shorten. Originally released as part of the NewTCP research project at - * Swinburne University's Centre for Advanced Internet Architectures, Melbourne, - * Australia, which was made possible in part by a grant from the Cisco - * University Research Program Fund at Community Foundation Silicon Valley. More - * details are available at: + * Swinburne University of Technology's Centre for Advanced Internet + * Architectures, Melbourne, Australia, which was made possible in part by a + * grant from the Cisco University Research Program Fund at Community Foundation + * Silicon Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* Fixed point math shifts. */ #define HTCP_SHIFT 8 #define HTCP_ALPHA_INC_SHIFT 4 #define HTCP_INIT_ALPHA 1 #define HTCP_DELTA_L hz /* 1 sec in ticks. */ #define HTCP_MINBETA 128 /* 0.5 << HTCP_SHIFT. */ #define HTCP_MAXBETA 204 /* ~0.8 << HTCP_SHIFT. */ #define HTCP_MINROWE 26 /* ~0.1 << HTCP_SHIFT. */ #define HTCP_MAXROWE 512 /* 2 << HTCP_SHIFT. */ /* RTT_ref (ms) used in the calculation of alpha if RTT scaling is enabled. */ #define HTCP_RTT_REF 100 /* Don't trust SRTT until this many samples have been taken. */ #define HTCP_MIN_RTT_SAMPLES 8 /* * HTCP_CALC_ALPHA performs a fixed point math calculation to determine the * value of alpha, based on the function defined in the HTCP spec. * * i.e. 1 + 10(delta - delta_l) + ((delta - delta_l) / 2) ^ 2 * * "diff" is passed in to the macro as "delta - delta_l" and is expected to be * in units of ticks. * * The joyousnous of fixed point maths means our function implementation looks a * little funky... * * In order to maintain some precision in the calculations, a fixed point shift * HTCP_ALPHA_INC_SHIFT is used to ensure the integer divisions don't * truncate the results too badly. * * The "16" value is the "1" term in the alpha function shifted up by * HTCP_ALPHA_INC_SHIFT * * The "160" value is the "10" multiplier in the alpha function multiplied by * 2^HTCP_ALPHA_INC_SHIFT * * Specifying these as constants reduces the computations required. After * up-shifting all the terms in the function and performing the required * calculations, we down-shift the final result by HTCP_ALPHA_INC_SHIFT to * ensure it is back in the correct range. * * The "hz" terms are required as kernels can be configured to run with * different tick timers, which we have to adjust for in the alpha calculation * (which originally was defined in terms of seconds). * * We also have to be careful to constrain the value of diff such that it won't * overflow whilst performing the calculation. The middle term i.e. (160 * diff) * / hz is the limiting factor in the calculation. We must constrain diff to be * less than the max size of an int divided by the constant 160 figure * i.e. diff < INT_MAX / 160 * * NB: Changing HTCP_ALPHA_INC_SHIFT will require you to MANUALLY update the * constants used in this function! */ #define HTCP_CALC_ALPHA(diff) \ ((\ (16) + \ ((160 * (diff)) / hz) + \ (((diff) / hz) * (((diff) << HTCP_ALPHA_INC_SHIFT) / (4 * hz))) \ ) >> HTCP_ALPHA_INC_SHIFT) static void htcp_ack_received(struct cc_var *ccv, uint16_t type); static void htcp_cb_destroy(struct cc_var *ccv); static int htcp_cb_init(struct cc_var *ccv); static void htcp_cong_signal(struct cc_var *ccv, uint32_t type); static int htcp_mod_init(void); static void htcp_post_recovery(struct cc_var *ccv); static void htcp_recalc_alpha(struct cc_var *ccv); static void htcp_recalc_beta(struct cc_var *ccv); static void htcp_record_rtt(struct cc_var *ccv); static void htcp_ssthresh_update(struct cc_var *ccv); struct htcp { /* cwnd before entering cong recovery. */ unsigned long prev_cwnd; /* cwnd additive increase parameter. */ int alpha; /* cwnd multiplicative decrease parameter. */ int beta; /* Largest rtt seen for the flow. */ int maxrtt; /* Shortest rtt seen for the flow. */ int minrtt; /* Time of last congestion event in ticks. */ int t_last_cong; }; static int htcp_rtt_ref; /* * The maximum number of ticks the value of diff can reach in * htcp_recalc_alpha() before alpha will stop increasing due to overflow. * See comment above HTCP_CALC_ALPHA for more info. */ static int htcp_max_diff = INT_MAX / ((1 << HTCP_ALPHA_INC_SHIFT) * 10); /* Per-netstack vars. */ static VNET_DEFINE(u_int, htcp_adaptive_backoff) = 0; static VNET_DEFINE(u_int, htcp_rtt_scaling) = 0; #define V_htcp_adaptive_backoff VNET(htcp_adaptive_backoff) #define V_htcp_rtt_scaling VNET(htcp_rtt_scaling) MALLOC_DECLARE(M_HTCP); MALLOC_DEFINE(M_HTCP, "htcp data", "Per connection data required for the HTCP congestion control algorithm"); struct cc_algo htcp_cc_algo = { .name = "htcp", .ack_received = htcp_ack_received, .cb_destroy = htcp_cb_destroy, .cb_init = htcp_cb_init, .cong_signal = htcp_cong_signal, .mod_init = htcp_mod_init, .post_recovery = htcp_post_recovery, }; static void htcp_ack_received(struct cc_var *ccv, uint16_t type) { struct htcp *htcp_data; htcp_data = ccv->cc_data; htcp_record_rtt(ccv); /* * Regular ACK and we're not in cong/fast recovery and we're cwnd * limited and we're either not doing ABC or are slow starting or are * doing ABC and we've sent a cwnd's worth of bytes. */ if (type == CC_ACK && !IN_RECOVERY(CCV(ccv, t_flags)) && (ccv->flags & CCF_CWND_LIMITED) && (!V_tcp_do_rfc3465 || CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh) || (V_tcp_do_rfc3465 && ccv->flags & CCF_ABC_SENTAWND))) { htcp_recalc_beta(ccv); htcp_recalc_alpha(ccv); /* * Use the logic in NewReno ack_received() for slow start and * for the first HTCP_DELTA_L ticks after either the flow starts * or a congestion event (when alpha equals 1). */ if (htcp_data->alpha == 1 || CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh)) newreno_cc_algo.ack_received(ccv, type); else { if (V_tcp_do_rfc3465) { /* Increment cwnd by alpha segments. */ CCV(ccv, snd_cwnd) += htcp_data->alpha * CCV(ccv, t_maxseg); ccv->flags &= ~CCF_ABC_SENTAWND; } else /* * Increment cwnd by alpha/cwnd segments to * approximate an increase of alpha segments * per RTT. */ CCV(ccv, snd_cwnd) += (((htcp_data->alpha << HTCP_SHIFT) / (CCV(ccv, snd_cwnd) / CCV(ccv, t_maxseg))) * CCV(ccv, t_maxseg)) >> HTCP_SHIFT; } } } static void htcp_cb_destroy(struct cc_var *ccv) { if (ccv->cc_data != NULL) free(ccv->cc_data, M_HTCP); } static int htcp_cb_init(struct cc_var *ccv) { struct htcp *htcp_data; htcp_data = malloc(sizeof(struct htcp), M_HTCP, M_NOWAIT); if (htcp_data == NULL) return (ENOMEM); /* Init some key variables with sensible defaults. */ htcp_data->alpha = HTCP_INIT_ALPHA; htcp_data->beta = HTCP_MINBETA; htcp_data->maxrtt = TCPTV_SRTTBASE; htcp_data->minrtt = TCPTV_SRTTBASE; htcp_data->prev_cwnd = 0; htcp_data->t_last_cong = ticks; ccv->cc_data = htcp_data; return (0); } /* * Perform any necessary tasks before we enter congestion recovery. */ static void htcp_cong_signal(struct cc_var *ccv, uint32_t type) { struct htcp *htcp_data; htcp_data = ccv->cc_data; switch (type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(CCV(ccv, t_flags))) { if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { /* * Apply hysteresis to maxrtt to ensure * reductions in the RTT are reflected in our * measurements. */ htcp_data->maxrtt = (htcp_data->minrtt + (htcp_data->maxrtt - htcp_data->minrtt) * 95) / 100; htcp_ssthresh_update(ccv); htcp_data->t_last_cong = ticks; htcp_data->prev_cwnd = CCV(ccv, snd_cwnd); } ENTER_RECOVERY(CCV(ccv, t_flags)); } break; case CC_ECN: if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { /* * Apply hysteresis to maxrtt to ensure reductions in * the RTT are reflected in our measurements. */ htcp_data->maxrtt = (htcp_data->minrtt + (htcp_data->maxrtt - htcp_data->minrtt) * 95) / 100; htcp_ssthresh_update(ccv); CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); htcp_data->t_last_cong = ticks; htcp_data->prev_cwnd = CCV(ccv, snd_cwnd); ENTER_CONGRECOVERY(CCV(ccv, t_flags)); } break; case CC_RTO: /* * Grab the current time and record it so we know when the * most recent congestion event was. Only record it when the * timeout has fired more than once, as there is a reasonable * chance the first one is a false alarm and may not indicate * congestion. */ if (CCV(ccv, t_rxtshift) >= 2) htcp_data->t_last_cong = ticks; break; } } static int htcp_mod_init(void) { htcp_cc_algo.after_idle = newreno_cc_algo.after_idle; /* * HTCP_RTT_REF is defined in ms, and t_srtt in the tcpcb is stored in * units of TCP_RTT_SCALE*hz. Scale HTCP_RTT_REF to be in the same units * as t_srtt. */ htcp_rtt_ref = (HTCP_RTT_REF * TCP_RTT_SCALE * hz) / 1000; return (0); } /* * Perform any necessary tasks before we exit congestion recovery. */ static void htcp_post_recovery(struct cc_var *ccv) { struct htcp *htcp_data; htcp_data = ccv->cc_data; if (IN_FASTRECOVERY(CCV(ccv, t_flags))) { /* * If inflight data is less than ssthresh, set cwnd * conservatively to avoid a burst of data, as suggested in the * NewReno RFC. Otherwise, use the HTCP method. * * XXXLAS: Find a way to do this without needing curack */ if (SEQ_GT(ccv->curack + CCV(ccv, snd_ssthresh), CCV(ccv, snd_max))) CCV(ccv, snd_cwnd) = CCV(ccv, snd_max) - ccv->curack + CCV(ccv, t_maxseg); else CCV(ccv, snd_cwnd) = max(1, ((htcp_data->beta * htcp_data->prev_cwnd / CCV(ccv, t_maxseg)) >> HTCP_SHIFT)) * CCV(ccv, t_maxseg); } } static void htcp_recalc_alpha(struct cc_var *ccv) { struct htcp *htcp_data; int alpha, diff, now; htcp_data = ccv->cc_data; now = ticks; /* * If ticks has wrapped around (will happen approximately once every 49 * days on a machine with the default kern.hz=1000) and a flow straddles * the wrap point, our alpha calcs will be completely wrong. We cut our * losses and restart alpha from scratch by setting t_last_cong = now - * HTCP_DELTA_L. * * This does not deflate our cwnd at all. It simply slows the rate cwnd * is growing by until alpha regains the value it held prior to taking * this drastic measure. */ if (now < htcp_data->t_last_cong) htcp_data->t_last_cong = now - HTCP_DELTA_L; diff = now - htcp_data->t_last_cong - HTCP_DELTA_L; /* Cap alpha if the value of diff would overflow HTCP_CALC_ALPHA(). */ if (diff < htcp_max_diff) { /* * If it has been more than HTCP_DELTA_L ticks since congestion, * increase alpha according to the function defined in the spec. */ if (diff > 0) { alpha = HTCP_CALC_ALPHA(diff); /* * Adaptive backoff fairness adjustment: * 2 * (1 - beta) * alpha_raw */ if (V_htcp_adaptive_backoff) alpha = max(1, (2 * ((1 << HTCP_SHIFT) - htcp_data->beta) * alpha) >> HTCP_SHIFT); /* * RTT scaling: (RTT / RTT_ref) * alpha * alpha will be the raw value from HTCP_CALC_ALPHA() if * adaptive backoff is off, or the adjusted value if * adaptive backoff is on. */ if (V_htcp_rtt_scaling) alpha = max(1, (min(max(HTCP_MINROWE, (CCV(ccv, t_srtt) << HTCP_SHIFT) / htcp_rtt_ref), HTCP_MAXROWE) * alpha) >> HTCP_SHIFT); } else alpha = 1; htcp_data->alpha = alpha; } } static void htcp_recalc_beta(struct cc_var *ccv) { struct htcp *htcp_data; htcp_data = ccv->cc_data; /* * TCPTV_SRTTBASE is the initialised value of each connection's SRTT, so * we only calc beta if the connection's SRTT has been changed from its * inital value. beta is bounded to ensure it is always between * HTCP_MINBETA and HTCP_MAXBETA. */ if (V_htcp_adaptive_backoff && htcp_data->minrtt != TCPTV_SRTTBASE && htcp_data->maxrtt != TCPTV_SRTTBASE) htcp_data->beta = min(max(HTCP_MINBETA, (htcp_data->minrtt << HTCP_SHIFT) / htcp_data->maxrtt), HTCP_MAXBETA); else htcp_data->beta = HTCP_MINBETA; } /* * Record the minimum and maximum RTT seen for the connection. These are used in * the calculation of beta if adaptive backoff is enabled. */ static void htcp_record_rtt(struct cc_var *ccv) { struct htcp *htcp_data; htcp_data = ccv->cc_data; /* XXXLAS: Should there be some hysteresis for minrtt? */ /* * Record the current SRTT as our minrtt if it's the smallest we've seen * or minrtt is currently equal to its initialised value. Ignore SRTT * until a min number of samples have been taken. */ if ((CCV(ccv, t_srtt) < htcp_data->minrtt || htcp_data->minrtt == TCPTV_SRTTBASE) && (CCV(ccv, t_rttupdated) >= HTCP_MIN_RTT_SAMPLES)) htcp_data->minrtt = CCV(ccv, t_srtt); /* * Record the current SRTT as our maxrtt if it's the largest we've * seen. Ignore SRTT until a min number of samples have been taken. */ if (CCV(ccv, t_srtt) > htcp_data->maxrtt && CCV(ccv, t_rttupdated) >= HTCP_MIN_RTT_SAMPLES) htcp_data->maxrtt = CCV(ccv, t_srtt); } /* * Update the ssthresh in the event of congestion. */ static void htcp_ssthresh_update(struct cc_var *ccv) { struct htcp *htcp_data; htcp_data = ccv->cc_data; /* * On the first congestion event, set ssthresh to cwnd * 0.5, on * subsequent congestion events, set it to cwnd * beta. */ if (CCV(ccv, snd_ssthresh) == TCP_MAXWIN << TCP_MAX_WINSHIFT) CCV(ccv, snd_ssthresh) = (CCV(ccv, snd_cwnd) * HTCP_MINBETA) >> HTCP_SHIFT; else { htcp_recalc_beta(ccv); CCV(ccv, snd_ssthresh) = (CCV(ccv, snd_cwnd) * htcp_data->beta) >> HTCP_SHIFT; } } SYSCTL_DECL(_net_inet_tcp_cc_htcp); SYSCTL_NODE(_net_inet_tcp_cc, OID_AUTO, htcp, CTLFLAG_RW, NULL, "H-TCP related settings"); SYSCTL_VNET_UINT(_net_inet_tcp_cc_htcp, OID_AUTO, adaptive_backoff, CTLFLAG_RW, &VNET_NAME(htcp_adaptive_backoff), 0, "enable H-TCP adaptive backoff"); SYSCTL_VNET_UINT(_net_inet_tcp_cc_htcp, OID_AUTO, rtt_scaling, CTLFLAG_RW, &VNET_NAME(htcp_rtt_scaling), 0, "enable H-TCP RTT scaling"); DECLARE_CC_MODULE(htcp, &htcp_cc_algo); Index: head/sys/netinet/cc/cc_module.h =================================================================== --- head/sys/netinet/cc/cc_module.h (revision 220559) +++ head/sys/netinet/cc/cc_module.h (revision 220560) @@ -1,70 +1,70 @@ /*- * Copyright (c) 2009-2010 Lawrence Stewart * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * This software was first released in 2009 by Lawrence Stewart as part of the - * NewTCP research project at Swinburne University's Centre for Advanced - * Internet Architectures, Melbourne, Australia, which was made possible in part - * by a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. More details are available at: + * NewTCP research project at Swinburne University of Technology's Centre for + * Advanced Internet Architectures, Melbourne, Australia, which was made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #ifndef _NETINET_CC_MODULE_H_ #define _NETINET_CC_MODULE_H_ /* * Allows a CC algorithm to manipulate a commonly named CC variable regardless * of the transport protocol and associated C struct. * XXXLAS: Out of action until the work to support SCTP is done. * #define CCV(ccv, what) \ (*( \ (ccv)->type == IPPROTO_TCP ? &(ccv)->ccvc.tcp->what : \ &(ccv)->ccvc.sctp->what \ )) */ #define CCV(ccv, what) (ccv)->ccvc.tcp->what #define DECLARE_CC_MODULE(ccname, ccalgo) \ static moduledata_t cc_##ccname = { \ .name = #ccname, \ .evhand = cc_modevent, \ .priv = ccalgo \ }; \ DECLARE_MODULE(ccname, cc_##ccname, \ SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_ANY) int cc_modevent(module_t mod, int type, void *data); #endif /* _NETINET_CC_MODULE_H_ */ Index: head/sys/netinet/cc/cc_newreno.c =================================================================== --- head/sys/netinet/cc/cc_newreno.c (revision 220559) +++ head/sys/netinet/cc/cc_newreno.c (revision 220560) @@ -1,236 +1,237 @@ /*- * Copyright (c) 1982, 1986, 1988, 1990, 1993, 1994, 1995 * The Regents of the University of California. * Copyright (c) 2007-2008,2010 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by Lawrence Stewart, James Healy and - * David Hayes, made possible in part by a grant from the Cisco University - * Research Program Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by Lawrence Stewart, James + * Healy and David Hayes, made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * This software was first released in 2007 by James Healy and Lawrence Stewart - * whilst working on the NewTCP research project at Swinburne University's - * Centre for Advanced Internet Architectures, Melbourne, Australia, which was - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. More details are available at: + * whilst working on the NewTCP research project at Swinburne University of + * Technology's Centre for Advanced Internet Architectures, Melbourne, + * Australia, which was made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. + * More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include static void newreno_ack_received(struct cc_var *ccv, uint16_t type); static void newreno_after_idle(struct cc_var *ccv); static void newreno_cong_signal(struct cc_var *ccv, uint32_t type); static void newreno_post_recovery(struct cc_var *ccv); struct cc_algo newreno_cc_algo = { .name = "newreno", .ack_received = newreno_ack_received, .after_idle = newreno_after_idle, .cong_signal = newreno_cong_signal, .post_recovery = newreno_post_recovery, }; static void newreno_ack_received(struct cc_var *ccv, uint16_t type) { if (type == CC_ACK && !IN_RECOVERY(CCV(ccv, t_flags)) && (ccv->flags & CCF_CWND_LIMITED)) { u_int cw = CCV(ccv, snd_cwnd); u_int incr = CCV(ccv, t_maxseg); /* * Regular in-order ACK, open the congestion window. * Method depends on which congestion control state we're * in (slow start or cong avoid) and if ABC (RFC 3465) is * enabled. * * slow start: cwnd <= ssthresh * cong avoid: cwnd > ssthresh * * slow start and ABC (RFC 3465): * Grow cwnd exponentially by the amount of data * ACKed capping the max increment per ACK to * (abc_l_var * maxseg) bytes. * * slow start without ABC (RFC 5681): * Grow cwnd exponentially by maxseg per ACK. * * cong avoid and ABC (RFC 3465): * Grow cwnd linearly by maxseg per RTT for each * cwnd worth of ACKed data. * * cong avoid without ABC (RFC 5681): * Grow cwnd linearly by approximately maxseg per RTT using * maxseg^2 / cwnd per ACK as the increment. * If cwnd > maxseg^2, fix the cwnd increment at 1 byte to * avoid capping cwnd. */ if (cw > CCV(ccv, snd_ssthresh)) { if (V_tcp_do_rfc3465) { if (ccv->flags & CCF_ABC_SENTAWND) ccv->flags &= ~CCF_ABC_SENTAWND; else incr = 0; } else incr = max((incr * incr / cw), 1); } else if (V_tcp_do_rfc3465) { /* * In slow-start with ABC enabled and no RTO in sight? * (Must not use abc_l_var > 1 if slow starting after * an RTO. On RTO, snd_nxt = snd_una, so the * snd_nxt == snd_max check is sufficient to * handle this). * * XXXLAS: Find a way to signal SS after RTO that * doesn't rely on tcpcb vars. */ if (CCV(ccv, snd_nxt) == CCV(ccv, snd_max)) incr = min(ccv->bytes_this_ack, V_tcp_abc_l_var * CCV(ccv, t_maxseg)); else incr = min(ccv->bytes_this_ack, CCV(ccv, t_maxseg)); } /* ABC is on by default, so incr equals 0 frequently. */ if (incr > 0) CCV(ccv, snd_cwnd) = min(cw + incr, TCP_MAXWIN << CCV(ccv, snd_scale)); } } static void newreno_after_idle(struct cc_var *ccv) { int rw; /* * If we've been idle for more than one retransmit timeout the old * congestion window is no longer current and we have to reduce it to * the restart window before we can transmit again. * * The restart window is the initial window or the last CWND, whichever * is smaller. * * This is done to prevent us from flooding the path with a full CWND at * wirespeed, overloading router and switch buffers along the way. * * See RFC5681 Section 4.1. "Restarting Idle Connections". */ if (V_tcp_do_rfc3390) rw = min(4 * CCV(ccv, t_maxseg), max(2 * CCV(ccv, t_maxseg), 4380)); else rw = CCV(ccv, t_maxseg) * 2; CCV(ccv, snd_cwnd) = min(rw, CCV(ccv, snd_cwnd)); } /* * Perform any necessary tasks before we enter congestion recovery. */ static void newreno_cong_signal(struct cc_var *ccv, uint32_t type) { u_int win; /* Catch algos which mistakenly leak private signal types. */ KASSERT((type & CC_SIGPRIVMASK) == 0, ("%s: congestion signal type 0x%08x is private\n", __func__, type)); win = max(CCV(ccv, snd_cwnd) / 2 / CCV(ccv, t_maxseg), 2) * CCV(ccv, t_maxseg); switch (type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(CCV(ccv, t_flags))) { if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) CCV(ccv, snd_ssthresh) = win; ENTER_RECOVERY(CCV(ccv, t_flags)); } break; case CC_ECN: if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { CCV(ccv, snd_ssthresh) = win; CCV(ccv, snd_cwnd) = win; ENTER_CONGRECOVERY(CCV(ccv, t_flags)); } break; } } /* * Perform any necessary tasks before we exit congestion recovery. */ static void newreno_post_recovery(struct cc_var *ccv) { if (IN_FASTRECOVERY(CCV(ccv, t_flags))) { /* * Fast recovery will conclude after returning from this * function. Window inflation should have left us with * approximately snd_ssthresh outstanding data. But in case we * would be inclined to send a burst, better to do it via the * slow start mechanism. * * XXXLAS: Find a way to do this without needing curack */ if (SEQ_GT(ccv->curack + CCV(ccv, snd_ssthresh), CCV(ccv, snd_max))) CCV(ccv, snd_cwnd) = CCV(ccv, snd_max) - ccv->curack + CCV(ccv, t_maxseg); else CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); } } DECLARE_CC_MODULE(newreno, &newreno_cc_algo); Index: head/sys/netinet/cc/cc_vegas.c =================================================================== --- head/sys/netinet/cc/cc_vegas.c (revision 220559) +++ head/sys/netinet/cc/cc_vegas.c (revision 220560) @@ -1,308 +1,308 @@ /*- * Copyright (c) 2009-2010 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010-2011 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by David Hayes and Lawrence Stewart, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by David Hayes and + * Lawrence Stewart, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced Internet * Architectures, Swinburne University of Technology, Melbourne, Australia by * David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the Vegas congestion control algorithm for FreeBSD, * based on L. S. Brakmo and L. L. Peterson, "TCP Vegas: end to end congestion * avoidance on a global internet", IEEE J. Sel. Areas Commun., vol. 13, no. 8, * pp. 1465-1480, Oct. 1995. The original Vegas duplicate ack policy has not * been implemented, since clock ticks are not as coarse as they were (i.e. * 500ms) when Vegas was designed. Also, packets are timed once per RTT as in * the original paper. * * Originally released as part of the NewTCP research project at Swinburne - * University's Centre for Advanced Internet Architectures, Melbourne, - * Australia, which was made possible in part by a grant from the Cisco - * University Research Program Fund at Community Foundation Silicon Valley. More - * details are available at: + * University of Technology's Centre for Advanced Internet Architectures, + * Melbourne, Australia, which was made possible in part by a grant from the + * Cisco University Research Program Fund at Community Foundation Silicon + * Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define CAST_PTR_INT(X) (*((int*)(X))) /* * Private signal type for rate based congestion signal. * See for appropriate bit-range to use for private signals. */ #define CC_VEGAS_RATE 0x01000000 static void vegas_ack_received(struct cc_var *ccv, uint16_t ack_type); static void vegas_cb_destroy(struct cc_var *ccv); static int vegas_cb_init(struct cc_var *ccv); static void vegas_cong_signal(struct cc_var *ccv, uint32_t signal_type); static void vegas_conn_init(struct cc_var *ccv); static int vegas_mod_init(void); struct vegas { int slow_start_toggle; }; static int32_t ertt_id; static VNET_DEFINE(uint32_t, vegas_alpha) = 1; static VNET_DEFINE(uint32_t, vegas_beta) = 3; #define V_vegas_alpha VNET(vegas_alpha) #define V_vegas_beta VNET(vegas_beta) MALLOC_DECLARE(M_VEGAS); MALLOC_DEFINE(M_VEGAS, "vegas data", "Per connection data required for the Vegas congestion control algorithm"); struct cc_algo vegas_cc_algo = { .name = "vegas", .ack_received = vegas_ack_received, .cb_destroy = vegas_cb_destroy, .cb_init = vegas_cb_init, .cong_signal = vegas_cong_signal, .conn_init = vegas_conn_init, .mod_init = vegas_mod_init }; /* * The vegas window adjustment is done once every RTT, as indicated by the * ERTT_NEW_MEASUREMENT flag. This flag is reset once the new measurment data * has been used. */ static void vegas_ack_received(struct cc_var *ccv, uint16_t ack_type) { struct ertt *e_t; struct vegas *vegas_data; long actual_tx_rate, expected_tx_rate, ndiff; e_t = khelp_get_osd(CCV(ccv, osd), ertt_id); vegas_data = ccv->cc_data; if (e_t->flags & ERTT_NEW_MEASUREMENT) { /* Once per RTT. */ if (e_t->minrtt && e_t->markedpkt_rtt) { expected_tx_rate = e_t->marked_snd_cwnd / e_t->minrtt; actual_tx_rate = e_t->bytes_tx_in_marked_rtt / e_t->markedpkt_rtt; ndiff = (expected_tx_rate - actual_tx_rate) * e_t->minrtt / CCV(ccv, t_maxseg); if (ndiff < V_vegas_alpha) { if (CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh)) { vegas_data->slow_start_toggle = vegas_data->slow_start_toggle ? 0 : 1; } else { vegas_data->slow_start_toggle = 0; CCV(ccv, snd_cwnd) = min(CCV(ccv, snd_cwnd) + CCV(ccv, t_maxseg), TCP_MAXWIN << CCV(ccv, snd_scale)); } } else if (ndiff > V_vegas_beta) { /* Rate-based congestion. */ vegas_cong_signal(ccv, CC_VEGAS_RATE); vegas_data->slow_start_toggle = 0; } } e_t->flags &= ~ERTT_NEW_MEASUREMENT; } if (vegas_data->slow_start_toggle) newreno_cc_algo.ack_received(ccv, ack_type); } static void vegas_cb_destroy(struct cc_var *ccv) { if (ccv->cc_data != NULL) free(ccv->cc_data, M_VEGAS); } static int vegas_cb_init(struct cc_var *ccv) { struct vegas *vegas_data; vegas_data = malloc(sizeof(struct vegas), M_VEGAS, M_NOWAIT); if (vegas_data == NULL) return (ENOMEM); vegas_data->slow_start_toggle = 1; ccv->cc_data = vegas_data; return (0); } /* * If congestion has been triggered triggered by the Vegas measured rates, it is * handled here, otherwise it falls back to newreno's congestion handling. */ static void vegas_cong_signal(struct cc_var *ccv, uint32_t signal_type) { struct vegas *vegas_data; int presignalrecov; vegas_data = ccv->cc_data; if (IN_RECOVERY(CCV(ccv, t_flags))) presignalrecov = 1; else presignalrecov = 0; switch(signal_type) { case CC_VEGAS_RATE: if (!IN_RECOVERY(CCV(ccv, t_flags))) { CCV(ccv, snd_cwnd) = max(2 * CCV(ccv, t_maxseg), CCV(ccv, snd_cwnd) - CCV(ccv, t_maxseg)); if (CCV(ccv, snd_cwnd) < CCV(ccv, snd_ssthresh)) /* Exit slow start. */ CCV(ccv, snd_ssthresh) = CCV(ccv, snd_cwnd); } break; default: newreno_cc_algo.cong_signal(ccv, signal_type); } if (IN_RECOVERY(CCV(ccv, t_flags)) && !presignalrecov) vegas_data->slow_start_toggle = (CCV(ccv, snd_cwnd) < CCV(ccv, snd_ssthresh)) ? 1 : 0; } static void vegas_conn_init(struct cc_var *ccv) { struct vegas *vegas_data; vegas_data = ccv->cc_data; vegas_data->slow_start_toggle = 1; } static int vegas_mod_init(void) { ertt_id = khelp_get_id("ertt"); if (ertt_id <= 0) { printf("%s: h_ertt module not found\n", __func__); return (ENOENT); } vegas_cc_algo.after_idle = newreno_cc_algo.after_idle; vegas_cc_algo.post_recovery = newreno_cc_algo.post_recovery; return (0); } static int vegas_alpha_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_vegas_alpha; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) < 1 || CAST_PTR_INT(req->newptr) > V_vegas_beta) error = EINVAL; else V_vegas_alpha = new; } return (error); } static int vegas_beta_handler(SYSCTL_HANDLER_ARGS) { int error; uint32_t new; new = V_vegas_beta; error = sysctl_handle_int(oidp, &new, 0, req); if (error == 0 && req->newptr != NULL) { if (CAST_PTR_INT(req->newptr) < 1 || CAST_PTR_INT(req->newptr) < V_vegas_alpha) error = EINVAL; else V_vegas_beta = new; } return (error); } SYSCTL_DECL(_net_inet_tcp_cc_vegas); SYSCTL_NODE(_net_inet_tcp_cc, OID_AUTO, vegas, CTLFLAG_RW, NULL, "Vegas related settings"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_vegas, OID_AUTO, alpha, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(vegas_alpha), 1, &vegas_alpha_handler, "IU", "vegas alpha, specified as number of \"buffers\" (0 < alpha < beta)"); SYSCTL_VNET_PROC(_net_inet_tcp_cc_vegas, OID_AUTO, beta, CTLTYPE_UINT|CTLFLAG_RW, &VNET_NAME(vegas_beta), 3, &vegas_beta_handler, "IU", "vegas beta, specified as number of \"buffers\" (0 < alpha < beta)"); DECLARE_CC_MODULE(vegas, &vegas_cc_algo); MODULE_DEPEND(vegas, ertt, 1, 1, 1); Index: head/sys/netinet/cc.h =================================================================== --- head/sys/netinet/cc.h (revision 220559) +++ head/sys/netinet/cc.h (revision 220560) @@ -1,166 +1,167 @@ /*- * Copyright (c) 2007-2008 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by Lawrence Stewart and James Healy, - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by Lawrence Stewart and + * James Healy, made possible in part by a grant from the Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * This software was first released in 2007 by James Healy and Lawrence Stewart - * whilst working on the NewTCP research project at Swinburne University's - * Centre for Advanced Internet Architectures, Melbourne, Australia, which was - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. More details are available at: + * whilst working on the NewTCP research project at Swinburne University of + * Technology's Centre for Advanced Internet Architectures, Melbourne, + * Australia, which was made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. + * More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #ifndef _NETINET_CC_H_ #define _NETINET_CC_H_ /* XXX: TCP_CA_NAME_MAX define lives in tcp.h for compat reasons. */ #include /* Global CC vars. */ extern STAILQ_HEAD(cc_head, cc_algo) cc_list; extern const int tcprexmtthresh; extern struct cc_algo newreno_cc_algo; /* Per-netstack bits. */ VNET_DECLARE(struct cc_algo *, default_cc_ptr); #define V_default_cc_ptr VNET(default_cc_ptr) /* Define the new net.inet.tcp.cc sysctl tree. */ SYSCTL_DECL(_net_inet_tcp_cc); /* CC housekeeping functions. */ int cc_register_algo(struct cc_algo *add_cc); int cc_deregister_algo(struct cc_algo *remove_cc); /* * Wrapper around transport structs that contain same-named congestion * control variables. Allows algos to be shared amongst multiple CC aware * transprots. */ struct cc_var { void *cc_data; /* Per-connection private CC algorithm data. */ int bytes_this_ack; /* # bytes acked by the current ACK. */ tcp_seq curack; /* Most recent ACK. */ uint32_t flags; /* Flags for cc_var (see below) */ int type; /* Indicates which ptr is valid in ccvc. */ union ccv_container { struct tcpcb *tcp; struct sctp_nets *sctp; } ccvc; }; /* cc_var flags. */ #define CCF_ABC_SENTAWND 0x0001 /* ABC counted cwnd worth of bytes? */ #define CCF_CWND_LIMITED 0x0002 /* Are we currently cwnd limited? */ /* ACK types passed to the ack_received() hook. */ #define CC_ACK 0x0001 /* Regular in sequence ACK. */ #define CC_DUPACK 0x0002 /* Duplicate ACK. */ #define CC_PARTIALACK 0x0004 /* Not yet. */ #define CC_SACK 0x0008 /* Not yet. */ /* * Congestion signal types passed to the cong_signal() hook. The highest order 8 * bits (0x01000000 - 0x80000000) are reserved for CC algos to declare their own * congestion signal types. */ #define CC_ECN 0x00000001 /* ECN marked packet received. */ #define CC_RTO 0x00000002 /* RTO fired. */ #define CC_RTO_ERR 0x00000004 /* RTO fired in error. */ #define CC_NDUPACK 0x00000008 /* Threshold of dupack's reached. */ #define CC_SIGPRIVMASK 0xFF000000 /* Mask to check if sig is private. */ /* * Structure to hold data and function pointers that together represent a * congestion control algorithm. */ struct cc_algo { char name[TCP_CA_NAME_MAX]; /* Init global module state on kldload. */ int (*mod_init)(void); /* Cleanup global module state on kldunload. */ int (*mod_destroy)(void); /* Init CC state for a new control block. */ int (*cb_init)(struct cc_var *ccv); /* Cleanup CC state for a terminating control block. */ void (*cb_destroy)(struct cc_var *ccv); /* Init variables for a newly established connection. */ void (*conn_init)(struct cc_var *ccv); /* Called on receipt of an ack. */ void (*ack_received)(struct cc_var *ccv, uint16_t type); /* Called on detection of a congestion signal. */ void (*cong_signal)(struct cc_var *ccv, uint32_t type); /* Called after exiting congestion recovery. */ void (*post_recovery)(struct cc_var *ccv); /* Called when data transfer resumes after an idle period. */ void (*after_idle)(struct cc_var *ccv); STAILQ_ENTRY (cc_algo) entries; }; /* Macro to obtain the CC algo's struct ptr. */ #define CC_ALGO(tp) ((tp)->cc_algo) /* Macro to obtain the CC algo's data ptr. */ #define CC_DATA(tp) ((tp)->ccv->cc_data) /* Macro to obtain the system default CC algo's struct ptr. */ #define CC_DEFAULT() V_default_cc_ptr extern struct rwlock cc_list_lock; #define CC_LIST_LOCK_INIT() rw_init(&cc_list_lock, "cc_list") #define CC_LIST_LOCK_DESTROY() rw_destroy(&cc_list_lock) #define CC_LIST_RLOCK() rw_rlock(&cc_list_lock) #define CC_LIST_RUNLOCK() rw_runlock(&cc_list_lock) #define CC_LIST_WLOCK() rw_wlock(&cc_list_lock) #define CC_LIST_WUNLOCK() rw_wunlock(&cc_list_lock) #define CC_LIST_LOCK_ASSERT() rw_assert(&cc_list_lock, RA_LOCKED) #endif /* _NETINET_CC_H_ */ Index: head/sys/netinet/khelp/h_ertt.c =================================================================== --- head/sys/netinet/khelp/h_ertt.c (revision 220559) +++ head/sys/netinet/khelp/h_ertt.c (revision 220560) @@ -1,545 +1,545 @@ /*- * Copyright (c) 2009-2010 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010-2011 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by David Hayes, made possible in part by - * a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by David Hayes, made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include uma_zone_t txseginfo_zone; /* Smoothing factor for delayed ack guess. */ #define DLYACK_SMOOTH 5 /* Max number of time stamp errors allowed in a session. */ #define MAX_TS_ERR 10 static int ertt_packet_measurement_hook(int hhook_type, int hhook_id, void *udata, void *ctx_data, void *hdata, struct osd *hosd); static int ertt_add_tx_segment_info_hook(int hhook_type, int hhook_id, void *udata, void *ctx_data, void *hdata, struct osd *hosd); static int ertt_mod_init(void); static int ertt_mod_destroy(void); static int ertt_uma_ctor(void *mem, int size, void *arg, int flags); static void ertt_uma_dtor(void *mem, int size, void *arg); /* * Contains information about the sent segment for comparison with the * corresponding ack. */ struct txseginfo { /* Segment length. */ long len; /* Segment sequence number. */ tcp_seq seq; /* Time stamp indicating when the packet was sent. */ uint32_t tx_ts; /* Last received receiver ts (if the TCP option is used). */ uint32_t rx_ts; uint32_t flags; TAILQ_ENTRY (txseginfo) txsegi_lnk; }; /* Flags for struct txseginfo. */ #define TXSI_TSO 0x01 /* TSO was used for this entry. */ #define TXSI_RTT_MEASURE_START 0x02 /* Start a per RTT measurement. */ #define TXSI_RX_MEASURE_END 0x04 /* Measure the rx rate until this txsi. */ struct helper ertt_helper = { .mod_init = ertt_mod_init, .mod_destroy = ertt_mod_destroy, .h_flags = HELPER_NEEDS_OSD, .h_classes = HELPER_CLASS_TCP }; /* Define the helper hook info required by ERTT. */ struct hookinfo ertt_hooks[] = { { .hook_type = HHOOK_TYPE_TCP, .hook_id = HHOOK_TCP_EST_IN, .hook_udata = NULL, .hook_func = &ertt_packet_measurement_hook }, { .hook_type = HHOOK_TYPE_TCP, .hook_id = HHOOK_TCP_EST_OUT, .hook_udata = NULL, .hook_func = &ertt_add_tx_segment_info_hook } }; /* Flags to indicate how marked_packet_rtt should handle this txsi. */ #define MULTI_ACK 0x01 /* More than this txsi is acked. */ #define OLD_TXSI 0x02 /* TXSI is old according to timestamps. */ #define CORRECT_ACK 0X04 /* Acks this TXSI. */ #define FORCED_MEASUREMENT 0X08 /* Force an RTT measurement. */ /* * This fuction measures the RTT of a particular segment/ack pair, or the next * closest if this will yield an inaccurate result due to delayed acking or * other issues. */ static void inline marked_packet_rtt(struct txseginfo *txsi, struct ertt *e_t, struct tcpcb *tp, uint32_t *pmeasurenext, int *pmeasurenext_len, int *prtt_bytes_adjust, int mflag) { /* * If we can't measure this one properly due to delayed acking adjust * byte counters and flag to measure next txsi. Note that since the * marked packet's transmitted bytes are measured we need to subtract the * transmitted bytes. Then pretend the next txsi was marked. */ if (mflag & (MULTI_ACK|OLD_TXSI)) { *pmeasurenext = txsi->tx_ts; *pmeasurenext_len = txsi->len; *prtt_bytes_adjust += *pmeasurenext_len; } else { if (mflag & FORCED_MEASUREMENT) { e_t->markedpkt_rtt = ticks - *pmeasurenext + 1; e_t->bytes_tx_in_marked_rtt = e_t->bytes_tx_in_rtt + *pmeasurenext_len - *prtt_bytes_adjust; } else { e_t->markedpkt_rtt = ticks - txsi->tx_ts + 1; e_t->bytes_tx_in_marked_rtt = e_t->bytes_tx_in_rtt - *prtt_bytes_adjust; } e_t->marked_snd_cwnd = tp->snd_cwnd; /* * Reset the ERTT_MEASUREMENT_IN_PROGRESS flag to indicate to * add_tx_segment_info that a new measurement should be started. */ e_t->flags &= ~ERTT_MEASUREMENT_IN_PROGRESS; /* * Set ERTT_NEW_MEASUREMENT to tell the congestion control * algorithm that a new marked RTT measurement has has been made * and is available for use. */ e_t->flags |= ERTT_NEW_MEASUREMENT; if (tp->t_flags & TF_TSO) { /* Temporarily disable TSO to aid a new measurment. */ tp->t_flags &= ~TF_TSO; /* Keep track that we've disabled it. */ e_t->flags |= ERTT_TSO_DISABLED; } } } /* * Ertt_packet_measurements uses a small amount of state kept on each packet * sent to match incoming acknowledgements. This enables more accurate and * secure round trip time measurements. The resulting measurement is used for * congestion control algorithms which require a more accurate time. * Ertt_packet_measurements is called via the helper hook in tcp_input.c */ static int ertt_packet_measurement_hook(int hhook_type, int hhook_id, void *udata, void *ctx_data, void *hdata, struct osd *hosd) { struct ertt *e_t; struct tcpcb *tp; struct tcphdr *th; struct tcpopt *to; struct tcp_hhook_data *thdp; struct txseginfo *txsi; int acked, measurenext_len, multiack, new_sacked_bytes, rtt_bytes_adjust; uint32_t measurenext, rts; tcp_seq ack; KASSERT(ctx_data != NULL, ("%s: ctx_data is NULL!", __func__)); KASSERT(hdata != NULL, ("%s: hdata is NULL!", __func__)); e_t = (struct ertt *)hdata; thdp = ctx_data; tp = thdp->tp; th = thdp->th; to = thdp->to; new_sacked_bytes = (tp->sackhint.last_sack_ack != 0); measurenext = measurenext_len = multiack = rts = rtt_bytes_adjust = 0; acked = th->th_ack - tp->snd_una; INP_WLOCK_ASSERT(tp->t_inpcb); /* Packet has provided new acknowledgements. */ if (acked > 0 || new_sacked_bytes) { if (acked == 0 && new_sacked_bytes) { /* Use last sacked data. */ ack = tp->sackhint.last_sack_ack; } else ack = th->th_ack; txsi = TAILQ_FIRST(&e_t->txsegi_q); while (txsi != NULL) { rts = 0; /* Acknowledgement is acking more than this txsi. */ if (SEQ_GT(ack, txsi->seq + txsi->len)) { if (txsi->flags & TXSI_RTT_MEASURE_START || measurenext) { marked_packet_rtt(txsi, e_t, tp, &measurenext, &measurenext_len, &rtt_bytes_adjust, MULTI_ACK); } TAILQ_REMOVE(&e_t->txsegi_q, txsi, txsegi_lnk); uma_zfree(txseginfo_zone, txsi); txsi = TAILQ_FIRST(&e_t->txsegi_q); continue; } /* * Guess if delayed acks are being used by the receiver. * * XXXDH: A simple heuristic that could be improved */ if (!new_sacked_bytes) { if (acked > tp->t_maxseg) { e_t->dlyack_rx += (e_t->dlyack_rx < DLYACK_SMOOTH) ? 1 : 0; multiack = 1; } else if (acked > txsi->len) { multiack = 1; e_t->dlyack_rx += (e_t->dlyack_rx < DLYACK_SMOOTH) ? 1 : 0; } else if (acked == tp->t_maxseg || acked == txsi->len) { e_t->dlyack_rx -= (e_t->dlyack_rx > 0) ? 1 : 0; } /* Otherwise leave dlyack_rx the way it was. */ } /* * Time stamps are only to help match the txsi with the * received acknowledgements. */ if (e_t->timestamp_errors < MAX_TS_ERR && (to->to_flags & TOF_TS) != 0 && to->to_tsecr) { /* * Note: All packets sent with the offload will * have the same time stamp. If we are sending * on a fast interface and the t_maxseg is much * smaller than one tick, this will be fine. The * time stamp would be the same whether we were * using tso or not. However, if the interface * is slow, this will cause problems with the * calculations. If the interface is slow, there * is not reason to be using tso, and it should * be turned off. */ /* * If there are too many time stamp errors, time * stamps won't be trusted */ rts = to->to_tsecr; /* Before this packet. */ if (!e_t->dlyack_rx && TSTMP_LT(rts, txsi->tx_ts)) /* When delayed acking is used, the * reflected time stamp is of the first * packet and thus may be before * txsi->tx_ts. */ break; if (TSTMP_GT(rts, txsi->tx_ts)) { /* * If reflected time stamp is later than * tx_tsi, then this txsi is old. */ if (txsi->flags & TXSI_RTT_MEASURE_START || measurenext) { marked_packet_rtt(txsi, e_t, tp, &measurenext, &measurenext_len, &rtt_bytes_adjust, OLD_TXSI); } TAILQ_REMOVE(&e_t->txsegi_q, txsi, txsegi_lnk); uma_zfree(txseginfo_zone, txsi); txsi = TAILQ_FIRST(&e_t->txsegi_q); continue; } if (rts == txsi->tx_ts && TSTMP_LT(to->to_tsval, txsi->rx_ts)) { /* * Segment received before sent! * Something is wrong with the received * timestamps so increment errors. If * this keeps up we will ignore * timestamps. */ e_t->timestamp_errors++; } } /* * Acknowledging a sequence number before this txsi. * If it is an old txsi that may have had the same seq * numbers, it should have been removed if time stamps * are being used. */ if (SEQ_LEQ(ack, txsi->seq)) break; /* Before first packet in txsi. */ /* * Only ack > txsi->seq and ack <= txsi->seq+txsi->len * past this point. * * If delayed acks are being used, an acknowledgement * for a single segment will have been delayed by the * receiver and will yield an inaccurate measurement. In * this case, we only make the measurement if more than * one segment is being acknowledged or sack is * currently being used. */ if (!e_t->dlyack_rx || multiack || new_sacked_bytes) { /* Make an accurate new measurement. */ e_t->rtt = ticks - txsi->tx_ts + 1; if (e_t->rtt < e_t->minrtt || e_t->minrtt == 0) e_t->minrtt = e_t->rtt; if (e_t->rtt > e_t->maxrtt || e_t->maxrtt == 0) e_t->maxrtt = e_t->rtt; } if (txsi->flags & TXSI_RTT_MEASURE_START || measurenext) marked_packet_rtt(txsi, e_t, tp, &measurenext, &measurenext_len, &rtt_bytes_adjust, CORRECT_ACK); if (txsi->flags & TXSI_TSO) { txsi->len -= acked; if (txsi->len > 0) { /* * This presumes ack for first bytes in * txsi, this may not be true but it * shouldn't cause problems for the * timing. * * We remeasure RTT even though we only * have a single txsi. The rationale * behind this is that it is better to * have a slightly inaccurate * measurement than no additional * measurement for the rest of the bulk * transfer. Since TSO is only used on * high speed interface cards, so the * packets should be transmitted at line * rate back to back with little * difference in transmission times (in * ticks). */ txsi->seq += acked; /* * Reset txsi measure flag so we don't * use it for another RTT measurement. */ txsi->flags &= ~TXSI_RTT_MEASURE_START; /* * There is still more data to be acked * from tso bulk transmission, so we * won't remove it from the TAILQ yet. */ break; } } TAILQ_REMOVE(&e_t->txsegi_q, txsi, txsegi_lnk); uma_zfree(txseginfo_zone, txsi); break; } if (measurenext) { /* * We need to do a RTT measurement. It won't be the best * if we do it here. */ marked_packet_rtt(txsi, e_t, tp, &measurenext, &measurenext_len, &rtt_bytes_adjust, FORCED_MEASUREMENT); } } return (0); } /* * Add information about a transmitted segment to a list. * This is called via the helper hook in tcp_output.c */ static int ertt_add_tx_segment_info_hook(int hhook_type, int hhook_id, void *udata, void *ctx_data, void *hdata, struct osd *hosd) { struct ertt *e_t; struct tcpcb *tp; struct tcphdr *th; struct tcpopt *to; struct tcp_hhook_data *thdp; struct txseginfo *txsi; long len; int tso; KASSERT(ctx_data != NULL, ("%s: ctx_data is NULL!", __func__)); KASSERT(hdata != NULL, ("%s: hdata is NULL!", __func__)); e_t = (struct ertt *)hdata; thdp = ctx_data; tp = thdp->tp; th = thdp->th; to = thdp->to; len = thdp->len; tso = thdp->tso; INP_WLOCK_ASSERT(tp->t_inpcb); if (len > 0) { txsi = uma_zalloc(txseginfo_zone, M_NOWAIT); if (txsi != NULL) { /* Construct txsi setting the necessary flags. */ txsi->flags = 0; /* Needs to be initialised. */ txsi->seq = ntohl(th->th_seq); txsi->len = len; if (tso) txsi->flags |= TXSI_TSO; else if (e_t->flags & ERTT_TSO_DISABLED) { tp->t_flags |= TF_TSO; e_t->flags &= ~ERTT_TSO_DISABLED; } if (e_t->flags & ERTT_MEASUREMENT_IN_PROGRESS) { e_t->bytes_tx_in_rtt += len; } else { txsi->flags |= TXSI_RTT_MEASURE_START; e_t->flags |= ERTT_MEASUREMENT_IN_PROGRESS; e_t->bytes_tx_in_rtt = len; } if (((tp->t_flags & TF_NOOPT) == 0) && (to->to_flags & TOF_TS)) { txsi->tx_ts = ntohl(to->to_tsval) - tp->ts_offset; txsi->rx_ts = ntohl(to->to_tsecr); } else { txsi->tx_ts = ticks; txsi->rx_ts = 0; /* No received time stamp. */ } TAILQ_INSERT_TAIL(&e_t->txsegi_q, txsi, txsegi_lnk); } } return (0); } static int ertt_mod_init(void) { txseginfo_zone = uma_zcreate("ertt_txseginfo", sizeof(struct txseginfo), NULL, NULL, NULL, NULL, 0, 0); return (0); } static int ertt_mod_destroy(void) { uma_zdestroy(txseginfo_zone); return (0); } static int ertt_uma_ctor(void *mem, int size, void *arg, int flags) { struct ertt *e_t; e_t = mem; TAILQ_INIT(&e_t->txsegi_q); e_t->timestamp_errors = 0; e_t->minrtt = 0; e_t->maxrtt = 0; e_t->rtt = 0; e_t->flags = 0; e_t->dlyack_rx = 0; e_t->bytes_tx_in_rtt = 0; e_t->markedpkt_rtt = 0; return (0); } static void ertt_uma_dtor(void *mem, int size, void *arg) { struct ertt *e_t; struct txseginfo *n_txsi, *txsi; e_t = mem; txsi = TAILQ_FIRST(&e_t->txsegi_q); while (txsi != NULL) { n_txsi = TAILQ_NEXT(txsi, txsegi_lnk); uma_zfree(txseginfo_zone, txsi); txsi = n_txsi; } } KHELP_DECLARE_MOD_UMA(ertt, &ertt_helper, ertt_hooks, 1, sizeof(struct ertt), ertt_uma_ctor, ertt_uma_dtor); Index: head/sys/netinet/khelp/h_ertt.h =================================================================== --- head/sys/netinet/khelp/h_ertt.h (revision 220559) +++ head/sys/netinet/khelp/h_ertt.h (revision 220560) @@ -1,89 +1,89 @@ /*- * Copyright (c) 2009-2010 * Swinburne University of Technology, Melbourne, Australia * Copyright (c) 2010 Lawrence Stewart * All rights reserved. * * This software was developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by David Hayes, made possible in part by - * a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by David Hayes, made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * The ERTT (Enhanced Round Trip Time) Khelp module calculates an estimate of * the instantaneous TCP RTT which, for example, is used by delay-based * congestion control schemes. When the module is loaded, ERTT data is * calculated for each active TCP connection and encapsulated within a * "struct ertt". * * This software was first released in 2010 by David Hayes and Lawrence Stewart - * whilst working on the NewTCP research project at Swinburne University's - * Centre for Advanced Internet Architectures, Melbourne, Australia, which was - * made possible in part by a grant from the Cisco University Research Program - * Fund at Community Foundation Silicon Valley. Testing and development was - * further assisted by a grant from the FreeBSD Foundation. More details are - * available at: + * whilst working on the NewTCP research project at Swinburne University of + * Technology's Centre for Advanced Internet Architectures, Melbourne, + * Australia, which was made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. + * Testing and development was further assisted by a grant from the FreeBSD + * Foundation. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #ifndef _NETINET_KHELP_H_ERTT_ #define _NETINET_KHELP_H_ERTT_ struct txseginfo; /* Structure used as the ertt data block. */ struct ertt { /* Information about transmitted segments to aid in RTT calculation. */ TAILQ_HEAD(txseginfo_head, txseginfo) txsegi_q; /* Bytes TX so far in marked RTT. */ long bytes_tx_in_rtt; /* Final version of above. */ long bytes_tx_in_marked_rtt; /* cwnd for marked RTT. */ unsigned long marked_snd_cwnd; /* Per-packet measured RTT. */ int rtt; /* Maximum RTT measured. */ int maxrtt; /* Minimum RTT measured. */ int minrtt; /* Guess if the receiver is using delayed ack. */ int dlyack_rx; /* Keep track of inconsistencies in packet timestamps. */ int timestamp_errors; /* RTT for a marked packet. */ int markedpkt_rtt; /* Flags to signal conditions between hook function calls. */ uint32_t flags; }; /* Flags for struct ertt. */ #define ERTT_NEW_MEASUREMENT 0x01 #define ERTT_MEASUREMENT_IN_PROGRESS 0x02 #define ERTT_TSO_DISABLED 0x04 #endif /* _NETINET_KHELP_H_ERTT_ */ Index: head/sys/netinet/siftr.c =================================================================== --- head/sys/netinet/siftr.c (revision 220559) +++ head/sys/netinet/siftr.c (revision 220560) @@ -1,1556 +1,1557 @@ /*- * Copyright (c) 2007-2009 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010, The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /****************************************************** * Statistical Information For TCP Research (SIFTR) * * A FreeBSD kernel module that adds very basic intrumentation to the * TCP stack, allowing internal stats to be recorded to a log file * for experimental, debugging and performance analysis purposes. * * SIFTR was first released in 2007 by James Healy and Lawrence Stewart whilst - * working on the NewTCP research project at Swinburne University's Centre for - * Advanced Internet Architectures, Melbourne, Australia, which was made - * possible in part by a grant from the Cisco University Research Program Fund - * at Community Foundation Silicon Valley. More details are available at: + * working on the NewTCP research project at Swinburne University of + * Technology's Centre for Advanced Internet Architectures, Melbourne, + * Australia, which was made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. + * More details are available at: * http://caia.swin.edu.au/urp/newtcp/ * * Work on SIFTR v1.2.x was sponsored by the FreeBSD Foundation as part of * the "Enhancing the FreeBSD TCP Implementation" project 2008-2009. * More details are available at: * http://www.freebsdfoundation.org/ * http://caia.swin.edu.au/freebsd/etcp09/ * * Lawrence Stewart is the current maintainer, and all contact regarding * SIFTR should be directed to him via email: lastewart@swin.edu.au * * Initial release date: June 2007 * Most recent update: September 2010 ******************************************************/ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef SIFTR_IPV6 #include #include #endif /* SIFTR_IPV6 */ #include /* * Three digit version number refers to X.Y.Z where: * X is the major version number * Y is bumped to mark backwards incompatible changes * Z is bumped to mark backwards compatible changes */ #define V_MAJOR 1 #define V_BACKBREAK 2 #define V_BACKCOMPAT 4 #define MODVERSION __CONCAT(V_MAJOR, __CONCAT(V_BACKBREAK, V_BACKCOMPAT)) #define MODVERSION_STR __XSTRING(V_MAJOR) "." __XSTRING(V_BACKBREAK) "." \ __XSTRING(V_BACKCOMPAT) #define HOOK 0 #define UNHOOK 1 #define SIFTR_EXPECTED_MAX_TCP_FLOWS 65536 #define SYS_NAME "FreeBSD" #define PACKET_TAG_SIFTR 100 #define PACKET_COOKIE_SIFTR 21749576 #define SIFTR_LOG_FILE_MODE 0644 #define SIFTR_DISABLE 0 #define SIFTR_ENABLE 1 /* * Hard upper limit on the length of log messages. Bump this up if you add new * data fields such that the line length could exceed the below value. */ #define MAX_LOG_MSG_LEN 200 /* XXX: Make this a sysctl tunable. */ #define SIFTR_ALQ_BUFLEN (1000*MAX_LOG_MSG_LEN) /* * 1 byte for IP version * IPv4: src/dst IP (4+4) + src/dst port (2+2) = 12 bytes * IPv6: src/dst IP (16+16) + src/dst port (2+2) = 36 bytes */ #ifdef SIFTR_IPV6 #define FLOW_KEY_LEN 37 #else #define FLOW_KEY_LEN 13 #endif #ifdef SIFTR_IPV6 #define SIFTR_IPMODE 6 #else #define SIFTR_IPMODE 4 #endif /* useful macros */ #define CAST_PTR_INT(X) (*((int*)(X))) #define UPPER_SHORT(X) (((X) & 0xFFFF0000) >> 16) #define LOWER_SHORT(X) ((X) & 0x0000FFFF) #define FIRST_OCTET(X) (((X) & 0xFF000000) >> 24) #define SECOND_OCTET(X) (((X) & 0x00FF0000) >> 16) #define THIRD_OCTET(X) (((X) & 0x0000FF00) >> 8) #define FOURTH_OCTET(X) ((X) & 0x000000FF) MALLOC_DECLARE(M_SIFTR); MALLOC_DEFINE(M_SIFTR, "siftr", "dynamic memory used by SIFTR"); MALLOC_DECLARE(M_SIFTR_PKTNODE); MALLOC_DEFINE(M_SIFTR_PKTNODE, "siftr_pktnode", "SIFTR pkt_node struct"); MALLOC_DECLARE(M_SIFTR_HASHNODE); MALLOC_DEFINE(M_SIFTR_HASHNODE, "siftr_hashnode", "SIFTR flow_hash_node struct"); /* Used as links in the pkt manager queue. */ struct pkt_node { /* Timestamp of pkt as noted in the pfil hook. */ struct timeval tval; /* Direction pkt is travelling; either PFIL_IN or PFIL_OUT. */ uint8_t direction; /* IP version pkt_node relates to; either INP_IPV4 or INP_IPV6. */ uint8_t ipver; /* Hash of the pkt which triggered the log message. */ uint32_t hash; /* Local/foreign IP address. */ #ifdef SIFTR_IPV6 uint32_t ip_laddr[4]; uint32_t ip_faddr[4]; #else uint8_t ip_laddr[4]; uint8_t ip_faddr[4]; #endif /* Local TCP port. */ uint16_t tcp_localport; /* Foreign TCP port. */ uint16_t tcp_foreignport; /* Congestion Window (bytes). */ u_long snd_cwnd; /* Sending Window (bytes). */ u_long snd_wnd; /* Receive Window (bytes). */ u_long rcv_wnd; /* Unused (was: Bandwidth Controlled Window (bytes)). */ u_long snd_bwnd; /* Slow Start Threshold (bytes). */ u_long snd_ssthresh; /* Current state of the TCP FSM. */ int conn_state; /* Max Segment Size (bytes). */ u_int max_seg_size; /* * Smoothed RTT stored as found in the TCP control block * in units of (TCP_RTT_SCALE*hz). */ int smoothed_rtt; /* Is SACK enabled? */ u_char sack_enabled; /* Window scaling for snd window. */ u_char snd_scale; /* Window scaling for recv window. */ u_char rcv_scale; /* TCP control block flags. */ u_int flags; /* Retransmit timeout length. */ int rxt_length; /* Size of the TCP send buffer in bytes. */ u_int snd_buf_hiwater; /* Current num bytes in the send socket buffer. */ u_int snd_buf_cc; /* Size of the TCP receive buffer in bytes. */ u_int rcv_buf_hiwater; /* Current num bytes in the receive socket buffer. */ u_int rcv_buf_cc; /* Number of bytes inflight that we are waiting on ACKs for. */ u_int sent_inflight_bytes; /* Number of segments currently in the reassembly queue. */ int t_segqlen; /* Link to next pkt_node in the list. */ STAILQ_ENTRY(pkt_node) nodes; }; struct flow_hash_node { uint16_t counter; uint8_t key[FLOW_KEY_LEN]; LIST_ENTRY(flow_hash_node) nodes; }; struct siftr_stats { /* # TCP pkts seen by the SIFTR PFIL hooks, including any skipped. */ uint64_t n_in; uint64_t n_out; /* # pkts skipped due to failed malloc calls. */ uint32_t nskip_in_malloc; uint32_t nskip_out_malloc; /* # pkts skipped due to failed mtx acquisition. */ uint32_t nskip_in_mtx; uint32_t nskip_out_mtx; /* # pkts skipped due to failed inpcb lookups. */ uint32_t nskip_in_inpcb; uint32_t nskip_out_inpcb; /* # pkts skipped due to failed tcpcb lookups. */ uint32_t nskip_in_tcpcb; uint32_t nskip_out_tcpcb; /* # pkts skipped due to stack reinjection. */ uint32_t nskip_in_dejavu; uint32_t nskip_out_dejavu; }; static DPCPU_DEFINE(struct siftr_stats, ss); static volatile unsigned int siftr_exit_pkt_manager_thread = 0; static unsigned int siftr_enabled = 0; static unsigned int siftr_pkts_per_log = 1; static unsigned int siftr_generate_hashes = 0; /* static unsigned int siftr_binary_log = 0; */ static char siftr_logfile[PATH_MAX] = "/var/log/siftr.log"; static u_long siftr_hashmask; STAILQ_HEAD(pkthead, pkt_node) pkt_queue = STAILQ_HEAD_INITIALIZER(pkt_queue); LIST_HEAD(listhead, flow_hash_node) *counter_hash; static int wait_for_pkt; static struct alq *siftr_alq = NULL; static struct mtx siftr_pkt_queue_mtx; static struct mtx siftr_pkt_mgr_mtx; static struct thread *siftr_pkt_manager_thr = NULL; /* * pfil.h defines PFIL_IN as 1 and PFIL_OUT as 2, * which we use as an index into this array. */ static char direction[3] = {'\0', 'i','o'}; /* Required function prototypes. */ static int siftr_sysctl_enabled_handler(SYSCTL_HANDLER_ARGS); static int siftr_sysctl_logfile_name_handler(SYSCTL_HANDLER_ARGS); /* Declare the net.inet.siftr sysctl tree and populate it. */ SYSCTL_DECL(_net_inet_siftr); SYSCTL_NODE(_net_inet, OID_AUTO, siftr, CTLFLAG_RW, NULL, "siftr related settings"); SYSCTL_PROC(_net_inet_siftr, OID_AUTO, enabled, CTLTYPE_UINT|CTLFLAG_RW, &siftr_enabled, 0, &siftr_sysctl_enabled_handler, "IU", "switch siftr module operations on/off"); SYSCTL_PROC(_net_inet_siftr, OID_AUTO, logfile, CTLTYPE_STRING|CTLFLAG_RW, &siftr_logfile, sizeof(siftr_logfile), &siftr_sysctl_logfile_name_handler, "A", "file to save siftr log messages to"); SYSCTL_UINT(_net_inet_siftr, OID_AUTO, ppl, CTLFLAG_RW, &siftr_pkts_per_log, 1, "number of packets between generating a log message"); SYSCTL_UINT(_net_inet_siftr, OID_AUTO, genhashes, CTLFLAG_RW, &siftr_generate_hashes, 0, "enable packet hash generation"); /* XXX: TODO SYSCTL_UINT(_net_inet_siftr, OID_AUTO, binary, CTLFLAG_RW, &siftr_binary_log, 0, "write log files in binary instead of ascii"); */ /* Begin functions. */ static void siftr_process_pkt(struct pkt_node * pkt_node) { struct flow_hash_node *hash_node; struct listhead *counter_list; struct siftr_stats *ss; struct ale *log_buf; uint8_t key[FLOW_KEY_LEN]; uint8_t found_match, key_offset; hash_node = NULL; ss = DPCPU_PTR(ss); found_match = 0; key_offset = 1; /* * Create the key that will be used to create a hash index * into our hash table. Our key consists of: * ipversion, localip, localport, foreignip, foreignport */ key[0] = pkt_node->ipver; memcpy(key + key_offset, &pkt_node->ip_laddr, sizeof(pkt_node->ip_laddr)); key_offset += sizeof(pkt_node->ip_laddr); memcpy(key + key_offset, &pkt_node->tcp_localport, sizeof(pkt_node->tcp_localport)); key_offset += sizeof(pkt_node->tcp_localport); memcpy(key + key_offset, &pkt_node->ip_faddr, sizeof(pkt_node->ip_faddr)); key_offset += sizeof(pkt_node->ip_faddr); memcpy(key + key_offset, &pkt_node->tcp_foreignport, sizeof(pkt_node->tcp_foreignport)); counter_list = counter_hash + (hash32_buf(key, sizeof(key), 0) & siftr_hashmask); /* * If the list is not empty i.e. the hash index has * been used by another flow previously. */ if (LIST_FIRST(counter_list) != NULL) { /* * Loop through the hash nodes in the list. * There should normally only be 1 hash node in the list, * except if there have been collisions at the hash index * computed by hash32_buf(). */ LIST_FOREACH(hash_node, counter_list, nodes) { /* * Check if the key for the pkt we are currently * processing is the same as the key stored in the * hash node we are currently processing. * If they are the same, then we've found the * hash node that stores the counter for the flow * the pkt belongs to. */ if (memcmp(hash_node->key, key, sizeof(key)) == 0) { found_match = 1; break; } } } /* If this flow hash hasn't been seen before or we have a collision. */ if (hash_node == NULL || !found_match) { /* Create a new hash node to store the flow's counter. */ hash_node = malloc(sizeof(struct flow_hash_node), M_SIFTR_HASHNODE, M_WAITOK); if (hash_node != NULL) { /* Initialise our new hash node list entry. */ hash_node->counter = 0; memcpy(hash_node->key, key, sizeof(key)); LIST_INSERT_HEAD(counter_list, hash_node, nodes); } else { /* Malloc failed. */ if (pkt_node->direction == PFIL_IN) ss->nskip_in_malloc++; else ss->nskip_out_malloc++; return; } } else if (siftr_pkts_per_log > 1) { /* * Taking the remainder of the counter divided * by the current value of siftr_pkts_per_log * and storing that in counter provides a neat * way to modulate the frequency of log * messages being written to the log file. */ hash_node->counter = (hash_node->counter + 1) % siftr_pkts_per_log; /* * If we have not seen enough packets since the last time * we wrote a log message for this connection, return. */ if (hash_node->counter > 0) return; } log_buf = alq_getn(siftr_alq, MAX_LOG_MSG_LEN, ALQ_WAITOK); if (log_buf == NULL) return; /* Should only happen if the ALQ is shutting down. */ #ifdef SIFTR_IPV6 pkt_node->ip_laddr[3] = ntohl(pkt_node->ip_laddr[3]); pkt_node->ip_faddr[3] = ntohl(pkt_node->ip_faddr[3]); if (pkt_node->ipver == INP_IPV6) { /* IPv6 packet */ pkt_node->ip_laddr[0] = ntohl(pkt_node->ip_laddr[0]); pkt_node->ip_laddr[1] = ntohl(pkt_node->ip_laddr[1]); pkt_node->ip_laddr[2] = ntohl(pkt_node->ip_laddr[2]); pkt_node->ip_faddr[0] = ntohl(pkt_node->ip_faddr[0]); pkt_node->ip_faddr[1] = ntohl(pkt_node->ip_faddr[1]); pkt_node->ip_faddr[2] = ntohl(pkt_node->ip_faddr[2]); /* Construct an IPv6 log message. */ log_buf->ae_bytesused = snprintf(log_buf->ae_data, MAX_LOG_MSG_LEN, "%c,0x%08x,%zd.%06ld,%x:%x:%x:%x:%x:%x:%x:%x,%u,%x:%x:%x:" "%x:%x:%x:%x:%x,%u,%ld,%ld,%ld,%ld,%ld,%u,%u,%u,%u,%u,%u," "%u,%d,%u,%u,%u,%u,%u,%u\n", direction[pkt_node->direction], pkt_node->hash, pkt_node->tval.tv_sec, pkt_node->tval.tv_usec, UPPER_SHORT(pkt_node->ip_laddr[0]), LOWER_SHORT(pkt_node->ip_laddr[0]), UPPER_SHORT(pkt_node->ip_laddr[1]), LOWER_SHORT(pkt_node->ip_laddr[1]), UPPER_SHORT(pkt_node->ip_laddr[2]), LOWER_SHORT(pkt_node->ip_laddr[2]), UPPER_SHORT(pkt_node->ip_laddr[3]), LOWER_SHORT(pkt_node->ip_laddr[3]), ntohs(pkt_node->tcp_localport), UPPER_SHORT(pkt_node->ip_faddr[0]), LOWER_SHORT(pkt_node->ip_faddr[0]), UPPER_SHORT(pkt_node->ip_faddr[1]), LOWER_SHORT(pkt_node->ip_faddr[1]), UPPER_SHORT(pkt_node->ip_faddr[2]), LOWER_SHORT(pkt_node->ip_faddr[2]), UPPER_SHORT(pkt_node->ip_faddr[3]), LOWER_SHORT(pkt_node->ip_faddr[3]), ntohs(pkt_node->tcp_foreignport), pkt_node->snd_ssthresh, pkt_node->snd_cwnd, pkt_node->snd_bwnd, pkt_node->snd_wnd, pkt_node->rcv_wnd, pkt_node->snd_scale, pkt_node->rcv_scale, pkt_node->conn_state, pkt_node->max_seg_size, pkt_node->smoothed_rtt, pkt_node->sack_enabled, pkt_node->flags, pkt_node->rxt_length, pkt_node->snd_buf_hiwater, pkt_node->snd_buf_cc, pkt_node->rcv_buf_hiwater, pkt_node->rcv_buf_cc, pkt_node->sent_inflight_bytes, pkt_node->t_segqlen); } else { /* IPv4 packet */ pkt_node->ip_laddr[0] = FIRST_OCTET(pkt_node->ip_laddr[3]); pkt_node->ip_laddr[1] = SECOND_OCTET(pkt_node->ip_laddr[3]); pkt_node->ip_laddr[2] = THIRD_OCTET(pkt_node->ip_laddr[3]); pkt_node->ip_laddr[3] = FOURTH_OCTET(pkt_node->ip_laddr[3]); pkt_node->ip_faddr[0] = FIRST_OCTET(pkt_node->ip_faddr[3]); pkt_node->ip_faddr[1] = SECOND_OCTET(pkt_node->ip_faddr[3]); pkt_node->ip_faddr[2] = THIRD_OCTET(pkt_node->ip_faddr[3]); pkt_node->ip_faddr[3] = FOURTH_OCTET(pkt_node->ip_faddr[3]); #endif /* SIFTR_IPV6 */ /* Construct an IPv4 log message. */ log_buf->ae_bytesused = snprintf(log_buf->ae_data, MAX_LOG_MSG_LEN, "%c,0x%08x,%jd.%06ld,%u.%u.%u.%u,%u,%u.%u.%u.%u,%u,%ld,%ld," "%ld,%ld,%ld,%u,%u,%u,%u,%u,%u,%u,%d,%u,%u,%u,%u,%u,%u\n", direction[pkt_node->direction], pkt_node->hash, (intmax_t)pkt_node->tval.tv_sec, pkt_node->tval.tv_usec, pkt_node->ip_laddr[0], pkt_node->ip_laddr[1], pkt_node->ip_laddr[2], pkt_node->ip_laddr[3], ntohs(pkt_node->tcp_localport), pkt_node->ip_faddr[0], pkt_node->ip_faddr[1], pkt_node->ip_faddr[2], pkt_node->ip_faddr[3], ntohs(pkt_node->tcp_foreignport), pkt_node->snd_ssthresh, pkt_node->snd_cwnd, pkt_node->snd_bwnd, pkt_node->snd_wnd, pkt_node->rcv_wnd, pkt_node->snd_scale, pkt_node->rcv_scale, pkt_node->conn_state, pkt_node->max_seg_size, pkt_node->smoothed_rtt, pkt_node->sack_enabled, pkt_node->flags, pkt_node->rxt_length, pkt_node->snd_buf_hiwater, pkt_node->snd_buf_cc, pkt_node->rcv_buf_hiwater, pkt_node->rcv_buf_cc, pkt_node->sent_inflight_bytes, pkt_node->t_segqlen); #ifdef SIFTR_IPV6 } #endif alq_post_flags(siftr_alq, log_buf, 0); } static void siftr_pkt_manager_thread(void *arg) { STAILQ_HEAD(pkthead, pkt_node) tmp_pkt_queue = STAILQ_HEAD_INITIALIZER(tmp_pkt_queue); struct pkt_node *pkt_node, *pkt_node_temp; uint8_t draining; draining = 2; mtx_lock(&siftr_pkt_mgr_mtx); /* draining == 0 when queue has been flushed and it's safe to exit. */ while (draining) { /* * Sleep until we are signalled to wake because thread has * been told to exit or until 1 tick has passed. */ mtx_sleep(&wait_for_pkt, &siftr_pkt_mgr_mtx, PWAIT, "pktwait", 1); /* Gain exclusive access to the pkt_node queue. */ mtx_lock(&siftr_pkt_queue_mtx); /* * Move pkt_queue to tmp_pkt_queue, which leaves * pkt_queue empty and ready to receive more pkt_nodes. */ STAILQ_CONCAT(&tmp_pkt_queue, &pkt_queue); /* * We've finished making changes to the list. Unlock it * so the pfil hooks can continue queuing pkt_nodes. */ mtx_unlock(&siftr_pkt_queue_mtx); /* * We can't hold a mutex whilst calling siftr_process_pkt * because ALQ might sleep waiting for buffer space. */ mtx_unlock(&siftr_pkt_mgr_mtx); /* Flush all pkt_nodes to the log file. */ STAILQ_FOREACH_SAFE(pkt_node, &tmp_pkt_queue, nodes, pkt_node_temp) { siftr_process_pkt(pkt_node); STAILQ_REMOVE_HEAD(&tmp_pkt_queue, nodes); free(pkt_node, M_SIFTR_PKTNODE); } KASSERT(STAILQ_EMPTY(&tmp_pkt_queue), ("SIFTR tmp_pkt_queue not empty after flush")); mtx_lock(&siftr_pkt_mgr_mtx); /* * If siftr_exit_pkt_manager_thread gets set during the window * where we are draining the tmp_pkt_queue above, there might * still be pkts in pkt_queue that need to be drained. * Allow one further iteration to occur after * siftr_exit_pkt_manager_thread has been set to ensure * pkt_queue is completely empty before we kill the thread. * * siftr_exit_pkt_manager_thread is set only after the pfil * hooks have been removed, so only 1 extra iteration * is needed to drain the queue. */ if (siftr_exit_pkt_manager_thread) draining--; } mtx_unlock(&siftr_pkt_mgr_mtx); /* Calls wakeup on this thread's struct thread ptr. */ kthread_exit(); } static uint32_t hash_pkt(struct mbuf *m, uint32_t offset) { uint32_t hash; hash = 0; while (m != NULL && offset > m->m_len) { /* * The IP packet payload does not start in this mbuf, so * need to figure out which mbuf it starts in and what offset * into the mbuf's data region the payload starts at. */ offset -= m->m_len; m = m->m_next; } while (m != NULL) { /* Ensure there is data in the mbuf */ if ((m->m_len - offset) > 0) hash = hash32_buf(m->m_data + offset, m->m_len - offset, hash); m = m->m_next; offset = 0; } return (hash); } /* * Check if a given mbuf has the SIFTR mbuf tag. If it does, log the fact that * it's a reinjected packet and return. If it doesn't, tag the mbuf and return. * Return value >0 means the caller should skip processing this mbuf. */ static inline int siftr_chkreinject(struct mbuf *m, int dir, struct siftr_stats *ss) { if (m_tag_locate(m, PACKET_COOKIE_SIFTR, PACKET_TAG_SIFTR, NULL) != NULL) { if (dir == PFIL_IN) ss->nskip_in_dejavu++; else ss->nskip_out_dejavu++; return (1); } else { struct m_tag *tag = m_tag_alloc(PACKET_COOKIE_SIFTR, PACKET_TAG_SIFTR, 0, M_NOWAIT); if (tag == NULL) { if (dir == PFIL_IN) ss->nskip_in_malloc++; else ss->nskip_out_malloc++; return (1); } m_tag_prepend(m, tag); } return (0); } /* * Look up an inpcb for a packet. Return the inpcb pointer if found, or NULL * otherwise. */ static inline struct inpcb * siftr_findinpcb(int ipver, struct ip *ip, struct mbuf *m, uint16_t sport, uint16_t dport, int dir, struct siftr_stats *ss) { struct inpcb *inp; /* We need the tcbinfo lock. */ INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); INP_INFO_RLOCK(&V_tcbinfo); if (dir == PFIL_IN) inp = (ipver == INP_IPV4 ? in_pcblookup_hash(&V_tcbinfo, ip->ip_src, sport, ip->ip_dst, dport, 0, m->m_pkthdr.rcvif) : #ifdef SIFTR_IPV6 in6_pcblookup_hash(&V_tcbinfo, &((struct ip6_hdr *)ip)->ip6_src, sport, &((struct ip6_hdr *)ip)->ip6_dst, dport, 0, m->m_pkthdr.rcvif) #else NULL #endif ); else inp = (ipver == INP_IPV4 ? in_pcblookup_hash(&V_tcbinfo, ip->ip_dst, dport, ip->ip_src, sport, 0, m->m_pkthdr.rcvif) : #ifdef SIFTR_IPV6 in6_pcblookup_hash(&V_tcbinfo, &((struct ip6_hdr *)ip)->ip6_dst, dport, &((struct ip6_hdr *)ip)->ip6_src, sport, 0, m->m_pkthdr.rcvif) #else NULL #endif ); /* If we can't find the inpcb, bail. */ if (inp == NULL) { if (dir == PFIL_IN) ss->nskip_in_inpcb++; else ss->nskip_out_inpcb++; } else { /* Acquire the inpcb lock. */ INP_UNLOCK_ASSERT(inp); INP_RLOCK(inp); } INP_INFO_RUNLOCK(&V_tcbinfo); return (inp); } static inline void siftr_siftdata(struct pkt_node *pn, struct inpcb *inp, struct tcpcb *tp, int ipver, int dir, int inp_locally_locked) { #ifdef SIFTR_IPV6 if (ipver == INP_IPV4) { pn->ip_laddr[3] = inp->inp_laddr.s_addr; pn->ip_faddr[3] = inp->inp_faddr.s_addr; #else *((uint32_t *)pn->ip_laddr) = inp->inp_laddr.s_addr; *((uint32_t *)pn->ip_faddr) = inp->inp_faddr.s_addr; #endif #ifdef SIFTR_IPV6 } else { pn->ip_laddr[0] = inp->in6p_laddr.s6_addr32[0]; pn->ip_laddr[1] = inp->in6p_laddr.s6_addr32[1]; pn->ip_laddr[2] = inp->in6p_laddr.s6_addr32[2]; pn->ip_laddr[3] = inp->in6p_laddr.s6_addr32[3]; pn->ip_faddr[0] = inp->in6p_faddr.s6_addr32[0]; pn->ip_faddr[1] = inp->in6p_faddr.s6_addr32[1]; pn->ip_faddr[2] = inp->in6p_faddr.s6_addr32[2]; pn->ip_faddr[3] = inp->in6p_faddr.s6_addr32[3]; } #endif pn->tcp_localport = inp->inp_lport; pn->tcp_foreignport = inp->inp_fport; pn->snd_cwnd = tp->snd_cwnd; pn->snd_wnd = tp->snd_wnd; pn->rcv_wnd = tp->rcv_wnd; pn->snd_bwnd = 0; /* Unused, kept for compat. */ pn->snd_ssthresh = tp->snd_ssthresh; pn->snd_scale = tp->snd_scale; pn->rcv_scale = tp->rcv_scale; pn->conn_state = tp->t_state; pn->max_seg_size = tp->t_maxseg; pn->smoothed_rtt = tp->t_srtt; pn->sack_enabled = (tp->t_flags & TF_SACK_PERMIT) != 0; pn->flags = tp->t_flags; pn->rxt_length = tp->t_rxtcur; pn->snd_buf_hiwater = inp->inp_socket->so_snd.sb_hiwat; pn->snd_buf_cc = inp->inp_socket->so_snd.sb_cc; pn->rcv_buf_hiwater = inp->inp_socket->so_rcv.sb_hiwat; pn->rcv_buf_cc = inp->inp_socket->so_rcv.sb_cc; pn->sent_inflight_bytes = tp->snd_max - tp->snd_una; pn->t_segqlen = tp->t_segqlen; /* We've finished accessing the tcb so release the lock. */ if (inp_locally_locked) INP_RUNLOCK(inp); pn->ipver = ipver; pn->direction = dir; /* * Significantly more accurate than using getmicrotime(), but slower! * Gives true microsecond resolution at the expense of a hit to * maximum pps throughput processing when SIFTR is loaded and enabled. */ microtime(&pn->tval); } /* * pfil hook that is called for each IPv4 packet making its way through the * stack in either direction. * The pfil subsystem holds a non-sleepable mutex somewhere when * calling our hook function, so we can't sleep at all. * It's very important to use the M_NOWAIT flag with all function calls * that support it so that they won't sleep, otherwise you get a panic. */ static int siftr_chkpkt(void *arg, struct mbuf **m, struct ifnet *ifp, int dir, struct inpcb *inp) { struct pkt_node *pn; struct ip *ip; struct tcphdr *th; struct tcpcb *tp; struct siftr_stats *ss; unsigned int ip_hl; int inp_locally_locked; inp_locally_locked = 0; ss = DPCPU_PTR(ss); /* * m_pullup is not required here because ip_{input|output} * already do the heavy lifting for us. */ ip = mtod(*m, struct ip *); /* Only continue processing if the packet is TCP. */ if (ip->ip_p != IPPROTO_TCP) goto ret; /* * If a kernel subsystem reinjects packets into the stack, our pfil * hook will be called multiple times for the same packet. * Make sure we only process unique packets. */ if (siftr_chkreinject(*m, dir, ss)) goto ret; if (dir == PFIL_IN) ss->n_in++; else ss->n_out++; /* * Create a tcphdr struct starting at the correct offset * in the IP packet. ip->ip_hl gives the ip header length * in 4-byte words, so multiply it to get the size in bytes. */ ip_hl = (ip->ip_hl << 2); th = (struct tcphdr *)((caddr_t)ip + ip_hl); /* * If the pfil hooks don't provide a pointer to the * inpcb, we need to find it ourselves and lock it. */ if (!inp) { /* Find the corresponding inpcb for this pkt. */ inp = siftr_findinpcb(INP_IPV4, ip, *m, th->th_sport, th->th_dport, dir, ss); if (inp == NULL) goto ret; else inp_locally_locked = 1; } INP_LOCK_ASSERT(inp); /* Find the TCP control block that corresponds with this packet */ tp = intotcpcb(inp); /* * If we can't find the TCP control block (happens occasionaly for a * packet sent during the shutdown phase of a TCP connection), * or we're in the timewait state, bail */ if (tp == NULL || inp->inp_flags & INP_TIMEWAIT) { if (dir == PFIL_IN) ss->nskip_in_tcpcb++; else ss->nskip_out_tcpcb++; goto inp_unlock; } pn = malloc(sizeof(struct pkt_node), M_SIFTR_PKTNODE, M_NOWAIT|M_ZERO); if (pn == NULL) { if (dir == PFIL_IN) ss->nskip_in_malloc++; else ss->nskip_out_malloc++; goto inp_unlock; } siftr_siftdata(pn, inp, tp, INP_IPV4, dir, inp_locally_locked); if (siftr_generate_hashes) { if ((*m)->m_pkthdr.csum_flags & CSUM_TCP) { /* * For outbound packets, the TCP checksum isn't * calculated yet. This is a problem for our packet * hashing as the receiver will calc a different hash * to ours if we don't include the correct TCP checksum * in the bytes being hashed. To work around this * problem, we manually calc the TCP checksum here in * software. We unset the CSUM_TCP flag so the lower * layers don't recalc it. */ (*m)->m_pkthdr.csum_flags &= ~CSUM_TCP; /* * Calculate the TCP checksum in software and assign * to correct TCP header field, which will follow the * packet mbuf down the stack. The trick here is that * tcp_output() sets th->th_sum to the checksum of the * pseudo header for us already. Because of the nature * of the checksumming algorithm, we can sum over the * entire IP payload (i.e. TCP header and data), which * will include the already calculated pseduo header * checksum, thus giving us the complete TCP checksum. * * To put it in simple terms, if checksum(1,2,3,4)=10, * then checksum(1,2,3,4,5) == checksum(10,5). * This property is what allows us to "cheat" and * checksum only the IP payload which has the TCP * th_sum field populated with the pseudo header's * checksum, and not need to futz around checksumming * pseudo header bytes and TCP header/data in one hit. * Refer to RFC 1071 for more info. * * NB: in_cksum_skip(struct mbuf *m, int len, int skip) * in_cksum_skip 2nd argument is NOT the number of * bytes to read from the mbuf at "skip" bytes offset * from the start of the mbuf (very counter intuitive!). * The number of bytes to read is calculated internally * by the function as len-skip i.e. to sum over the IP * payload (TCP header + data) bytes, it is INCORRECT * to call the function like this: * in_cksum_skip(at, ip->ip_len - offset, offset) * Rather, it should be called like this: * in_cksum_skip(at, ip->ip_len, offset) * which means read "ip->ip_len - offset" bytes from * the mbuf cluster "at" at offset "offset" bytes from * the beginning of the "at" mbuf's data pointer. */ th->th_sum = in_cksum_skip(*m, ip->ip_len, ip_hl); } /* * XXX: Having to calculate the checksum in software and then * hash over all bytes is really inefficient. Would be nice to * find a way to create the hash and checksum in the same pass * over the bytes. */ pn->hash = hash_pkt(*m, ip_hl); } mtx_lock(&siftr_pkt_queue_mtx); STAILQ_INSERT_TAIL(&pkt_queue, pn, nodes); mtx_unlock(&siftr_pkt_queue_mtx); goto ret; inp_unlock: if (inp_locally_locked) INP_RUNLOCK(inp); ret: /* Returning 0 ensures pfil will not discard the pkt */ return (0); } #ifdef SIFTR_IPV6 static int siftr_chkpkt6(void *arg, struct mbuf **m, struct ifnet *ifp, int dir, struct inpcb *inp) { struct pkt_node *pn; struct ip6_hdr *ip6; struct tcphdr *th; struct tcpcb *tp; struct siftr_stats *ss; unsigned int ip6_hl; int inp_locally_locked; inp_locally_locked = 0; ss = DPCPU_PTR(ss); /* * m_pullup is not required here because ip6_{input|output} * already do the heavy lifting for us. */ ip6 = mtod(*m, struct ip6_hdr *); /* * Only continue processing if the packet is TCP * XXX: We should follow the next header fields * as shown on Pg 6 RFC 2460, but right now we'll * only check pkts that have no extension headers. */ if (ip6->ip6_nxt != IPPROTO_TCP) goto ret6; /* * If a kernel subsystem reinjects packets into the stack, our pfil * hook will be called multiple times for the same packet. * Make sure we only process unique packets. */ if (siftr_chkreinject(*m, dir, ss)) goto ret6; if (dir == PFIL_IN) ss->n_in++; else ss->n_out++; ip6_hl = sizeof(struct ip6_hdr); /* * Create a tcphdr struct starting at the correct offset * in the ipv6 packet. ip->ip_hl gives the ip header length * in 4-byte words, so multiply it to get the size in bytes. */ th = (struct tcphdr *)((caddr_t)ip6 + ip6_hl); /* * For inbound packets, the pfil hooks don't provide a pointer to the * inpcb, so we need to find it ourselves and lock it. */ if (!inp) { /* Find the corresponding inpcb for this pkt. */ inp = siftr_findinpcb(INP_IPV6, (struct ip *)ip6, *m, th->th_sport, th->th_dport, dir, ss); if (inp == NULL) goto ret6; else inp_locally_locked = 1; } /* Find the TCP control block that corresponds with this packet. */ tp = intotcpcb(inp); /* * If we can't find the TCP control block (happens occasionaly for a * packet sent during the shutdown phase of a TCP connection), * or we're in the timewait state, bail. */ if (tp == NULL || inp->inp_flags & INP_TIMEWAIT) { if (dir == PFIL_IN) ss->nskip_in_tcpcb++; else ss->nskip_out_tcpcb++; goto inp_unlock6; } pn = malloc(sizeof(struct pkt_node), M_SIFTR_PKTNODE, M_NOWAIT|M_ZERO); if (pn == NULL) { if (dir == PFIL_IN) ss->nskip_in_malloc++; else ss->nskip_out_malloc++; goto inp_unlock6; } siftr_siftdata(pn, inp, tp, INP_IPV6, dir, inp_locally_locked); /* XXX: Figure out how to generate hashes for IPv6 packets. */ mtx_lock(&siftr_pkt_queue_mtx); STAILQ_INSERT_TAIL(&pkt_queue, pn, nodes); mtx_unlock(&siftr_pkt_queue_mtx); goto ret6; inp_unlock6: if (inp_locally_locked) INP_RUNLOCK(inp); ret6: /* Returning 0 ensures pfil will not discard the pkt. */ return (0); } #endif /* #ifdef SIFTR_IPV6 */ static int siftr_pfil(int action) { struct pfil_head *pfh_inet; #ifdef SIFTR_IPV6 struct pfil_head *pfh_inet6; #endif VNET_ITERATOR_DECL(vnet_iter); VNET_LIST_RLOCK(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); pfh_inet = pfil_head_get(PFIL_TYPE_AF, AF_INET); #ifdef SIFTR_IPV6 pfh_inet6 = pfil_head_get(PFIL_TYPE_AF, AF_INET6); #endif if (action == HOOK) { pfil_add_hook(siftr_chkpkt, NULL, PFIL_IN | PFIL_OUT | PFIL_WAITOK, pfh_inet); #ifdef SIFTR_IPV6 pfil_add_hook(siftr_chkpkt6, NULL, PFIL_IN | PFIL_OUT | PFIL_WAITOK, pfh_inet6); #endif } else if (action == UNHOOK) { pfil_remove_hook(siftr_chkpkt, NULL, PFIL_IN | PFIL_OUT | PFIL_WAITOK, pfh_inet); #ifdef SIFTR_IPV6 pfil_remove_hook(siftr_chkpkt6, NULL, PFIL_IN | PFIL_OUT | PFIL_WAITOK, pfh_inet6); #endif } CURVNET_RESTORE(); } VNET_LIST_RUNLOCK(); return (0); } static int siftr_sysctl_logfile_name_handler(SYSCTL_HANDLER_ARGS) { struct alq *new_alq; int error; if (req->newptr == NULL) goto skip; /* If old filename and new filename are different. */ if (strncmp(siftr_logfile, (char *)req->newptr, PATH_MAX)) { error = alq_open(&new_alq, req->newptr, curthread->td_ucred, SIFTR_LOG_FILE_MODE, SIFTR_ALQ_BUFLEN, 0); /* Bail if unable to create new alq. */ if (error) return (1); /* * If disabled, siftr_alq == NULL so we simply close * the alq as we've proved it can be opened. * If enabled, close the existing alq and switch the old * for the new. */ if (siftr_alq == NULL) alq_close(new_alq); else { alq_close(siftr_alq); siftr_alq = new_alq; } } skip: return (sysctl_handle_string(oidp, arg1, arg2, req)); } static int siftr_manage_ops(uint8_t action) { struct siftr_stats totalss; struct timeval tval; struct flow_hash_node *counter, *tmp_counter; struct sbuf *s; int i, key_index, ret, error; uint32_t bytes_to_write, total_skipped_pkts; uint16_t lport, fport; uint8_t *key, ipver; #ifdef SIFTR_IPV6 uint32_t laddr[4]; uint32_t faddr[4]; #else uint8_t laddr[4]; uint8_t faddr[4]; #endif error = 0; total_skipped_pkts = 0; /* Init an autosizing sbuf that initially holds 200 chars. */ if ((s = sbuf_new(NULL, NULL, 200, SBUF_AUTOEXTEND)) == NULL) return (-1); if (action == SIFTR_ENABLE) { /* * Create our alq * XXX: We should abort if alq_open fails! */ alq_open(&siftr_alq, siftr_logfile, curthread->td_ucred, SIFTR_LOG_FILE_MODE, SIFTR_ALQ_BUFLEN, 0); STAILQ_INIT(&pkt_queue); DPCPU_ZERO(ss); siftr_exit_pkt_manager_thread = 0; ret = kthread_add(&siftr_pkt_manager_thread, NULL, NULL, &siftr_pkt_manager_thr, RFNOWAIT, 0, "siftr_pkt_manager_thr"); siftr_pfil(HOOK); microtime(&tval); sbuf_printf(s, "enable_time_secs=%jd\tenable_time_usecs=%06ld\t" "siftrver=%s\thz=%u\ttcp_rtt_scale=%u\tsysname=%s\t" "sysver=%u\tipmode=%u\n", (intmax_t)tval.tv_sec, tval.tv_usec, MODVERSION_STR, hz, TCP_RTT_SCALE, SYS_NAME, __FreeBSD_version, SIFTR_IPMODE); sbuf_finish(s); alq_writen(siftr_alq, sbuf_data(s), sbuf_len(s), ALQ_WAITOK); } else if (action == SIFTR_DISABLE && siftr_pkt_manager_thr != NULL) { /* * Remove the pfil hook functions. All threads currently in * the hook functions are allowed to exit before siftr_pfil() * returns. */ siftr_pfil(UNHOOK); /* This will block until the pkt manager thread unlocks it. */ mtx_lock(&siftr_pkt_mgr_mtx); /* Tell the pkt manager thread that it should exit now. */ siftr_exit_pkt_manager_thread = 1; /* * Wake the pkt_manager thread so it realises that * siftr_exit_pkt_manager_thread == 1 and exits gracefully. * The wakeup won't be delivered until we unlock * siftr_pkt_mgr_mtx so this isn't racy. */ wakeup(&wait_for_pkt); /* Wait for the pkt_manager thread to exit. */ mtx_sleep(siftr_pkt_manager_thr, &siftr_pkt_mgr_mtx, PWAIT, "thrwait", 0); siftr_pkt_manager_thr = NULL; mtx_unlock(&siftr_pkt_mgr_mtx); totalss.n_in = DPCPU_VARSUM(ss, n_in); totalss.n_out = DPCPU_VARSUM(ss, n_out); totalss.nskip_in_malloc = DPCPU_VARSUM(ss, nskip_in_malloc); totalss.nskip_out_malloc = DPCPU_VARSUM(ss, nskip_out_malloc); totalss.nskip_in_mtx = DPCPU_VARSUM(ss, nskip_in_mtx); totalss.nskip_out_mtx = DPCPU_VARSUM(ss, nskip_out_mtx); totalss.nskip_in_tcpcb = DPCPU_VARSUM(ss, nskip_in_tcpcb); totalss.nskip_out_tcpcb = DPCPU_VARSUM(ss, nskip_out_tcpcb); totalss.nskip_in_inpcb = DPCPU_VARSUM(ss, nskip_in_inpcb); totalss.nskip_out_inpcb = DPCPU_VARSUM(ss, nskip_out_inpcb); total_skipped_pkts = totalss.nskip_in_malloc + totalss.nskip_out_malloc + totalss.nskip_in_mtx + totalss.nskip_out_mtx + totalss.nskip_in_tcpcb + totalss.nskip_out_tcpcb + totalss.nskip_in_inpcb + totalss.nskip_out_inpcb; microtime(&tval); sbuf_printf(s, "disable_time_secs=%jd\tdisable_time_usecs=%06ld\t" "num_inbound_tcp_pkts=%ju\tnum_outbound_tcp_pkts=%ju\t" "total_tcp_pkts=%ju\tnum_inbound_skipped_pkts_malloc=%u\t" "num_outbound_skipped_pkts_malloc=%u\t" "num_inbound_skipped_pkts_mtx=%u\t" "num_outbound_skipped_pkts_mtx=%u\t" "num_inbound_skipped_pkts_tcpcb=%u\t" "num_outbound_skipped_pkts_tcpcb=%u\t" "num_inbound_skipped_pkts_inpcb=%u\t" "num_outbound_skipped_pkts_inpcb=%u\t" "total_skipped_tcp_pkts=%u\tflow_list=", (intmax_t)tval.tv_sec, tval.tv_usec, (uintmax_t)totalss.n_in, (uintmax_t)totalss.n_out, (uintmax_t)(totalss.n_in + totalss.n_out), totalss.nskip_in_malloc, totalss.nskip_out_malloc, totalss.nskip_in_mtx, totalss.nskip_out_mtx, totalss.nskip_in_tcpcb, totalss.nskip_out_tcpcb, totalss.nskip_in_inpcb, totalss.nskip_out_inpcb, total_skipped_pkts); /* * Iterate over the flow hash, printing a summary of each * flow seen and freeing any malloc'd memory. * The hash consists of an array of LISTs (man 3 queue). */ for (i = 0; i < siftr_hashmask; i++) { LIST_FOREACH_SAFE(counter, counter_hash + i, nodes, tmp_counter) { key = counter->key; key_index = 1; ipver = key[0]; memcpy(laddr, key + key_index, sizeof(laddr)); key_index += sizeof(laddr); memcpy(&lport, key + key_index, sizeof(lport)); key_index += sizeof(lport); memcpy(faddr, key + key_index, sizeof(faddr)); key_index += sizeof(faddr); memcpy(&fport, key + key_index, sizeof(fport)); #ifdef SIFTR_IPV6 laddr[3] = ntohl(laddr[3]); faddr[3] = ntohl(faddr[3]); if (ipver == INP_IPV6) { laddr[0] = ntohl(laddr[0]); laddr[1] = ntohl(laddr[1]); laddr[2] = ntohl(laddr[2]); faddr[0] = ntohl(faddr[0]); faddr[1] = ntohl(faddr[1]); faddr[2] = ntohl(faddr[2]); sbuf_printf(s, "%x:%x:%x:%x:%x:%x:%x:%x;%u-" "%x:%x:%x:%x:%x:%x:%x:%x;%u,", UPPER_SHORT(laddr[0]), LOWER_SHORT(laddr[0]), UPPER_SHORT(laddr[1]), LOWER_SHORT(laddr[1]), UPPER_SHORT(laddr[2]), LOWER_SHORT(laddr[2]), UPPER_SHORT(laddr[3]), LOWER_SHORT(laddr[3]), ntohs(lport), UPPER_SHORT(faddr[0]), LOWER_SHORT(faddr[0]), UPPER_SHORT(faddr[1]), LOWER_SHORT(faddr[1]), UPPER_SHORT(faddr[2]), LOWER_SHORT(faddr[2]), UPPER_SHORT(faddr[3]), LOWER_SHORT(faddr[3]), ntohs(fport)); } else { laddr[0] = FIRST_OCTET(laddr[3]); laddr[1] = SECOND_OCTET(laddr[3]); laddr[2] = THIRD_OCTET(laddr[3]); laddr[3] = FOURTH_OCTET(laddr[3]); faddr[0] = FIRST_OCTET(faddr[3]); faddr[1] = SECOND_OCTET(faddr[3]); faddr[2] = THIRD_OCTET(faddr[3]); faddr[3] = FOURTH_OCTET(faddr[3]); #endif sbuf_printf(s, "%u.%u.%u.%u;%u-%u.%u.%u.%u;%u,", laddr[0], laddr[1], laddr[2], laddr[3], ntohs(lport), faddr[0], faddr[1], faddr[2], faddr[3], ntohs(fport)); #ifdef SIFTR_IPV6 } #endif free(counter, M_SIFTR_HASHNODE); } LIST_INIT(counter_hash + i); } sbuf_printf(s, "\n"); sbuf_finish(s); i = 0; do { bytes_to_write = min(SIFTR_ALQ_BUFLEN, sbuf_len(s)-i); alq_writen(siftr_alq, sbuf_data(s)+i, bytes_to_write, ALQ_WAITOK); i += bytes_to_write; } while (i < sbuf_len(s)); alq_close(siftr_alq); siftr_alq = NULL; } sbuf_delete(s); /* * XXX: Should be using ret to check if any functions fail * and set error appropriately */ return (error); } static int siftr_sysctl_enabled_handler(SYSCTL_HANDLER_ARGS) { if (req->newptr == NULL) goto skip; /* If the value passed in isn't 0 or 1, return an error. */ if (CAST_PTR_INT(req->newptr) != 0 && CAST_PTR_INT(req->newptr) != 1) return (1); /* If we are changing state (0 to 1 or 1 to 0). */ if (CAST_PTR_INT(req->newptr) != siftr_enabled ) if (siftr_manage_ops(CAST_PTR_INT(req->newptr))) { siftr_manage_ops(SIFTR_DISABLE); return (1); } skip: return (sysctl_handle_int(oidp, arg1, arg2, req)); } static void siftr_shutdown_handler(void *arg) { siftr_manage_ops(SIFTR_DISABLE); } /* * Module is being unloaded or machine is shutting down. Take care of cleanup. */ static int deinit_siftr(void) { /* Cleanup. */ siftr_manage_ops(SIFTR_DISABLE); hashdestroy(counter_hash, M_SIFTR, siftr_hashmask); mtx_destroy(&siftr_pkt_queue_mtx); mtx_destroy(&siftr_pkt_mgr_mtx); return (0); } /* * Module has just been loaded into the kernel. */ static int init_siftr(void) { EVENTHANDLER_REGISTER(shutdown_pre_sync, siftr_shutdown_handler, NULL, SHUTDOWN_PRI_FIRST); /* Initialise our flow counter hash table. */ counter_hash = hashinit(SIFTR_EXPECTED_MAX_TCP_FLOWS, M_SIFTR, &siftr_hashmask); mtx_init(&siftr_pkt_queue_mtx, "siftr_pkt_queue_mtx", NULL, MTX_DEF); mtx_init(&siftr_pkt_mgr_mtx, "siftr_pkt_mgr_mtx", NULL, MTX_DEF); /* Print message to the user's current terminal. */ uprintf("\nStatistical Information For TCP Research (SIFTR) %s\n" " http://caia.swin.edu.au/urp/newtcp\n\n", MODVERSION_STR); return (0); } /* * This is the function that is called to load and unload the module. * When the module is loaded, this function is called once with * "what" == MOD_LOAD * When the module is unloaded, this function is called twice with * "what" = MOD_QUIESCE first, followed by "what" = MOD_UNLOAD second * When the system is shut down e.g. CTRL-ALT-DEL or using the shutdown command, * this function is called once with "what" = MOD_SHUTDOWN * When the system is shut down, the handler isn't called until the very end * of the shutdown sequence i.e. after the disks have been synced. */ static int siftr_load_handler(module_t mod, int what, void *arg) { int ret; switch (what) { case MOD_LOAD: ret = init_siftr(); break; case MOD_QUIESCE: case MOD_SHUTDOWN: ret = deinit_siftr(); break; case MOD_UNLOAD: ret = 0; break; default: ret = EINVAL; break; } return (ret); } static moduledata_t siftr_mod = { .name = "siftr", .evhand = siftr_load_handler, }; /* * Param 1: name of the kernel module * Param 2: moduledata_t struct containing info about the kernel module * and the execution entry point for the module * Param 3: From sysinit_sub_id enumeration in /usr/include/sys/kernel.h * Defines the module initialisation order * Param 4: From sysinit_elem_order enumeration in /usr/include/sys/kernel.h * Defines the initialisation order of this kld relative to others * within the same subsystem as defined by param 3 */ DECLARE_MODULE(siftr, siftr_mod, SI_SUB_SMP, SI_ORDER_ANY); MODULE_DEPEND(siftr, alq, 1, 1, 1); MODULE_VERSION(siftr, MODVERSION); Index: head/sys/netinet/tcp_input.c =================================================================== --- head/sys/netinet/tcp_input.c (revision 220559) +++ head/sys/netinet/tcp_input.c (revision 220560) @@ -1,3498 +1,3498 @@ /*- * Copyright (c) 1982, 1986, 1988, 1990, 1993, 1994, 1995 * The Regents of the University of California. All rights reserved. * Copyright (c) 2007-2008,2010 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed at the Centre for Advanced Internet - * Architectures, Swinburne University, by Lawrence Stewart, James Healy and - * David Hayes, made possible in part by a grant from the Cisco University - * Research Program Fund at Community Foundation Silicon Valley. + * Architectures, Swinburne University of Technology, by Lawrence Stewart, + * James Healy and David Hayes, made possible in part by a grant from the Cisco + * University Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)tcp_input.c 8.12 (Berkeley) 5/24/95 */ #include __FBSDID("$FreeBSD$"); #include "opt_ipfw.h" /* for ipfw_fwd */ #include "opt_inet.h" #include "opt_inet6.h" #include "opt_ipsec.h" #include "opt_tcpdebug.h" #include #include #include #include #include #include /* for proc0 declaration */ #include #include #include #include #include #include #include #include /* before tcp_seq.h, for tcp_random18() */ #include #include #include #include #define TCPSTATES /* for logging */ #include #include #include #include #include #include #include /* required for icmp_var.h */ #include /* for ICMP_BANDLIM */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef TCPDEBUG #include #endif /* TCPDEBUG */ #ifdef IPSEC #include #include #endif /*IPSEC*/ #include #include const int tcprexmtthresh = 3; VNET_DEFINE(struct tcpstat, tcpstat); SYSCTL_VNET_STRUCT(_net_inet_tcp, TCPCTL_STATS, stats, CTLFLAG_RW, &VNET_NAME(tcpstat), tcpstat, "TCP statistics (struct tcpstat, netinet/tcp_var.h)"); int tcp_log_in_vain = 0; SYSCTL_INT(_net_inet_tcp, OID_AUTO, log_in_vain, CTLFLAG_RW, &tcp_log_in_vain, 0, "Log all incoming TCP segments to closed ports"); VNET_DEFINE(int, blackhole) = 0; #define V_blackhole VNET(blackhole) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, blackhole, CTLFLAG_RW, &VNET_NAME(blackhole), 0, "Do not send RST on segments to closed ports"); VNET_DEFINE(int, tcp_delack_enabled) = 1; SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, delayed_ack, CTLFLAG_RW, &VNET_NAME(tcp_delack_enabled), 0, "Delay ACK to try and piggyback it onto a data packet"); VNET_DEFINE(int, drop_synfin) = 0; #define V_drop_synfin VNET(drop_synfin) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, drop_synfin, CTLFLAG_RW, &VNET_NAME(drop_synfin), 0, "Drop TCP packets with SYN+FIN set"); VNET_DEFINE(int, tcp_do_rfc3042) = 1; #define V_tcp_do_rfc3042 VNET(tcp_do_rfc3042) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, rfc3042, CTLFLAG_RW, &VNET_NAME(tcp_do_rfc3042), 0, "Enable RFC 3042 (Limited Transmit)"); VNET_DEFINE(int, tcp_do_rfc3390) = 1; SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, rfc3390, CTLFLAG_RW, &VNET_NAME(tcp_do_rfc3390), 0, "Enable RFC 3390 (Increasing TCP's Initial Congestion Window)"); VNET_DEFINE(int, tcp_do_rfc3465) = 1; SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, rfc3465, CTLFLAG_RW, &VNET_NAME(tcp_do_rfc3465), 0, "Enable RFC 3465 (Appropriate Byte Counting)"); VNET_DEFINE(int, tcp_abc_l_var) = 2; SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, abc_l_var, CTLFLAG_RW, &VNET_NAME(tcp_abc_l_var), 2, "Cap the max cwnd increment during slow-start to this number of segments"); SYSCTL_NODE(_net_inet_tcp, OID_AUTO, ecn, CTLFLAG_RW, 0, "TCP ECN"); VNET_DEFINE(int, tcp_do_ecn) = 0; SYSCTL_VNET_INT(_net_inet_tcp_ecn, OID_AUTO, enable, CTLFLAG_RW, &VNET_NAME(tcp_do_ecn), 0, "TCP ECN support"); VNET_DEFINE(int, tcp_ecn_maxretries) = 1; SYSCTL_VNET_INT(_net_inet_tcp_ecn, OID_AUTO, maxretries, CTLFLAG_RW, &VNET_NAME(tcp_ecn_maxretries), 0, "Max retries before giving up on ECN"); VNET_DEFINE(int, tcp_insecure_rst) = 0; #define V_tcp_insecure_rst VNET(tcp_insecure_rst) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, insecure_rst, CTLFLAG_RW, &VNET_NAME(tcp_insecure_rst), 0, "Follow the old (insecure) criteria for accepting RST packets"); VNET_DEFINE(int, tcp_do_autorcvbuf) = 1; #define V_tcp_do_autorcvbuf VNET(tcp_do_autorcvbuf) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_auto, CTLFLAG_RW, &VNET_NAME(tcp_do_autorcvbuf), 0, "Enable automatic receive buffer sizing"); VNET_DEFINE(int, tcp_autorcvbuf_inc) = 16*1024; #define V_tcp_autorcvbuf_inc VNET(tcp_autorcvbuf_inc) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_inc, CTLFLAG_RW, &VNET_NAME(tcp_autorcvbuf_inc), 0, "Incrementor step size of automatic receive buffer"); VNET_DEFINE(int, tcp_autorcvbuf_max) = 256*1024; #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, recvbuf_max, CTLFLAG_RW, &VNET_NAME(tcp_autorcvbuf_max), 0, "Max size of automatic receive buffer"); int tcp_read_locking = 1; SYSCTL_INT(_net_inet_tcp, OID_AUTO, read_locking, CTLFLAG_RW, &tcp_read_locking, 0, "Enable read locking strategy"); VNET_DEFINE(struct inpcbhead, tcb); #define tcb6 tcb /* for KAME src sync over BSD*'s */ VNET_DEFINE(struct inpcbinfo, tcbinfo); static void tcp_dooptions(struct tcpopt *, u_char *, int, int); static void tcp_do_segment(struct mbuf *, struct tcphdr *, struct socket *, struct tcpcb *, int, int, uint8_t, int); static void tcp_dropwithreset(struct mbuf *, struct tcphdr *, struct tcpcb *, int, int); static void tcp_pulloutofband(struct socket *, struct tcphdr *, struct mbuf *, int); static void tcp_xmit_timer(struct tcpcb *, int); static void tcp_newreno_partial_ack(struct tcpcb *, struct tcphdr *); static void inline cc_ack_received(struct tcpcb *tp, struct tcphdr *th, uint16_t type); static void inline cc_conn_init(struct tcpcb *tp); static void inline cc_post_recovery(struct tcpcb *tp, struct tcphdr *th); static void inline hhook_run_tcp_est_in(struct tcpcb *tp, struct tcphdr *th, struct tcpopt *to); /* * Kernel module interface for updating tcpstat. The argument is an index * into tcpstat treated as an array of u_long. While this encodes the * general layout of tcpstat into the caller, it doesn't encode its location, * so that future changes to add, for example, per-CPU stats support won't * cause binary compatibility problems for kernel modules. */ void kmod_tcpstat_inc(int statnum) { (*((u_long *)&V_tcpstat + statnum))++; } /* * Wrapper for the TCP established input helper hook. */ static void inline hhook_run_tcp_est_in(struct tcpcb *tp, struct tcphdr *th, struct tcpopt *to) { struct tcp_hhook_data hhook_data; if (V_tcp_hhh[HHOOK_TCP_EST_IN]->hhh_nhooks > 0) { hhook_data.tp = tp; hhook_data.th = th; hhook_data.to = to; hhook_run_hooks(V_tcp_hhh[HHOOK_TCP_EST_IN], &hhook_data, tp->osd); } } /* * CC wrapper hook functions */ static void inline cc_ack_received(struct tcpcb *tp, struct tcphdr *th, uint16_t type) { INP_WLOCK_ASSERT(tp->t_inpcb); tp->ccv->bytes_this_ack = BYTES_THIS_ACK(tp, th); if (tp->snd_cwnd == min(tp->snd_cwnd, tp->snd_wnd)) tp->ccv->flags |= CCF_CWND_LIMITED; else tp->ccv->flags &= ~CCF_CWND_LIMITED; if (type == CC_ACK) { if (tp->snd_cwnd > tp->snd_ssthresh) { tp->t_bytes_acked += min(tp->ccv->bytes_this_ack, V_tcp_abc_l_var * tp->t_maxseg); if (tp->t_bytes_acked >= tp->snd_cwnd) { tp->t_bytes_acked -= tp->snd_cwnd; tp->ccv->flags |= CCF_ABC_SENTAWND; } } else { tp->ccv->flags &= ~CCF_ABC_SENTAWND; tp->t_bytes_acked = 0; } } if (CC_ALGO(tp)->ack_received != NULL) { /* XXXLAS: Find a way to live without this */ tp->ccv->curack = th->th_ack; CC_ALGO(tp)->ack_received(tp->ccv, type); } } static void inline cc_conn_init(struct tcpcb *tp) { struct hc_metrics_lite metrics; struct inpcb *inp = tp->t_inpcb; int rtt; #ifdef INET6 int isipv6 = ((inp->inp_vflag & INP_IPV6) != 0) ? 1 : 0; #endif INP_WLOCK_ASSERT(tp->t_inpcb); tcp_hc_get(&inp->inp_inc, &metrics); if (tp->t_srtt == 0 && (rtt = metrics.rmx_rtt)) { tp->t_srtt = rtt; tp->t_rttbest = tp->t_srtt + TCP_RTT_SCALE; TCPSTAT_INC(tcps_usedrtt); if (metrics.rmx_rttvar) { tp->t_rttvar = metrics.rmx_rttvar; TCPSTAT_INC(tcps_usedrttvar); } else { /* default variation is +- 1 rtt */ tp->t_rttvar = tp->t_srtt * TCP_RTTVAR_SCALE / TCP_RTT_SCALE; } TCPT_RANGESET(tp->t_rxtcur, ((tp->t_srtt >> 2) + tp->t_rttvar) >> 1, tp->t_rttmin, TCPTV_REXMTMAX); } if (metrics.rmx_ssthresh) { /* * There's some sort of gateway or interface * buffer limit on the path. Use this to set * the slow start threshhold, but set the * threshold to no less than 2*mss. */ tp->snd_ssthresh = max(2 * tp->t_maxseg, metrics.rmx_ssthresh); TCPSTAT_INC(tcps_usedssthresh); } /* * Set the slow-start flight size depending on whether this * is a local network or not. * * Extend this so we cache the cwnd too and retrieve it here. * Make cwnd even bigger than RFC3390 suggests but only if we * have previous experience with the remote host. Be careful * not make cwnd bigger than remote receive window or our own * send socket buffer. Maybe put some additional upper bound * on the retrieved cwnd. Should do incremental updates to * hostcache when cwnd collapses so next connection doesn't * overloads the path again. * * XXXAO: Initializing the CWND from the hostcache is broken * and in its current form not RFC conformant. It is disabled * until fixed or removed entirely. * * RFC3390 says only do this if SYN or SYN/ACK didn't got lost. * We currently check only in syncache_socket for that. */ /* #define TCP_METRICS_CWND */ #ifdef TCP_METRICS_CWND if (metrics.rmx_cwnd) tp->snd_cwnd = max(tp->t_maxseg, min(metrics.rmx_cwnd / 2, min(tp->snd_wnd, so->so_snd.sb_hiwat))); else #endif if (V_tcp_do_rfc3390) tp->snd_cwnd = min(4 * tp->t_maxseg, max(2 * tp->t_maxseg, 4380)); #ifdef INET6 else if ((isipv6 && in6_localaddr(&inp->in6p_faddr)) || (!isipv6 && in_localaddr(inp->inp_faddr))) #else else if (in_localaddr(inp->inp_faddr)) #endif tp->snd_cwnd = tp->t_maxseg * V_ss_fltsz_local; else tp->snd_cwnd = tp->t_maxseg * V_ss_fltsz; if (CC_ALGO(tp)->conn_init != NULL) CC_ALGO(tp)->conn_init(tp->ccv); } void inline cc_cong_signal(struct tcpcb *tp, struct tcphdr *th, uint32_t type) { INP_WLOCK_ASSERT(tp->t_inpcb); switch(type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(tp->t_flags)) { tp->snd_recover = tp->snd_max; if (tp->t_flags & TF_ECN_PERMIT) tp->t_flags |= TF_ECN_SND_CWR; } break; case CC_ECN: if (!IN_CONGRECOVERY(tp->t_flags)) { TCPSTAT_INC(tcps_ecn_rcwnd); tp->snd_recover = tp->snd_max; if (tp->t_flags & TF_ECN_PERMIT) tp->t_flags |= TF_ECN_SND_CWR; } break; case CC_RTO: tp->t_dupacks = 0; tp->t_bytes_acked = 0; EXIT_RECOVERY(tp->t_flags); tp->snd_ssthresh = max(2, min(tp->snd_wnd, tp->snd_cwnd) / 2 / tp->t_maxseg) * tp->t_maxseg; tp->snd_cwnd = tp->t_maxseg; break; case CC_RTO_ERR: TCPSTAT_INC(tcps_sndrexmitbad); /* RTO was unnecessary, so reset everything. */ tp->snd_cwnd = tp->snd_cwnd_prev; tp->snd_ssthresh = tp->snd_ssthresh_prev; tp->snd_recover = tp->snd_recover_prev; if (tp->t_flags & TF_WASFRECOVERY) ENTER_FASTRECOVERY(tp->t_flags); if (tp->t_flags & TF_WASCRECOVERY) ENTER_CONGRECOVERY(tp->t_flags); tp->snd_nxt = tp->snd_max; tp->t_badrxtwin = 0; break; } if (CC_ALGO(tp)->cong_signal != NULL) { if (th != NULL) tp->ccv->curack = th->th_ack; CC_ALGO(tp)->cong_signal(tp->ccv, type); } } static void inline cc_post_recovery(struct tcpcb *tp, struct tcphdr *th) { INP_WLOCK_ASSERT(tp->t_inpcb); /* XXXLAS: KASSERT that we're in recovery? */ if (CC_ALGO(tp)->post_recovery != NULL) { tp->ccv->curack = th->th_ack; CC_ALGO(tp)->post_recovery(tp->ccv); } /* XXXLAS: EXIT_RECOVERY ? */ tp->t_bytes_acked = 0; } /* Neighbor Discovery, Neighbor Unreachability Detection Upper layer hint. */ #ifdef INET6 #define ND6_HINT(tp) \ do { \ if ((tp) && (tp)->t_inpcb && \ ((tp)->t_inpcb->inp_vflag & INP_IPV6) != 0) \ nd6_nud_hint(NULL, NULL, 0); \ } while (0) #else #define ND6_HINT(tp) #endif /* * Indicate whether this ack should be delayed. We can delay the ack if * - there is no delayed ack timer in progress and * - our last ack wasn't a 0-sized window. We never want to delay * the ack that opens up a 0-sized window and * - delayed acks are enabled or * - this is a half-synchronized T/TCP connection. */ #define DELAY_ACK(tp) \ ((!tcp_timer_active(tp, TT_DELACK) && \ (tp->t_flags & TF_RXWIN0SENT) == 0) && \ (V_tcp_delack_enabled || (tp->t_flags & TF_NEEDSYN))) /* * TCP input handling is split into multiple parts: * tcp6_input is a thin wrapper around tcp_input for the extended * ip6_protox[] call format in ip6_input * tcp_input handles primary segment validation, inpcb lookup and * SYN processing on listen sockets * tcp_do_segment processes the ACK and text of the segment for * establishing, established and closing connections */ #ifdef INET6 int tcp6_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m = *mp; struct in6_ifaddr *ia6; IP6_EXTHDR_CHECK(m, *offp, sizeof(struct tcphdr), IPPROTO_DONE); /* * draft-itojun-ipv6-tcp-to-anycast * better place to put this in? */ ia6 = ip6_getdstifaddr(m); if (ia6 && (ia6->ia6_flags & IN6_IFF_ANYCAST)) { struct ip6_hdr *ip6; ifa_free(&ia6->ia_ifa); ip6 = mtod(m, struct ip6_hdr *); icmp6_error(m, ICMP6_DST_UNREACH, ICMP6_DST_UNREACH_ADDR, (caddr_t)&ip6->ip6_dst - (caddr_t)ip6); return IPPROTO_DONE; } tcp_input(m, *offp); return IPPROTO_DONE; } #endif void tcp_input(struct mbuf *m, int off0) { struct tcphdr *th; struct ip *ip = NULL; struct ipovly *ipov; struct inpcb *inp = NULL; struct tcpcb *tp = NULL; struct socket *so = NULL; u_char *optp = NULL; int optlen = 0; int len, tlen, off; int drop_hdrlen; int thflags; int rstreason = 0; /* For badport_bandlim accounting purposes */ uint8_t iptos; #ifdef IPFIREWALL_FORWARD struct m_tag *fwd_tag; #endif #ifdef INET6 struct ip6_hdr *ip6 = NULL; int isipv6; #else const void *ip6 = NULL; const int isipv6 = 0; #endif struct tcpopt to; /* options in this segment */ char *s = NULL; /* address and port logging */ int ti_locked; #define TI_UNLOCKED 1 #define TI_RLOCKED 2 #define TI_WLOCKED 3 #ifdef TCPDEBUG /* * The size of tcp_saveipgen must be the size of the max ip header, * now IPv6. */ u_char tcp_saveipgen[IP6_HDR_LEN]; struct tcphdr tcp_savetcp; short ostate = 0; #endif #ifdef INET6 isipv6 = (mtod(m, struct ip *)->ip_v == 6) ? 1 : 0; #endif to.to_flags = 0; TCPSTAT_INC(tcps_rcvtotal); if (isipv6) { #ifdef INET6 /* IP6_EXTHDR_CHECK() is already done at tcp6_input(). */ ip6 = mtod(m, struct ip6_hdr *); tlen = sizeof(*ip6) + ntohs(ip6->ip6_plen) - off0; if (in6_cksum(m, IPPROTO_TCP, off0, tlen)) { TCPSTAT_INC(tcps_rcvbadsum); goto drop; } th = (struct tcphdr *)((caddr_t)ip6 + off0); /* * Be proactive about unspecified IPv6 address in source. * As we use all-zero to indicate unbounded/unconnected pcb, * unspecified IPv6 address can be used to confuse us. * * Note that packets with unspecified IPv6 destination is * already dropped in ip6_input. */ if (IN6_IS_ADDR_UNSPECIFIED(&ip6->ip6_src)) { /* XXX stat */ goto drop; } #else th = NULL; /* XXX: Avoid compiler warning. */ #endif } else { /* * Get IP and TCP header together in first mbuf. * Note: IP leaves IP header in first mbuf. */ if (off0 > sizeof (struct ip)) { ip_stripoptions(m, (struct mbuf *)0); off0 = sizeof(struct ip); } if (m->m_len < sizeof (struct tcpiphdr)) { if ((m = m_pullup(m, sizeof (struct tcpiphdr))) == NULL) { TCPSTAT_INC(tcps_rcvshort); return; } } ip = mtod(m, struct ip *); ipov = (struct ipovly *)ip; th = (struct tcphdr *)((caddr_t)ip + off0); tlen = ip->ip_len; if (m->m_pkthdr.csum_flags & CSUM_DATA_VALID) { if (m->m_pkthdr.csum_flags & CSUM_PSEUDO_HDR) th->th_sum = m->m_pkthdr.csum_data; else th->th_sum = in_pseudo(ip->ip_src.s_addr, ip->ip_dst.s_addr, htonl(m->m_pkthdr.csum_data + ip->ip_len + IPPROTO_TCP)); th->th_sum ^= 0xffff; #ifdef TCPDEBUG ipov->ih_len = (u_short)tlen; ipov->ih_len = htons(ipov->ih_len); #endif } else { /* * Checksum extended TCP header and data. */ len = sizeof (struct ip) + tlen; bzero(ipov->ih_x1, sizeof(ipov->ih_x1)); ipov->ih_len = (u_short)tlen; ipov->ih_len = htons(ipov->ih_len); th->th_sum = in_cksum(m, len); } if (th->th_sum) { TCPSTAT_INC(tcps_rcvbadsum); goto drop; } /* Re-initialization for later version check */ ip->ip_v = IPVERSION; } #ifdef INET6 if (isipv6) iptos = (ntohl(ip6->ip6_flow) >> 20) & 0xff; else #endif iptos = ip->ip_tos; /* * Check that TCP offset makes sense, * pull out TCP options and adjust length. XXX */ off = th->th_off << 2; if (off < sizeof (struct tcphdr) || off > tlen) { TCPSTAT_INC(tcps_rcvbadoff); goto drop; } tlen -= off; /* tlen is used instead of ti->ti_len */ if (off > sizeof (struct tcphdr)) { if (isipv6) { #ifdef INET6 IP6_EXTHDR_CHECK(m, off0, off, ); ip6 = mtod(m, struct ip6_hdr *); th = (struct tcphdr *)((caddr_t)ip6 + off0); #endif } else { if (m->m_len < sizeof(struct ip) + off) { if ((m = m_pullup(m, sizeof (struct ip) + off)) == NULL) { TCPSTAT_INC(tcps_rcvshort); return; } ip = mtod(m, struct ip *); ipov = (struct ipovly *)ip; th = (struct tcphdr *)((caddr_t)ip + off0); } } optlen = off - sizeof (struct tcphdr); optp = (u_char *)(th + 1); } thflags = th->th_flags; /* * Convert TCP protocol specific fields to host format. */ th->th_seq = ntohl(th->th_seq); th->th_ack = ntohl(th->th_ack); th->th_win = ntohs(th->th_win); th->th_urp = ntohs(th->th_urp); /* * Delay dropping TCP, IP headers, IPv6 ext headers, and TCP options. */ drop_hdrlen = off0 + off; /* * Locate pcb for segment, which requires a lock on tcbinfo. * Optimisticaly acquire a global read lock rather than a write lock * unless header flags necessarily imply a state change. There are * two cases where we might discover later we need a write lock * despite the flags: ACKs moving a connection out of the syncache, * and ACKs for a connection in TIMEWAIT. */ if ((thflags & (TH_SYN | TH_FIN | TH_RST)) != 0 || tcp_read_locking == 0) { INP_INFO_WLOCK(&V_tcbinfo); ti_locked = TI_WLOCKED; } else { INP_INFO_RLOCK(&V_tcbinfo); ti_locked = TI_RLOCKED; } findpcb: #ifdef INVARIANTS if (ti_locked == TI_RLOCKED) INP_INFO_RLOCK_ASSERT(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WLOCK_ASSERT(&V_tcbinfo); else panic("%s: findpcb ti_locked %d\n", __func__, ti_locked); #endif #ifdef IPFIREWALL_FORWARD /* * Grab info from PACKET_TAG_IPFORWARD tag prepended to the chain. */ fwd_tag = m_tag_find(m, PACKET_TAG_IPFORWARD, NULL); if (fwd_tag != NULL && isipv6 == 0) { /* IPv6 support is not yet */ struct sockaddr_in *next_hop; next_hop = (struct sockaddr_in *)(fwd_tag+1); /* * Transparently forwarded. Pretend to be the destination. * already got one like this? */ inp = in_pcblookup_hash(&V_tcbinfo, ip->ip_src, th->th_sport, ip->ip_dst, th->th_dport, 0, m->m_pkthdr.rcvif); if (!inp) { /* It's new. Try to find the ambushing socket. */ inp = in_pcblookup_hash(&V_tcbinfo, ip->ip_src, th->th_sport, next_hop->sin_addr, next_hop->sin_port ? ntohs(next_hop->sin_port) : th->th_dport, INPLOOKUP_WILDCARD, m->m_pkthdr.rcvif); } /* Remove the tag from the packet. We don't need it anymore. */ m_tag_delete(m, fwd_tag); } else #endif /* IPFIREWALL_FORWARD */ { if (isipv6) { #ifdef INET6 inp = in6_pcblookup_hash(&V_tcbinfo, &ip6->ip6_src, th->th_sport, &ip6->ip6_dst, th->th_dport, INPLOOKUP_WILDCARD, m->m_pkthdr.rcvif); #endif } else inp = in_pcblookup_hash(&V_tcbinfo, ip->ip_src, th->th_sport, ip->ip_dst, th->th_dport, INPLOOKUP_WILDCARD, m->m_pkthdr.rcvif); } /* * If the INPCB does not exist then all data in the incoming * segment is discarded and an appropriate RST is sent back. * XXX MRT Send RST using which routing table? */ if (inp == NULL) { /* * Log communication attempts to ports that are not * in use. */ if ((tcp_log_in_vain == 1 && (thflags & TH_SYN)) || tcp_log_in_vain == 2) { if ((s = tcp_log_vain(NULL, th, (void *)ip, ip6))) log(LOG_INFO, "%s; %s: Connection attempt " "to closed port\n", s, __func__); } /* * When blackholing do not respond with a RST but * completely ignore the segment and drop it. */ if ((V_blackhole == 1 && (thflags & TH_SYN)) || V_blackhole == 2) goto dropunlock; rstreason = BANDLIM_RST_CLOSEDPORT; goto dropwithreset; } INP_WLOCK(inp); if (!(inp->inp_flags & INP_HW_FLOWID) && (m->m_flags & M_FLOWID) && ((inp->inp_socket == NULL) || !(inp->inp_socket->so_options & SO_ACCEPTCONN))) { inp->inp_flags |= INP_HW_FLOWID; inp->inp_flags &= ~INP_SW_FLOWID; inp->inp_flowid = m->m_pkthdr.flowid; } #ifdef IPSEC #ifdef INET6 if (isipv6 && ipsec6_in_reject(m, inp)) { V_ipsec6stat.in_polvio++; goto dropunlock; } else #endif /* INET6 */ if (ipsec4_in_reject(m, inp) != 0) { V_ipsec4stat.in_polvio++; goto dropunlock; } #endif /* IPSEC */ /* * Check the minimum TTL for socket. */ if (inp->inp_ip_minttl != 0) { #ifdef INET6 if (isipv6 && inp->inp_ip_minttl > ip6->ip6_hlim) goto dropunlock; else #endif if (inp->inp_ip_minttl > ip->ip_ttl) goto dropunlock; } /* * A previous connection in TIMEWAIT state is supposed to catch stray * or duplicate segments arriving late. If this segment was a * legitimate new connection attempt the old INPCB gets removed and * we can try again to find a listening socket. * * At this point, due to earlier optimism, we may hold a read lock on * the inpcbinfo, rather than a write lock. If so, we need to * upgrade, or if that fails, acquire a reference on the inpcb, drop * all locks, acquire a global write lock, and then re-acquire the * inpcb lock. We may at that point discover that another thread has * tried to free the inpcb, in which case we need to loop back and * try to find a new inpcb to deliver to. */ relocked: if (inp->inp_flags & INP_TIMEWAIT) { KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("%s: INP_TIMEWAIT ti_locked %d", __func__, ti_locked)); if (ti_locked == TI_RLOCKED) { if (INP_INFO_TRY_UPGRADE(&V_tcbinfo) == 0) { in_pcbref(inp); INP_WUNLOCK(inp); INP_INFO_RUNLOCK(&V_tcbinfo); INP_INFO_WLOCK(&V_tcbinfo); ti_locked = TI_WLOCKED; INP_WLOCK(inp); if (in_pcbrele(inp)) { inp = NULL; goto findpcb; } } else ti_locked = TI_WLOCKED; } INP_INFO_WLOCK_ASSERT(&V_tcbinfo); if (thflags & TH_SYN) tcp_dooptions(&to, optp, optlen, TO_SYN); /* * NB: tcp_twcheck unlocks the INP and frees the mbuf. */ if (tcp_twcheck(inp, &to, th, m, tlen)) goto findpcb; INP_INFO_WUNLOCK(&V_tcbinfo); return; } /* * The TCPCB may no longer exist if the connection is winding * down or it is in the CLOSED state. Either way we drop the * segment and send an appropriate response. */ tp = intotcpcb(inp); if (tp == NULL || tp->t_state == TCPS_CLOSED) { rstreason = BANDLIM_RST_CLOSEDPORT; goto dropwithreset; } /* * We've identified a valid inpcb, but it could be that we need an * inpcbinfo write lock and have only a read lock. In this case, * attempt to upgrade/relock using the same strategy as the TIMEWAIT * case above. If we relock, we have to jump back to 'relocked' as * the connection might now be in TIMEWAIT. */ if (tp->t_state != TCPS_ESTABLISHED || (thflags & (TH_SYN | TH_FIN | TH_RST)) != 0 || tcp_read_locking == 0) { KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("%s: upgrade check ti_locked %d", __func__, ti_locked)); if (ti_locked == TI_RLOCKED) { if (INP_INFO_TRY_UPGRADE(&V_tcbinfo) == 0) { in_pcbref(inp); INP_WUNLOCK(inp); INP_INFO_RUNLOCK(&V_tcbinfo); INP_INFO_WLOCK(&V_tcbinfo); ti_locked = TI_WLOCKED; INP_WLOCK(inp); if (in_pcbrele(inp)) { inp = NULL; goto findpcb; } goto relocked; } else ti_locked = TI_WLOCKED; } INP_INFO_WLOCK_ASSERT(&V_tcbinfo); } #ifdef MAC INP_WLOCK_ASSERT(inp); if (mac_inpcb_check_deliver(inp, m)) goto dropunlock; #endif so = inp->inp_socket; KASSERT(so != NULL, ("%s: so == NULL", __func__)); #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) { ostate = tp->t_state; if (isipv6) { #ifdef INET6 bcopy((char *)ip6, (char *)tcp_saveipgen, sizeof(*ip6)); #endif } else bcopy((char *)ip, (char *)tcp_saveipgen, sizeof(*ip)); tcp_savetcp = *th; } #endif /* * When the socket is accepting connections (the INPCB is in LISTEN * state) we look into the SYN cache if this is a new connection * attempt or the completion of a previous one. */ if (so->so_options & SO_ACCEPTCONN) { struct in_conninfo inc; KASSERT(tp->t_state == TCPS_LISTEN, ("%s: so accepting but " "tp not listening", __func__)); bzero(&inc, sizeof(inc)); #ifdef INET6 if (isipv6) { inc.inc_flags |= INC_ISIPV6; inc.inc6_faddr = ip6->ip6_src; inc.inc6_laddr = ip6->ip6_dst; } else #endif { inc.inc_faddr = ip->ip_src; inc.inc_laddr = ip->ip_dst; } inc.inc_fport = th->th_sport; inc.inc_lport = th->th_dport; inc.inc_fibnum = so->so_fibnum; /* * Check for an existing connection attempt in syncache if * the flag is only ACK. A successful lookup creates a new * socket appended to the listen queue in SYN_RECEIVED state. */ if ((thflags & (TH_RST|TH_ACK|TH_SYN)) == TH_ACK) { /* * Parse the TCP options here because * syncookies need access to the reflected * timestamp. */ tcp_dooptions(&to, optp, optlen, 0); /* * NB: syncache_expand() doesn't unlock * inp and tcpinfo locks. */ if (!syncache_expand(&inc, &to, th, &so, m)) { /* * No syncache entry or ACK was not * for our SYN/ACK. Send a RST. * NB: syncache did its own logging * of the failure cause. */ rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } if (so == NULL) { /* * We completed the 3-way handshake * but could not allocate a socket * either due to memory shortage, * listen queue length limits or * global socket limits. Send RST * or wait and have the remote end * retransmit the ACK for another * try. */ if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Socket allocation failed due to " "limits or memory shortage, %s\n", s, __func__, V_tcp_sc_rst_sock_fail ? "sending RST" : "try again"); if (V_tcp_sc_rst_sock_fail) { rstreason = BANDLIM_UNLIMITED; goto dropwithreset; } else goto dropunlock; } /* * Socket is created in state SYN_RECEIVED. * Unlock the listen socket, lock the newly * created socket and update the tp variable. */ INP_WUNLOCK(inp); /* listen socket */ inp = sotoinpcb(so); INP_WLOCK(inp); /* new connection */ tp = intotcpcb(inp); KASSERT(tp->t_state == TCPS_SYN_RECEIVED, ("%s: ", __func__)); /* * Process the segment and the data it * contains. tcp_do_segment() consumes * the mbuf chain and unlocks the inpcb. */ tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen, iptos, ti_locked); INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); return; } /* * Segment flag validation for new connection attempts: * * Our (SYN|ACK) response was rejected. * Check with syncache and remove entry to prevent * retransmits. * * NB: syncache_chkrst does its own logging of failure * causes. */ if (thflags & TH_RST) { syncache_chkrst(&inc, th); goto dropunlock; } /* * We can't do anything without SYN. */ if ((thflags & TH_SYN) == 0) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "SYN is missing, segment ignored\n", s, __func__); TCPSTAT_INC(tcps_badsyn); goto dropunlock; } /* * (SYN|ACK) is bogus on a listen socket. */ if (thflags & TH_ACK) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "SYN|ACK invalid, segment rejected\n", s, __func__); syncache_badack(&inc); /* XXX: Not needed! */ TCPSTAT_INC(tcps_badsyn); rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } /* * If the drop_synfin option is enabled, drop all * segments with both the SYN and FIN bits set. * This prevents e.g. nmap from identifying the * TCP/IP stack. * XXX: Poor reasoning. nmap has other methods * and is constantly refining its stack detection * strategies. * XXX: This is a violation of the TCP specification * and was used by RFC1644. */ if ((thflags & TH_FIN) && V_drop_synfin) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "SYN|FIN segment ignored (based on " "sysctl setting)\n", s, __func__); TCPSTAT_INC(tcps_badsyn); goto dropunlock; } /* * Segment's flags are (SYN) or (SYN|FIN). * * TH_PUSH, TH_URG, TH_ECE, TH_CWR are ignored * as they do not affect the state of the TCP FSM. * The data pointed to by TH_URG and th_urp is ignored. */ KASSERT((thflags & (TH_RST|TH_ACK)) == 0, ("%s: Listen socket: TH_RST or TH_ACK set", __func__)); KASSERT(thflags & (TH_SYN), ("%s: Listen socket: TH_SYN not set", __func__)); #ifdef INET6 /* * If deprecated address is forbidden, * we do not accept SYN to deprecated interface * address to prevent any new inbound connection from * getting established. * When we do not accept SYN, we send a TCP RST, * with deprecated source address (instead of dropping * it). We compromise it as it is much better for peer * to send a RST, and RST will be the final packet * for the exchange. * * If we do not forbid deprecated addresses, we accept * the SYN packet. RFC2462 does not suggest dropping * SYN in this case. * If we decipher RFC2462 5.5.4, it says like this: * 1. use of deprecated addr with existing * communication is okay - "SHOULD continue to be * used" * 2. use of it with new communication: * (2a) "SHOULD NOT be used if alternate address * with sufficient scope is available" * (2b) nothing mentioned otherwise. * Here we fall into (2b) case as we have no choice in * our source address selection - we must obey the peer. * * The wording in RFC2462 is confusing, and there are * multiple description text for deprecated address * handling - worse, they are not exactly the same. * I believe 5.5.4 is the best one, so we follow 5.5.4. */ if (isipv6 && !V_ip6_use_deprecated) { struct in6_ifaddr *ia6; ia6 = ip6_getdstifaddr(m); if (ia6 != NULL && (ia6->ia6_flags & IN6_IFF_DEPRECATED)) { ifa_free(&ia6->ia_ifa); if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt to deprecated " "IPv6 address rejected\n", s, __func__); rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } ifa_free(&ia6->ia_ifa); } #endif /* * Basic sanity checks on incoming SYN requests: * Don't respond if the destination is a link layer * broadcast according to RFC1122 4.2.3.10, p. 104. * If it is from this socket it must be forged. * Don't respond if the source or destination is a * global or subnet broad- or multicast address. * Note that it is quite possible to receive unicast * link-layer packets with a broadcast IP address. Use * in_broadcast() to find them. */ if (m->m_flags & (M_BCAST|M_MCAST)) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt from broad- or multicast " "link layer address ignored\n", s, __func__); goto dropunlock; } if (isipv6) { #ifdef INET6 if (th->th_dport == th->th_sport && IN6_ARE_ADDR_EQUAL(&ip6->ip6_dst, &ip6->ip6_src)) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt to/from self " "ignored\n", s, __func__); goto dropunlock; } if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst) || IN6_IS_ADDR_MULTICAST(&ip6->ip6_src)) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt from/to multicast " "address ignored\n", s, __func__); goto dropunlock; } #endif } else { if (th->th_dport == th->th_sport && ip->ip_dst.s_addr == ip->ip_src.s_addr) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt from/to self " "ignored\n", s, __func__); goto dropunlock; } if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) || IN_MULTICAST(ntohl(ip->ip_src.s_addr)) || ip->ip_src.s_addr == htonl(INADDR_BROADCAST) || in_broadcast(ip->ip_dst, m->m_pkthdr.rcvif)) { if ((s = tcp_log_addrs(&inc, th, NULL, NULL))) log(LOG_DEBUG, "%s; %s: Listen socket: " "Connection attempt from/to broad- " "or multicast address ignored\n", s, __func__); goto dropunlock; } } /* * SYN appears to be valid. Create compressed TCP state * for syncache. */ #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) tcp_trace(TA_INPUT, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif tcp_dooptions(&to, optp, optlen, TO_SYN); syncache_add(&inc, &to, th, inp, &so, m); /* * Entry added to syncache and mbuf consumed. * Everything already unlocked by syncache_add(). */ INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); return; } /* * Segment belongs to a connection in SYN_SENT, ESTABLISHED or later * state. tcp_do_segment() always consumes the mbuf chain, unlocks * the inpcb, and unlocks pcbinfo. */ tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen, iptos, ti_locked); INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); return; dropwithreset: if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: dropwithreset ti_locked %d", __func__, ti_locked); ti_locked = TI_UNLOCKED; if (inp != NULL) { tcp_dropwithreset(m, th, tp, tlen, rstreason); INP_WUNLOCK(inp); } else tcp_dropwithreset(m, th, NULL, tlen, rstreason); m = NULL; /* mbuf chain got consumed. */ goto drop; dropunlock: if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: dropunlock ti_locked %d", __func__, ti_locked); ti_locked = TI_UNLOCKED; if (inp != NULL) INP_WUNLOCK(inp); drop: INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); if (s != NULL) free(s, M_TCPLOG); if (m != NULL) m_freem(m); } static void tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, struct tcpcb *tp, int drop_hdrlen, int tlen, uint8_t iptos, int ti_locked) { int thflags, acked, ourfinisacked, needoutput = 0; int rstreason, todrop, win; u_long tiwin; struct tcpopt to; #ifdef TCPDEBUG /* * The size of tcp_saveipgen must be the size of the max ip header, * now IPv6. */ u_char tcp_saveipgen[IP6_HDR_LEN]; struct tcphdr tcp_savetcp; short ostate = 0; #endif thflags = th->th_flags; tp->sackhint.last_sack_ack = 0; /* * If this is either a state-changing packet or current state isn't * established, we require a write lock on tcbinfo. Otherwise, we * allow either a read lock or a write lock, as we may have acquired * a write lock due to a race. * * Require a global write lock for SYN/FIN/RST segments or * non-established connections; otherwise accept either a read or * write lock, as we may have conservatively acquired a write lock in * certain cases in tcp_input() (is this still true?). Currently we * will never enter with no lock, so we try to drop it quickly in the * common pure ack/pure data cases. */ if ((thflags & (TH_SYN | TH_FIN | TH_RST)) != 0 || tp->t_state != TCPS_ESTABLISHED) { KASSERT(ti_locked == TI_WLOCKED, ("%s ti_locked %d for " "SYN/FIN/RST/!EST", __func__, ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); } else { #ifdef INVARIANTS if (ti_locked == TI_RLOCKED) INP_INFO_RLOCK_ASSERT(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WLOCK_ASSERT(&V_tcbinfo); else panic("%s: ti_locked %d for EST", __func__, ti_locked); #endif } INP_WLOCK_ASSERT(tp->t_inpcb); KASSERT(tp->t_state > TCPS_LISTEN, ("%s: TCPS_LISTEN", __func__)); KASSERT(tp->t_state != TCPS_TIME_WAIT, ("%s: TCPS_TIME_WAIT", __func__)); /* * Segment received on connection. * Reset idle time and keep-alive timer. * XXX: This should be done after segment * validation to ignore broken/spoofed segs. */ tp->t_rcvtime = ticks; if (TCPS_HAVEESTABLISHED(tp->t_state)) tcp_timer_activate(tp, TT_KEEP, tcp_keepidle); /* * Unscale the window into a 32-bit value. * For the SYN_SENT state the scale is zero. */ tiwin = th->th_win << tp->snd_scale; /* * TCP ECN processing. */ if (tp->t_flags & TF_ECN_PERMIT) { if (thflags & TH_CWR) tp->t_flags &= ~TF_ECN_SND_ECE; switch (iptos & IPTOS_ECN_MASK) { case IPTOS_ECN_CE: tp->t_flags |= TF_ECN_SND_ECE; TCPSTAT_INC(tcps_ecn_ce); break; case IPTOS_ECN_ECT0: TCPSTAT_INC(tcps_ecn_ect0); break; case IPTOS_ECN_ECT1: TCPSTAT_INC(tcps_ecn_ect1); break; } /* Congestion experienced. */ if (thflags & TH_ECE) { cc_cong_signal(tp, th, CC_ECN); } } /* * Parse options on any incoming segment. */ tcp_dooptions(&to, (u_char *)(th + 1), (th->th_off << 2) - sizeof(struct tcphdr), (thflags & TH_SYN) ? TO_SYN : 0); /* * If echoed timestamp is later than the current time, * fall back to non RFC1323 RTT calculation. Normalize * timestamp if syncookies were used when this connection * was established. */ if ((to.to_flags & TOF_TS) && (to.to_tsecr != 0)) { to.to_tsecr -= tp->ts_offset; if (TSTMP_GT(to.to_tsecr, ticks)) to.to_tsecr = 0; } /* * Process options only when we get SYN/ACK back. The SYN case * for incoming connections is handled in tcp_syncache. * According to RFC1323 the window field in a SYN (i.e., a * or ) segment itself is never scaled. * XXX this is traditional behavior, may need to be cleaned up. */ if (tp->t_state == TCPS_SYN_SENT && (thflags & TH_SYN)) { if ((to.to_flags & TOF_SCALE) && (tp->t_flags & TF_REQ_SCALE)) { tp->t_flags |= TF_RCVD_SCALE; tp->snd_scale = to.to_wscale; } /* * Initial send window. It will be updated with * the next incoming segment to the scaled value. */ tp->snd_wnd = th->th_win; if (to.to_flags & TOF_TS) { tp->t_flags |= TF_RCVD_TSTMP; tp->ts_recent = to.to_tsval; tp->ts_recent_age = ticks; } if (to.to_flags & TOF_MSS) tcp_mss(tp, to.to_mss); if ((tp->t_flags & TF_SACK_PERMIT) && (to.to_flags & TOF_SACKPERM) == 0) tp->t_flags &= ~TF_SACK_PERMIT; } /* * Header prediction: check for the two common cases * of a uni-directional data xfer. If the packet has * no control flags, is in-sequence, the window didn't * change and we're not retransmitting, it's a * candidate. If the length is zero and the ack moved * forward, we're the sender side of the xfer. Just * free the data acked & wake any higher level process * that was blocked waiting for space. If the length * is non-zero and the ack didn't move, we're the * receiver side. If we're getting packets in-order * (the reassembly queue is empty), add the data to * the socket buffer and note that we need a delayed ack. * Make sure that the hidden state-flags are also off. * Since we check for TCPS_ESTABLISHED first, it can only * be TH_NEEDSYN. */ if (tp->t_state == TCPS_ESTABLISHED && th->th_seq == tp->rcv_nxt && (thflags & (TH_SYN|TH_FIN|TH_RST|TH_URG|TH_ACK)) == TH_ACK && tp->snd_nxt == tp->snd_max && tiwin && tiwin == tp->snd_wnd && ((tp->t_flags & (TF_NEEDSYN|TF_NEEDFIN)) == 0) && LIST_EMPTY(&tp->t_segq) && ((to.to_flags & TOF_TS) == 0 || TSTMP_GEQ(to.to_tsval, tp->ts_recent)) ) { /* * If last ACK falls within this segment's sequence numbers, * record the timestamp. * NOTE that the test is modified according to the latest * proposal of the tcplw@cray.com list (Braden 1993/04/26). */ if ((to.to_flags & TOF_TS) != 0 && SEQ_LEQ(th->th_seq, tp->last_ack_sent)) { tp->ts_recent_age = ticks; tp->ts_recent = to.to_tsval; } if (tlen == 0) { if (SEQ_GT(th->th_ack, tp->snd_una) && SEQ_LEQ(th->th_ack, tp->snd_max) && !IN_RECOVERY(tp->t_flags) && (to.to_flags & TOF_SACK) == 0 && TAILQ_EMPTY(&tp->snd_holes)) { /* * This is a pure ack for outstanding data. */ if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: ti_locked %d on pure ACK", __func__, ti_locked); ti_locked = TI_UNLOCKED; TCPSTAT_INC(tcps_predack); /* * "bad retransmit" recovery. */ if (tp->t_rxtshift == 1 && (int)(ticks - tp->t_badrxtwin) < 0) { cc_cong_signal(tp, th, CC_RTO_ERR); } /* * Recalculate the transmit timer / rtt. * * Some boxes send broken timestamp replies * during the SYN+ACK phase, ignore * timestamps of 0 or we could calculate a * huge RTT and blow up the retransmit timer. */ if ((to.to_flags & TOF_TS) != 0 && to.to_tsecr) { if (!tp->t_rttlow || tp->t_rttlow > ticks - to.to_tsecr) tp->t_rttlow = ticks - to.to_tsecr; tcp_xmit_timer(tp, ticks - to.to_tsecr + 1); } else if (tp->t_rtttime && SEQ_GT(th->th_ack, tp->t_rtseq)) { if (!tp->t_rttlow || tp->t_rttlow > ticks - tp->t_rtttime) tp->t_rttlow = ticks - tp->t_rtttime; tcp_xmit_timer(tp, ticks - tp->t_rtttime); } acked = BYTES_THIS_ACK(tp, th); /* Run HHOOK_TCP_ESTABLISHED_IN helper hooks. */ hhook_run_tcp_est_in(tp, th, &to); TCPSTAT_INC(tcps_rcvackpack); TCPSTAT_ADD(tcps_rcvackbyte, acked); sbdrop(&so->so_snd, acked); if (SEQ_GT(tp->snd_una, tp->snd_recover) && SEQ_LEQ(th->th_ack, tp->snd_recover)) tp->snd_recover = th->th_ack - 1; /* * Let the congestion control algorithm update * congestion control related information. This * typically means increasing the congestion * window. */ cc_ack_received(tp, th, CC_ACK); tp->snd_una = th->th_ack; /* * Pull snd_wl2 up to prevent seq wrap relative * to th_ack. */ tp->snd_wl2 = th->th_ack; tp->t_dupacks = 0; m_freem(m); ND6_HINT(tp); /* Some progress has been made. */ /* * If all outstanding data are acked, stop * retransmit timer, otherwise restart timer * using current (possibly backed-off) value. * If process is waiting for space, * wakeup/selwakeup/signal. If data * are ready to send, let tcp_output * decide between more output or persist. */ #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) tcp_trace(TA_INPUT, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif if (tp->snd_una == tp->snd_max) tcp_timer_activate(tp, TT_REXMT, 0); else if (!tcp_timer_active(tp, TT_PERSIST)) tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur); sowwakeup(so); if (so->so_snd.sb_cc) (void) tcp_output(tp); goto check_delack; } } else if (th->th_ack == tp->snd_una && tlen <= sbspace(&so->so_rcv)) { int newsize = 0; /* automatic sockbuf scaling */ /* * This is a pure, in-sequence data packet with * nothing on the reassembly queue and we have enough * buffer space to take it. */ if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: ti_locked %d on pure data " "segment", __func__, ti_locked); ti_locked = TI_UNLOCKED; /* Clean receiver SACK report if present */ if ((tp->t_flags & TF_SACK_PERMIT) && tp->rcv_numsacks) tcp_clean_sackreport(tp); TCPSTAT_INC(tcps_preddat); tp->rcv_nxt += tlen; /* * Pull snd_wl1 up to prevent seq wrap relative to * th_seq. */ tp->snd_wl1 = th->th_seq; /* * Pull rcv_up up to prevent seq wrap relative to * rcv_nxt. */ tp->rcv_up = tp->rcv_nxt; TCPSTAT_INC(tcps_rcvpack); TCPSTAT_ADD(tcps_rcvbyte, tlen); ND6_HINT(tp); /* Some progress has been made */ #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) tcp_trace(TA_INPUT, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif /* * Automatic sizing of receive socket buffer. Often the send * buffer size is not optimally adjusted to the actual network * conditions at hand (delay bandwidth product). Setting the * buffer size too small limits throughput on links with high * bandwidth and high delay (eg. trans-continental/oceanic links). * * On the receive side the socket buffer memory is only rarely * used to any significant extent. This allows us to be much * more aggressive in scaling the receive socket buffer. For * the case that the buffer space is actually used to a large * extent and we run out of kernel memory we can simply drop * the new segments; TCP on the sender will just retransmit it * later. Setting the buffer size too big may only consume too * much kernel memory if the application doesn't read() from * the socket or packet loss or reordering makes use of the * reassembly queue. * * The criteria to step up the receive buffer one notch are: * 1. the number of bytes received during the time it takes * one timestamp to be reflected back to us (the RTT); * 2. received bytes per RTT is within seven eighth of the * current socket buffer size; * 3. receive buffer size has not hit maximal automatic size; * * This algorithm does one step per RTT at most and only if * we receive a bulk stream w/o packet losses or reorderings. * Shrinking the buffer during idle times is not necessary as * it doesn't consume any memory when idle. * * TODO: Only step up if the application is actually serving * the buffer to better manage the socket buffer resources. */ if (V_tcp_do_autorcvbuf && to.to_tsecr && (so->so_rcv.sb_flags & SB_AUTOSIZE)) { if (TSTMP_GT(to.to_tsecr, tp->rfbuf_ts) && to.to_tsecr - tp->rfbuf_ts < hz) { if (tp->rfbuf_cnt > (so->so_rcv.sb_hiwat / 8 * 7) && so->so_rcv.sb_hiwat < V_tcp_autorcvbuf_max) { newsize = min(so->so_rcv.sb_hiwat + V_tcp_autorcvbuf_inc, V_tcp_autorcvbuf_max); } /* Start over with next RTT. */ tp->rfbuf_ts = 0; tp->rfbuf_cnt = 0; } else tp->rfbuf_cnt += tlen; /* add up */ } /* Add data to socket buffer. */ SOCKBUF_LOCK(&so->so_rcv); if (so->so_rcv.sb_state & SBS_CANTRCVMORE) { m_freem(m); } else { /* * Set new socket buffer size. * Give up when limit is reached. */ if (newsize) if (!sbreserve_locked(&so->so_rcv, newsize, so, NULL)) so->so_rcv.sb_flags &= ~SB_AUTOSIZE; m_adj(m, drop_hdrlen); /* delayed header drop */ sbappendstream_locked(&so->so_rcv, m); } /* NB: sorwakeup_locked() does an implicit unlock. */ sorwakeup_locked(so); if (DELAY_ACK(tp)) { tp->t_flags |= TF_DELACK; } else { tp->t_flags |= TF_ACKNOW; tcp_output(tp); } goto check_delack; } } /* * Calculate amount of space in receive window, * and then do TCP input processing. * Receive window is amount of space in rcv queue, * but not less than advertised window. */ win = sbspace(&so->so_rcv); if (win < 0) win = 0; tp->rcv_wnd = imax(win, (int)(tp->rcv_adv - tp->rcv_nxt)); /* Reset receive buffer auto scaling when not in bulk receive mode. */ tp->rfbuf_ts = 0; tp->rfbuf_cnt = 0; switch (tp->t_state) { /* * If the state is SYN_RECEIVED: * if seg contains an ACK, but not for our SYN/ACK, send a RST. */ case TCPS_SYN_RECEIVED: if ((thflags & TH_ACK) && (SEQ_LEQ(th->th_ack, tp->snd_una) || SEQ_GT(th->th_ack, tp->snd_max))) { rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } break; /* * If the state is SYN_SENT: * if seg contains an ACK, but not for our SYN, drop the input. * if seg contains a RST, then drop the connection. * if seg does not contain SYN, then drop it. * Otherwise this is an acceptable SYN segment * initialize tp->rcv_nxt and tp->irs * if seg contains ack then advance tp->snd_una * if seg contains an ECE and ECN support is enabled, the stream * is ECN capable. * if SYN has been acked change to ESTABLISHED else SYN_RCVD state * arrange for segment to be acked (eventually) * continue processing rest of data/controls, beginning with URG */ case TCPS_SYN_SENT: if ((thflags & TH_ACK) && (SEQ_LEQ(th->th_ack, tp->iss) || SEQ_GT(th->th_ack, tp->snd_max))) { rstreason = BANDLIM_UNLIMITED; goto dropwithreset; } if ((thflags & (TH_ACK|TH_RST)) == (TH_ACK|TH_RST)) tp = tcp_drop(tp, ECONNREFUSED); if (thflags & TH_RST) goto drop; if (!(thflags & TH_SYN)) goto drop; tp->irs = th->th_seq; tcp_rcvseqinit(tp); if (thflags & TH_ACK) { TCPSTAT_INC(tcps_connects); soisconnected(so); #ifdef MAC mac_socketpeer_set_from_mbuf(m, so); #endif /* Do window scaling on this connection? */ if ((tp->t_flags & (TF_RCVD_SCALE|TF_REQ_SCALE)) == (TF_RCVD_SCALE|TF_REQ_SCALE)) { tp->rcv_scale = tp->request_r_scale; } tp->rcv_adv += imin(tp->rcv_wnd, TCP_MAXWIN << tp->rcv_scale); tp->snd_una++; /* SYN is acked */ /* * If there's data, delay ACK; if there's also a FIN * ACKNOW will be turned on later. */ if (DELAY_ACK(tp) && tlen != 0) tcp_timer_activate(tp, TT_DELACK, tcp_delacktime); else tp->t_flags |= TF_ACKNOW; if ((thflags & TH_ECE) && V_tcp_do_ecn) { tp->t_flags |= TF_ECN_PERMIT; TCPSTAT_INC(tcps_ecn_shs); } /* * Received in SYN_SENT[*] state. * Transitions: * SYN_SENT --> ESTABLISHED * SYN_SENT* --> FIN_WAIT_1 */ tp->t_starttime = ticks; if (tp->t_flags & TF_NEEDFIN) { tp->t_state = TCPS_FIN_WAIT_1; tp->t_flags &= ~TF_NEEDFIN; thflags &= ~TH_SYN; } else { tp->t_state = TCPS_ESTABLISHED; cc_conn_init(tp); tcp_timer_activate(tp, TT_KEEP, tcp_keepidle); } } else { /* * Received initial SYN in SYN-SENT[*] state => * simultaneous open. If segment contains CC option * and there is a cached CC, apply TAO test. * If it succeeds, connection is * half-synchronized. * Otherwise, do 3-way handshake: * SYN-SENT -> SYN-RECEIVED * SYN-SENT* -> SYN-RECEIVED* * If there was no CC option, clear cached CC value. */ tp->t_flags |= (TF_ACKNOW | TF_NEEDSYN); tcp_timer_activate(tp, TT_REXMT, 0); tp->t_state = TCPS_SYN_RECEIVED; } KASSERT(ti_locked == TI_WLOCKED, ("%s: trimthenstep6: " "ti_locked %d", __func__, ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tp->t_inpcb); /* * Advance th->th_seq to correspond to first data byte. * If data, trim to stay within window, * dropping FIN if necessary. */ th->th_seq++; if (tlen > tp->rcv_wnd) { todrop = tlen - tp->rcv_wnd; m_adj(m, -todrop); tlen = tp->rcv_wnd; thflags &= ~TH_FIN; TCPSTAT_INC(tcps_rcvpackafterwin); TCPSTAT_ADD(tcps_rcvbyteafterwin, todrop); } tp->snd_wl1 = th->th_seq - 1; tp->rcv_up = th->th_seq; /* * Client side of transaction: already sent SYN and data. * If the remote host used T/TCP to validate the SYN, * our data will be ACK'd; if so, enter normal data segment * processing in the middle of step 5, ack processing. * Otherwise, goto step 6. */ if (thflags & TH_ACK) goto process_ACK; goto step6; /* * If the state is LAST_ACK or CLOSING or TIME_WAIT: * do normal processing. * * NB: Leftover from RFC1644 T/TCP. Cases to be reused later. */ case TCPS_LAST_ACK: case TCPS_CLOSING: break; /* continue normal processing */ } /* * States other than LISTEN or SYN_SENT. * First check the RST flag and sequence number since reset segments * are exempt from the timestamp and connection count tests. This * fixes a bug introduced by the Stevens, vol. 2, p. 960 bugfix * below which allowed reset segments in half the sequence space * to fall though and be processed (which gives forged reset * segments with a random sequence number a 50 percent chance of * killing a connection). * Then check timestamp, if present. * Then check the connection count, if present. * Then check that at least some bytes of segment are within * receive window. If segment begins before rcv_nxt, * drop leading data (and SYN); if nothing left, just ack. * * * If the RST bit is set, check the sequence number to see * if this is a valid reset segment. * RFC 793 page 37: * In all states except SYN-SENT, all reset (RST) segments * are validated by checking their SEQ-fields. A reset is * valid if its sequence number is in the window. * Note: this does not take into account delayed ACKs, so * we should test against last_ack_sent instead of rcv_nxt. * The sequence number in the reset segment is normally an * echo of our outgoing acknowlegement numbers, but some hosts * send a reset with the sequence number at the rightmost edge * of our receive window, and we have to handle this case. * Note 2: Paul Watson's paper "Slipping in the Window" has shown * that brute force RST attacks are possible. To combat this, * we use a much stricter check while in the ESTABLISHED state, * only accepting RSTs where the sequence number is equal to * last_ack_sent. In all other states (the states in which a * RST is more likely), the more permissive check is used. * If we have multiple segments in flight, the initial reset * segment sequence numbers will be to the left of last_ack_sent, * but they will eventually catch up. * In any case, it never made sense to trim reset segments to * fit the receive window since RFC 1122 says: * 4.2.2.12 RST Segment: RFC-793 Section 3.4 * * A TCP SHOULD allow a received RST segment to include data. * * DISCUSSION * It has been suggested that a RST segment could contain * ASCII text that encoded and explained the cause of the * RST. No standard has yet been established for such * data. * * If the reset segment passes the sequence number test examine * the state: * SYN_RECEIVED STATE: * If passive open, return to LISTEN state. * If active open, inform user that connection was refused. * ESTABLISHED, FIN_WAIT_1, FIN_WAIT_2, CLOSE_WAIT STATES: * Inform user that connection was reset, and close tcb. * CLOSING, LAST_ACK STATES: * Close the tcb. * TIME_WAIT STATE: * Drop the segment - see Stevens, vol. 2, p. 964 and * RFC 1337. */ if (thflags & TH_RST) { if (SEQ_GEQ(th->th_seq, tp->last_ack_sent - 1) && SEQ_LEQ(th->th_seq, tp->last_ack_sent + tp->rcv_wnd)) { switch (tp->t_state) { case TCPS_SYN_RECEIVED: so->so_error = ECONNREFUSED; goto close; case TCPS_ESTABLISHED: if (V_tcp_insecure_rst == 0 && !(SEQ_GEQ(th->th_seq, tp->rcv_nxt - 1) && SEQ_LEQ(th->th_seq, tp->rcv_nxt + 1)) && !(SEQ_GEQ(th->th_seq, tp->last_ack_sent - 1) && SEQ_LEQ(th->th_seq, tp->last_ack_sent + 1))) { TCPSTAT_INC(tcps_badrst); goto drop; } /* FALLTHROUGH */ case TCPS_FIN_WAIT_1: case TCPS_FIN_WAIT_2: case TCPS_CLOSE_WAIT: so->so_error = ECONNRESET; close: KASSERT(ti_locked == TI_WLOCKED, ("tcp_do_segment: TH_RST 1 ti_locked %d", ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); tp->t_state = TCPS_CLOSED; TCPSTAT_INC(tcps_drops); tp = tcp_close(tp); break; case TCPS_CLOSING: case TCPS_LAST_ACK: KASSERT(ti_locked == TI_WLOCKED, ("tcp_do_segment: TH_RST 2 ti_locked %d", ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); tp = tcp_close(tp); break; } } goto drop; } /* * RFC 1323 PAWS: If we have a timestamp reply on this segment * and it's less than ts_recent, drop it. */ if ((to.to_flags & TOF_TS) != 0 && tp->ts_recent && TSTMP_LT(to.to_tsval, tp->ts_recent)) { /* Check to see if ts_recent is over 24 days old. */ if (ticks - tp->ts_recent_age > TCP_PAWS_IDLE) { /* * Invalidate ts_recent. If this segment updates * ts_recent, the age will be reset later and ts_recent * will get a valid value. If it does not, setting * ts_recent to zero will at least satisfy the * requirement that zero be placed in the timestamp * echo reply when ts_recent isn't valid. The * age isn't reset until we get a valid ts_recent * because we don't want out-of-order segments to be * dropped when ts_recent is old. */ tp->ts_recent = 0; } else { TCPSTAT_INC(tcps_rcvduppack); TCPSTAT_ADD(tcps_rcvdupbyte, tlen); TCPSTAT_INC(tcps_pawsdrop); if (tlen) goto dropafterack; goto drop; } } /* * In the SYN-RECEIVED state, validate that the packet belongs to * this connection before trimming the data to fit the receive * window. Check the sequence number versus IRS since we know * the sequence numbers haven't wrapped. This is a partial fix * for the "LAND" DoS attack. */ if (tp->t_state == TCPS_SYN_RECEIVED && SEQ_LT(th->th_seq, tp->irs)) { rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } todrop = tp->rcv_nxt - th->th_seq; if (todrop > 0) { /* * If this is a duplicate SYN for our current connection, * advance over it and pretend and it's not a SYN. */ if (thflags & TH_SYN && th->th_seq == tp->irs) { thflags &= ~TH_SYN; th->th_seq++; if (th->th_urp > 1) th->th_urp--; else thflags &= ~TH_URG; todrop--; } /* * Following if statement from Stevens, vol. 2, p. 960. */ if (todrop > tlen || (todrop == tlen && (thflags & TH_FIN) == 0)) { /* * Any valid FIN must be to the left of the window. * At this point the FIN must be a duplicate or out * of sequence; drop it. */ thflags &= ~TH_FIN; /* * Send an ACK to resynchronize and drop any data. * But keep on processing for RST or ACK. */ tp->t_flags |= TF_ACKNOW; todrop = tlen; TCPSTAT_INC(tcps_rcvduppack); TCPSTAT_ADD(tcps_rcvdupbyte, todrop); } else { TCPSTAT_INC(tcps_rcvpartduppack); TCPSTAT_ADD(tcps_rcvpartdupbyte, todrop); } drop_hdrlen += todrop; /* drop from the top afterwards */ th->th_seq += todrop; tlen -= todrop; if (th->th_urp > todrop) th->th_urp -= todrop; else { thflags &= ~TH_URG; th->th_urp = 0; } } /* * If new data are received on a connection after the * user processes are gone, then RST the other end. */ if ((so->so_state & SS_NOFDREF) && tp->t_state > TCPS_CLOSE_WAIT && tlen) { char *s; KASSERT(ti_locked == TI_WLOCKED, ("%s: SS_NOFDEREF && " "CLOSE_WAIT && tlen ti_locked %d", __func__, ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); if ((s = tcp_log_addrs(&tp->t_inpcb->inp_inc, th, NULL, NULL))) { log(LOG_DEBUG, "%s; %s: %s: Received %d bytes of data after socket " "was closed, sending RST and removing tcpcb\n", s, __func__, tcpstates[tp->t_state], tlen); free(s, M_TCPLOG); } tp = tcp_close(tp); TCPSTAT_INC(tcps_rcvafterclose); rstreason = BANDLIM_UNLIMITED; goto dropwithreset; } /* * If segment ends after window, drop trailing data * (and PUSH and FIN); if nothing left, just ACK. */ todrop = (th->th_seq + tlen) - (tp->rcv_nxt + tp->rcv_wnd); if (todrop > 0) { TCPSTAT_INC(tcps_rcvpackafterwin); if (todrop >= tlen) { TCPSTAT_ADD(tcps_rcvbyteafterwin, tlen); /* * If window is closed can only take segments at * window edge, and have to drop data and PUSH from * incoming segments. Continue processing, but * remember to ack. Otherwise, drop segment * and ack. */ if (tp->rcv_wnd == 0 && th->th_seq == tp->rcv_nxt) { tp->t_flags |= TF_ACKNOW; TCPSTAT_INC(tcps_rcvwinprobe); } else goto dropafterack; } else TCPSTAT_ADD(tcps_rcvbyteafterwin, todrop); m_adj(m, -todrop); tlen -= todrop; thflags &= ~(TH_PUSH|TH_FIN); } /* * If last ACK falls within this segment's sequence numbers, * record its timestamp. * NOTE: * 1) That the test incorporates suggestions from the latest * proposal of the tcplw@cray.com list (Braden 1993/04/26). * 2) That updating only on newer timestamps interferes with * our earlier PAWS tests, so this check should be solely * predicated on the sequence space of this segment. * 3) That we modify the segment boundary check to be * Last.ACK.Sent <= SEG.SEQ + SEG.Len * instead of RFC1323's * Last.ACK.Sent < SEG.SEQ + SEG.Len, * This modified check allows us to overcome RFC1323's * limitations as described in Stevens TCP/IP Illustrated * Vol. 2 p.869. In such cases, we can still calculate the * RTT correctly when RCV.NXT == Last.ACK.Sent. */ if ((to.to_flags & TOF_TS) != 0 && SEQ_LEQ(th->th_seq, tp->last_ack_sent) && SEQ_LEQ(tp->last_ack_sent, th->th_seq + tlen + ((thflags & (TH_SYN|TH_FIN)) != 0))) { tp->ts_recent_age = ticks; tp->ts_recent = to.to_tsval; } /* * If a SYN is in the window, then this is an * error and we send an RST and drop the connection. */ if (thflags & TH_SYN) { KASSERT(ti_locked == TI_WLOCKED, ("tcp_do_segment: TH_SYN ti_locked %d", ti_locked)); INP_INFO_WLOCK_ASSERT(&V_tcbinfo); tp = tcp_drop(tp, ECONNRESET); rstreason = BANDLIM_UNLIMITED; goto drop; } /* * If the ACK bit is off: if in SYN-RECEIVED state or SENDSYN * flag is on (half-synchronized state), then queue data for * later processing; else drop segment and return. */ if ((thflags & TH_ACK) == 0) { if (tp->t_state == TCPS_SYN_RECEIVED || (tp->t_flags & TF_NEEDSYN)) goto step6; else if (tp->t_flags & TF_ACKNOW) goto dropafterack; else goto drop; } /* * Ack processing. */ switch (tp->t_state) { /* * In SYN_RECEIVED state, the ack ACKs our SYN, so enter * ESTABLISHED state and continue processing. * The ACK was checked above. */ case TCPS_SYN_RECEIVED: TCPSTAT_INC(tcps_connects); soisconnected(so); /* Do window scaling? */ if ((tp->t_flags & (TF_RCVD_SCALE|TF_REQ_SCALE)) == (TF_RCVD_SCALE|TF_REQ_SCALE)) { tp->rcv_scale = tp->request_r_scale; tp->snd_wnd = tiwin; } /* * Make transitions: * SYN-RECEIVED -> ESTABLISHED * SYN-RECEIVED* -> FIN-WAIT-1 */ tp->t_starttime = ticks; if (tp->t_flags & TF_NEEDFIN) { tp->t_state = TCPS_FIN_WAIT_1; tp->t_flags &= ~TF_NEEDFIN; } else { tp->t_state = TCPS_ESTABLISHED; cc_conn_init(tp); tcp_timer_activate(tp, TT_KEEP, tcp_keepidle); } /* * If segment contains data or ACK, will call tcp_reass() * later; if not, do so now to pass queued data to user. */ if (tlen == 0 && (thflags & TH_FIN) == 0) (void) tcp_reass(tp, (struct tcphdr *)0, 0, (struct mbuf *)0); tp->snd_wl1 = th->th_seq - 1; /* FALLTHROUGH */ /* * In ESTABLISHED state: drop duplicate ACKs; ACK out of range * ACKs. If the ack is in the range * tp->snd_una < th->th_ack <= tp->snd_max * then advance tp->snd_una to th->th_ack and drop * data from the retransmission queue. If this ACK reflects * more up to date window information we update our window information. */ case TCPS_ESTABLISHED: case TCPS_FIN_WAIT_1: case TCPS_FIN_WAIT_2: case TCPS_CLOSE_WAIT: case TCPS_CLOSING: case TCPS_LAST_ACK: if (SEQ_GT(th->th_ack, tp->snd_max)) { TCPSTAT_INC(tcps_rcvacktoomuch); goto dropafterack; } if ((tp->t_flags & TF_SACK_PERMIT) && ((to.to_flags & TOF_SACK) || !TAILQ_EMPTY(&tp->snd_holes))) tcp_sack_doack(tp, &to, th->th_ack); /* Run HHOOK_TCP_ESTABLISHED_IN helper hooks. */ hhook_run_tcp_est_in(tp, th, &to); if (SEQ_LEQ(th->th_ack, tp->snd_una)) { if (tlen == 0 && tiwin == tp->snd_wnd) { TCPSTAT_INC(tcps_rcvdupack); /* * If we have outstanding data (other than * a window probe), this is a completely * duplicate ack (ie, window info didn't * change), the ack is the biggest we've * seen and we've seen exactly our rexmt * threshhold of them, assume a packet * has been dropped and retransmit it. * Kludge snd_nxt & the congestion * window so we send only this one * packet. * * We know we're losing at the current * window size so do congestion avoidance * (set ssthresh to half the current window * and pull our congestion window back to * the new ssthresh). * * Dup acks mean that packets have left the * network (they're now cached at the receiver) * so bump cwnd by the amount in the receiver * to keep a constant cwnd packets in the * network. * * When using TCP ECN, notify the peer that * we reduced the cwnd. */ if (!tcp_timer_active(tp, TT_REXMT) || th->th_ack != tp->snd_una) tp->t_dupacks = 0; else if (++tp->t_dupacks > tcprexmtthresh || IN_FASTRECOVERY(tp->t_flags)) { cc_ack_received(tp, th, CC_DUPACK); if ((tp->t_flags & TF_SACK_PERMIT) && IN_FASTRECOVERY(tp->t_flags)) { int awnd; /* * Compute the amount of data in flight first. * We can inject new data into the pipe iff * we have less than 1/2 the original window's * worth of data in flight. */ awnd = (tp->snd_nxt - tp->snd_fack) + tp->sackhint.sack_bytes_rexmit; if (awnd < tp->snd_ssthresh) { tp->snd_cwnd += tp->t_maxseg; if (tp->snd_cwnd > tp->snd_ssthresh) tp->snd_cwnd = tp->snd_ssthresh; } } else tp->snd_cwnd += tp->t_maxseg; (void) tcp_output(tp); goto drop; } else if (tp->t_dupacks == tcprexmtthresh) { tcp_seq onxt = tp->snd_nxt; /* * If we're doing sack, check to * see if we're already in sack * recovery. If we're not doing sack, * check to see if we're in newreno * recovery. */ if (tp->t_flags & TF_SACK_PERMIT) { if (IN_FASTRECOVERY(tp->t_flags)) { tp->t_dupacks = 0; break; } } else { if (SEQ_LEQ(th->th_ack, tp->snd_recover)) { tp->t_dupacks = 0; break; } } /* Congestion signal before ack. */ cc_cong_signal(tp, th, CC_NDUPACK); cc_ack_received(tp, th, CC_DUPACK); tcp_timer_activate(tp, TT_REXMT, 0); tp->t_rtttime = 0; if (tp->t_flags & TF_SACK_PERMIT) { TCPSTAT_INC( tcps_sack_recovery_episode); tp->sack_newdata = tp->snd_nxt; tp->snd_cwnd = tp->t_maxseg; (void) tcp_output(tp); goto drop; } tp->snd_nxt = th->th_ack; tp->snd_cwnd = tp->t_maxseg; (void) tcp_output(tp); KASSERT(tp->snd_limited <= 2, ("%s: tp->snd_limited too big", __func__)); tp->snd_cwnd = tp->snd_ssthresh + tp->t_maxseg * (tp->t_dupacks - tp->snd_limited); if (SEQ_GT(onxt, tp->snd_nxt)) tp->snd_nxt = onxt; goto drop; } else if (V_tcp_do_rfc3042) { cc_ack_received(tp, th, CC_DUPACK); u_long oldcwnd = tp->snd_cwnd; tcp_seq oldsndmax = tp->snd_max; u_int sent; KASSERT(tp->t_dupacks == 1 || tp->t_dupacks == 2, ("%s: dupacks not 1 or 2", __func__)); if (tp->t_dupacks == 1) tp->snd_limited = 0; tp->snd_cwnd = (tp->snd_nxt - tp->snd_una) + (tp->t_dupacks - tp->snd_limited) * tp->t_maxseg; (void) tcp_output(tp); sent = tp->snd_max - oldsndmax; if (sent > tp->t_maxseg) { KASSERT((tp->t_dupacks == 2 && tp->snd_limited == 0) || (sent == tp->t_maxseg + 1 && tp->t_flags & TF_SENTFIN), ("%s: sent too much", __func__)); tp->snd_limited = 2; } else if (sent > 0) ++tp->snd_limited; tp->snd_cwnd = oldcwnd; goto drop; } } else tp->t_dupacks = 0; break; } KASSERT(SEQ_GT(th->th_ack, tp->snd_una), ("%s: th_ack <= snd_una", __func__)); /* * If the congestion window was inflated to account * for the other side's cached packets, retract it. */ if (IN_FASTRECOVERY(tp->t_flags)) { if (SEQ_LT(th->th_ack, tp->snd_recover)) { if (tp->t_flags & TF_SACK_PERMIT) tcp_sack_partialack(tp, th); else tcp_newreno_partial_ack(tp, th); } else cc_post_recovery(tp, th); } tp->t_dupacks = 0; /* * If we reach this point, ACK is not a duplicate, * i.e., it ACKs something we sent. */ if (tp->t_flags & TF_NEEDSYN) { /* * T/TCP: Connection was half-synchronized, and our * SYN has been ACK'd (so connection is now fully * synchronized). Go to non-starred state, * increment snd_una for ACK of SYN, and check if * we can do window scaling. */ tp->t_flags &= ~TF_NEEDSYN; tp->snd_una++; /* Do window scaling? */ if ((tp->t_flags & (TF_RCVD_SCALE|TF_REQ_SCALE)) == (TF_RCVD_SCALE|TF_REQ_SCALE)) { tp->rcv_scale = tp->request_r_scale; /* Send window already scaled. */ } } process_ACK: INP_INFO_LOCK_ASSERT(&V_tcbinfo); KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("tcp_input: process_ACK ti_locked %d", ti_locked)); INP_WLOCK_ASSERT(tp->t_inpcb); acked = BYTES_THIS_ACK(tp, th); TCPSTAT_INC(tcps_rcvackpack); TCPSTAT_ADD(tcps_rcvackbyte, acked); /* * If we just performed our first retransmit, and the ACK * arrives within our recovery window, then it was a mistake * to do the retransmit in the first place. Recover our * original cwnd and ssthresh, and proceed to transmit where * we left off. */ if (tp->t_rxtshift == 1 && (int)(ticks - tp->t_badrxtwin) < 0) cc_cong_signal(tp, th, CC_RTO_ERR); /* * If we have a timestamp reply, update smoothed * round trip time. If no timestamp is present but * transmit timer is running and timed sequence * number was acked, update smoothed round trip time. * Since we now have an rtt measurement, cancel the * timer backoff (cf., Phil Karn's retransmit alg.). * Recompute the initial retransmit timer. * * Some boxes send broken timestamp replies * during the SYN+ACK phase, ignore * timestamps of 0 or we could calculate a * huge RTT and blow up the retransmit timer. */ if ((to.to_flags & TOF_TS) != 0 && to.to_tsecr) { if (!tp->t_rttlow || tp->t_rttlow > ticks - to.to_tsecr) tp->t_rttlow = ticks - to.to_tsecr; tcp_xmit_timer(tp, ticks - to.to_tsecr + 1); } else if (tp->t_rtttime && SEQ_GT(th->th_ack, tp->t_rtseq)) { if (!tp->t_rttlow || tp->t_rttlow > ticks - tp->t_rtttime) tp->t_rttlow = ticks - tp->t_rtttime; tcp_xmit_timer(tp, ticks - tp->t_rtttime); } /* * If all outstanding data is acked, stop retransmit * timer and remember to restart (more output or persist). * If there is more data to be acked, restart retransmit * timer, using current (possibly backed-off) value. */ if (th->th_ack == tp->snd_max) { tcp_timer_activate(tp, TT_REXMT, 0); needoutput = 1; } else if (!tcp_timer_active(tp, TT_PERSIST)) tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur); /* * If no data (only SYN) was ACK'd, * skip rest of ACK processing. */ if (acked == 0) goto step6; /* * Let the congestion control algorithm update congestion * control related information. This typically means increasing * the congestion window. */ cc_ack_received(tp, th, CC_ACK); SOCKBUF_LOCK(&so->so_snd); if (acked > so->so_snd.sb_cc) { tp->snd_wnd -= so->so_snd.sb_cc; sbdrop_locked(&so->so_snd, (int)so->so_snd.sb_cc); ourfinisacked = 1; } else { sbdrop_locked(&so->so_snd, acked); tp->snd_wnd -= acked; ourfinisacked = 0; } /* NB: sowwakeup_locked() does an implicit unlock. */ sowwakeup_locked(so); /* Detect una wraparound. */ if (!IN_RECOVERY(tp->t_flags) && SEQ_GT(tp->snd_una, tp->snd_recover) && SEQ_LEQ(th->th_ack, tp->snd_recover)) tp->snd_recover = th->th_ack - 1; /* XXXLAS: Can this be moved up into cc_post_recovery? */ if (IN_RECOVERY(tp->t_flags) && SEQ_GEQ(th->th_ack, tp->snd_recover)) { EXIT_RECOVERY(tp->t_flags); } tp->snd_una = th->th_ack; if (tp->t_flags & TF_SACK_PERMIT) { if (SEQ_GT(tp->snd_una, tp->snd_recover)) tp->snd_recover = tp->snd_una; } if (SEQ_LT(tp->snd_nxt, tp->snd_una)) tp->snd_nxt = tp->snd_una; switch (tp->t_state) { /* * In FIN_WAIT_1 STATE in addition to the processing * for the ESTABLISHED state if our FIN is now acknowledged * then enter FIN_WAIT_2. */ case TCPS_FIN_WAIT_1: if (ourfinisacked) { /* * If we can't receive any more * data, then closing user can proceed. * Starting the timer is contrary to the * specification, but if we don't get a FIN * we'll hang forever. * * XXXjl: * we should release the tp also, and use a * compressed state. */ if (so->so_rcv.sb_state & SBS_CANTRCVMORE) { int timeout; soisdisconnected(so); timeout = (tcp_fast_finwait2_recycle) ? tcp_finwait2_timeout : tcp_maxidle; tcp_timer_activate(tp, TT_2MSL, timeout); } tp->t_state = TCPS_FIN_WAIT_2; } break; /* * In CLOSING STATE in addition to the processing for * the ESTABLISHED state if the ACK acknowledges our FIN * then enter the TIME-WAIT state, otherwise ignore * the segment. */ case TCPS_CLOSING: if (ourfinisacked) { INP_INFO_WLOCK_ASSERT(&V_tcbinfo); tcp_twstart(tp); INP_INFO_WUNLOCK(&V_tcbinfo); m_freem(m); return; } break; /* * In LAST_ACK, we may still be waiting for data to drain * and/or to be acked, as well as for the ack of our FIN. * If our FIN is now acknowledged, delete the TCB, * enter the closed state and return. */ case TCPS_LAST_ACK: if (ourfinisacked) { INP_INFO_WLOCK_ASSERT(&V_tcbinfo); tp = tcp_close(tp); goto drop; } break; } } step6: INP_INFO_LOCK_ASSERT(&V_tcbinfo); KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("tcp_do_segment: step6 ti_locked %d", ti_locked)); INP_WLOCK_ASSERT(tp->t_inpcb); /* * Update window information. * Don't look at window if no ACK: TAC's send garbage on first SYN. */ if ((thflags & TH_ACK) && (SEQ_LT(tp->snd_wl1, th->th_seq) || (tp->snd_wl1 == th->th_seq && (SEQ_LT(tp->snd_wl2, th->th_ack) || (tp->snd_wl2 == th->th_ack && tiwin > tp->snd_wnd))))) { /* keep track of pure window updates */ if (tlen == 0 && tp->snd_wl2 == th->th_ack && tiwin > tp->snd_wnd) TCPSTAT_INC(tcps_rcvwinupd); tp->snd_wnd = tiwin; tp->snd_wl1 = th->th_seq; tp->snd_wl2 = th->th_ack; if (tp->snd_wnd > tp->max_sndwnd) tp->max_sndwnd = tp->snd_wnd; needoutput = 1; } /* * Process segments with URG. */ if ((thflags & TH_URG) && th->th_urp && TCPS_HAVERCVDFIN(tp->t_state) == 0) { /* * This is a kludge, but if we receive and accept * random urgent pointers, we'll crash in * soreceive. It's hard to imagine someone * actually wanting to send this much urgent data. */ SOCKBUF_LOCK(&so->so_rcv); if (th->th_urp + so->so_rcv.sb_cc > sb_max) { th->th_urp = 0; /* XXX */ thflags &= ~TH_URG; /* XXX */ SOCKBUF_UNLOCK(&so->so_rcv); /* XXX */ goto dodata; /* XXX */ } /* * If this segment advances the known urgent pointer, * then mark the data stream. This should not happen * in CLOSE_WAIT, CLOSING, LAST_ACK or TIME_WAIT STATES since * a FIN has been received from the remote side. * In these states we ignore the URG. * * According to RFC961 (Assigned Protocols), * the urgent pointer points to the last octet * of urgent data. We continue, however, * to consider it to indicate the first octet * of data past the urgent section as the original * spec states (in one of two places). */ if (SEQ_GT(th->th_seq+th->th_urp, tp->rcv_up)) { tp->rcv_up = th->th_seq + th->th_urp; so->so_oobmark = so->so_rcv.sb_cc + (tp->rcv_up - tp->rcv_nxt) - 1; if (so->so_oobmark == 0) so->so_rcv.sb_state |= SBS_RCVATMARK; sohasoutofband(so); tp->t_oobflags &= ~(TCPOOB_HAVEDATA | TCPOOB_HADDATA); } SOCKBUF_UNLOCK(&so->so_rcv); /* * Remove out of band data so doesn't get presented to user. * This can happen independent of advancing the URG pointer, * but if two URG's are pending at once, some out-of-band * data may creep in... ick. */ if (th->th_urp <= (u_long)tlen && !(so->so_options & SO_OOBINLINE)) { /* hdr drop is delayed */ tcp_pulloutofband(so, th, m, drop_hdrlen); } } else { /* * If no out of band data is expected, * pull receive urgent pointer along * with the receive window. */ if (SEQ_GT(tp->rcv_nxt, tp->rcv_up)) tp->rcv_up = tp->rcv_nxt; } dodata: /* XXX */ INP_INFO_LOCK_ASSERT(&V_tcbinfo); KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("tcp_do_segment: dodata ti_locked %d", ti_locked)); INP_WLOCK_ASSERT(tp->t_inpcb); /* * Process the segment text, merging it into the TCP sequencing queue, * and arranging for acknowledgment of receipt if necessary. * This process logically involves adjusting tp->rcv_wnd as data * is presented to the user (this happens in tcp_usrreq.c, * case PRU_RCVD). If a FIN has already been received on this * connection then we just ignore the text. */ if ((tlen || (thflags & TH_FIN)) && TCPS_HAVERCVDFIN(tp->t_state) == 0) { tcp_seq save_start = th->th_seq; m_adj(m, drop_hdrlen); /* delayed header drop */ /* * Insert segment which includes th into TCP reassembly queue * with control block tp. Set thflags to whether reassembly now * includes a segment with FIN. This handles the common case * inline (segment is the next to be received on an established * connection, and the queue is empty), avoiding linkage into * and removal from the queue and repetition of various * conversions. * Set DELACK for segments received in order, but ack * immediately when segments are out of order (so * fast retransmit can work). */ if (th->th_seq == tp->rcv_nxt && LIST_EMPTY(&tp->t_segq) && TCPS_HAVEESTABLISHED(tp->t_state)) { if (DELAY_ACK(tp)) tp->t_flags |= TF_DELACK; else tp->t_flags |= TF_ACKNOW; tp->rcv_nxt += tlen; thflags = th->th_flags & TH_FIN; TCPSTAT_INC(tcps_rcvpack); TCPSTAT_ADD(tcps_rcvbyte, tlen); ND6_HINT(tp); SOCKBUF_LOCK(&so->so_rcv); if (so->so_rcv.sb_state & SBS_CANTRCVMORE) m_freem(m); else sbappendstream_locked(&so->so_rcv, m); /* NB: sorwakeup_locked() does an implicit unlock. */ sorwakeup_locked(so); } else { /* * XXX: Due to the header drop above "th" is * theoretically invalid by now. Fortunately * m_adj() doesn't actually frees any mbufs * when trimming from the head. */ thflags = tcp_reass(tp, th, &tlen, m); tp->t_flags |= TF_ACKNOW; } if (tlen > 0 && (tp->t_flags & TF_SACK_PERMIT)) tcp_update_sack_list(tp, save_start, save_start + tlen); #if 0 /* * Note the amount of data that peer has sent into * our window, in order to estimate the sender's * buffer size. * XXX: Unused. */ len = so->so_rcv.sb_hiwat - (tp->rcv_adv - tp->rcv_nxt); #endif } else { m_freem(m); thflags &= ~TH_FIN; } /* * If FIN is received ACK the FIN and let the user know * that the connection is closing. */ if (thflags & TH_FIN) { if (TCPS_HAVERCVDFIN(tp->t_state) == 0) { socantrcvmore(so); /* * If connection is half-synchronized * (ie NEEDSYN flag on) then delay ACK, * so it may be piggybacked when SYN is sent. * Otherwise, since we received a FIN then no * more input can be expected, send ACK now. */ if (tp->t_flags & TF_NEEDSYN) tp->t_flags |= TF_DELACK; else tp->t_flags |= TF_ACKNOW; tp->rcv_nxt++; } switch (tp->t_state) { /* * In SYN_RECEIVED and ESTABLISHED STATES * enter the CLOSE_WAIT state. */ case TCPS_SYN_RECEIVED: tp->t_starttime = ticks; /* FALLTHROUGH */ case TCPS_ESTABLISHED: tp->t_state = TCPS_CLOSE_WAIT; break; /* * If still in FIN_WAIT_1 STATE FIN has not been acked so * enter the CLOSING state. */ case TCPS_FIN_WAIT_1: tp->t_state = TCPS_CLOSING; break; /* * In FIN_WAIT_2 state enter the TIME_WAIT state, * starting the time-wait timer, turning off the other * standard timers. */ case TCPS_FIN_WAIT_2: INP_INFO_WLOCK_ASSERT(&V_tcbinfo); KASSERT(ti_locked == TI_WLOCKED, ("%s: dodata " "TCP_FIN_WAIT_2 ti_locked: %d", __func__, ti_locked)); tcp_twstart(tp); INP_INFO_WUNLOCK(&V_tcbinfo); return; } } if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: dodata epilogue ti_locked %d", __func__, ti_locked); ti_locked = TI_UNLOCKED; #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) tcp_trace(TA_INPUT, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif /* * Return any desired output. */ if (needoutput || (tp->t_flags & TF_ACKNOW)) (void) tcp_output(tp); check_delack: KASSERT(ti_locked == TI_UNLOCKED, ("%s: check_delack ti_locked %d", __func__, ti_locked)); INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tp->t_inpcb); if (tp->t_flags & TF_DELACK) { tp->t_flags &= ~TF_DELACK; tcp_timer_activate(tp, TT_DELACK, tcp_delacktime); } INP_WUNLOCK(tp->t_inpcb); return; dropafterack: KASSERT(ti_locked == TI_RLOCKED || ti_locked == TI_WLOCKED, ("tcp_do_segment: dropafterack ti_locked %d", ti_locked)); /* * Generate an ACK dropping incoming segment if it occupies * sequence space, where the ACK reflects our state. * * We can now skip the test for the RST flag since all * paths to this code happen after packets containing * RST have been dropped. * * In the SYN-RECEIVED state, don't send an ACK unless the * segment we received passes the SYN-RECEIVED ACK test. * If it fails send a RST. This breaks the loop in the * "LAND" DoS attack, and also prevents an ACK storm * between two listening ports that have been sent forged * SYN segments, each with the source address of the other. */ if (tp->t_state == TCPS_SYN_RECEIVED && (thflags & TH_ACK) && (SEQ_GT(tp->snd_una, th->th_ack) || SEQ_GT(th->th_ack, tp->snd_max)) ) { rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } #ifdef TCPDEBUG if (so->so_options & SO_DEBUG) tcp_trace(TA_DROP, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: dropafterack epilogue ti_locked %d", __func__, ti_locked); ti_locked = TI_UNLOCKED; tp->t_flags |= TF_ACKNOW; (void) tcp_output(tp); INP_WUNLOCK(tp->t_inpcb); m_freem(m); return; dropwithreset: if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); else panic("%s: dropwithreset ti_locked %d", __func__, ti_locked); ti_locked = TI_UNLOCKED; if (tp != NULL) { tcp_dropwithreset(m, th, tp, tlen, rstreason); INP_WUNLOCK(tp->t_inpcb); } else tcp_dropwithreset(m, th, NULL, tlen, rstreason); return; drop: if (ti_locked == TI_RLOCKED) INP_INFO_RUNLOCK(&V_tcbinfo); else if (ti_locked == TI_WLOCKED) INP_INFO_WUNLOCK(&V_tcbinfo); #ifdef INVARIANTS else INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); #endif ti_locked = TI_UNLOCKED; /* * Drop space held by incoming segment and return. */ #ifdef TCPDEBUG if (tp == NULL || (tp->t_inpcb->inp_socket->so_options & SO_DEBUG)) tcp_trace(TA_DROP, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif if (tp != NULL) INP_WUNLOCK(tp->t_inpcb); m_freem(m); } /* * Issue RST and make ACK acceptable to originator of segment. * The mbuf must still include the original packet header. * tp may be NULL. */ static void tcp_dropwithreset(struct mbuf *m, struct tcphdr *th, struct tcpcb *tp, int tlen, int rstreason) { struct ip *ip; #ifdef INET6 struct ip6_hdr *ip6; #endif if (tp != NULL) { INP_WLOCK_ASSERT(tp->t_inpcb); } /* Don't bother if destination was broadcast/multicast. */ if ((th->th_flags & TH_RST) || m->m_flags & (M_BCAST|M_MCAST)) goto drop; #ifdef INET6 if (mtod(m, struct ip *)->ip_v == 6) { ip6 = mtod(m, struct ip6_hdr *); if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst) || IN6_IS_ADDR_MULTICAST(&ip6->ip6_src)) goto drop; /* IPv6 anycast check is done at tcp6_input() */ } else #endif { ip = mtod(m, struct ip *); if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) || IN_MULTICAST(ntohl(ip->ip_src.s_addr)) || ip->ip_src.s_addr == htonl(INADDR_BROADCAST) || in_broadcast(ip->ip_dst, m->m_pkthdr.rcvif)) goto drop; } /* Perform bandwidth limiting. */ if (badport_bandlim(rstreason) < 0) goto drop; /* tcp_respond consumes the mbuf chain. */ if (th->th_flags & TH_ACK) { tcp_respond(tp, mtod(m, void *), th, m, (tcp_seq)0, th->th_ack, TH_RST); } else { if (th->th_flags & TH_SYN) tlen++; tcp_respond(tp, mtod(m, void *), th, m, th->th_seq+tlen, (tcp_seq)0, TH_RST|TH_ACK); } return; drop: m_freem(m); } /* * Parse TCP options and place in tcpopt. */ static void tcp_dooptions(struct tcpopt *to, u_char *cp, int cnt, int flags) { int opt, optlen; to->to_flags = 0; for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[0]; if (opt == TCPOPT_EOL) break; if (opt == TCPOPT_NOP) optlen = 1; else { if (cnt < 2) break; optlen = cp[1]; if (optlen < 2 || optlen > cnt) break; } switch (opt) { case TCPOPT_MAXSEG: if (optlen != TCPOLEN_MAXSEG) continue; if (!(flags & TO_SYN)) continue; to->to_flags |= TOF_MSS; bcopy((char *)cp + 2, (char *)&to->to_mss, sizeof(to->to_mss)); to->to_mss = ntohs(to->to_mss); break; case TCPOPT_WINDOW: if (optlen != TCPOLEN_WINDOW) continue; if (!(flags & TO_SYN)) continue; to->to_flags |= TOF_SCALE; to->to_wscale = min(cp[2], TCP_MAX_WINSHIFT); break; case TCPOPT_TIMESTAMP: if (optlen != TCPOLEN_TIMESTAMP) continue; to->to_flags |= TOF_TS; bcopy((char *)cp + 2, (char *)&to->to_tsval, sizeof(to->to_tsval)); to->to_tsval = ntohl(to->to_tsval); bcopy((char *)cp + 6, (char *)&to->to_tsecr, sizeof(to->to_tsecr)); to->to_tsecr = ntohl(to->to_tsecr); break; #ifdef TCP_SIGNATURE /* * XXX In order to reply to a host which has set the * TCP_SIGNATURE option in its initial SYN, we have to * record the fact that the option was observed here * for the syncache code to perform the correct response. */ case TCPOPT_SIGNATURE: if (optlen != TCPOLEN_SIGNATURE) continue; to->to_flags |= TOF_SIGNATURE; to->to_signature = cp + 2; break; #endif case TCPOPT_SACK_PERMITTED: if (optlen != TCPOLEN_SACK_PERMITTED) continue; if (!(flags & TO_SYN)) continue; if (!V_tcp_do_sack) continue; to->to_flags |= TOF_SACKPERM; break; case TCPOPT_SACK: if (optlen <= 2 || (optlen - 2) % TCPOLEN_SACK != 0) continue; if (flags & TO_SYN) continue; to->to_flags |= TOF_SACK; to->to_nsacks = (optlen - 2) / TCPOLEN_SACK; to->to_sacks = cp + 2; TCPSTAT_INC(tcps_sack_rcv_blocks); break; default: continue; } } } /* * Pull out of band byte out of a segment so * it doesn't appear in the user's data queue. * It is still reflected in the segment length for * sequencing purposes. */ static void tcp_pulloutofband(struct socket *so, struct tcphdr *th, struct mbuf *m, int off) { int cnt = off + th->th_urp - 1; while (cnt >= 0) { if (m->m_len > cnt) { char *cp = mtod(m, caddr_t) + cnt; struct tcpcb *tp = sototcpcb(so); INP_WLOCK_ASSERT(tp->t_inpcb); tp->t_iobc = *cp; tp->t_oobflags |= TCPOOB_HAVEDATA; bcopy(cp+1, cp, (unsigned)(m->m_len - cnt - 1)); m->m_len--; if (m->m_flags & M_PKTHDR) m->m_pkthdr.len--; return; } cnt -= m->m_len; m = m->m_next; if (m == NULL) break; } panic("tcp_pulloutofband"); } /* * Collect new round-trip time estimate * and update averages and current timeout. */ static void tcp_xmit_timer(struct tcpcb *tp, int rtt) { int delta; INP_WLOCK_ASSERT(tp->t_inpcb); TCPSTAT_INC(tcps_rttupdated); tp->t_rttupdated++; if (tp->t_srtt != 0) { /* * srtt is stored as fixed point with 5 bits after the * binary point (i.e., scaled by 8). The following magic * is equivalent to the smoothing algorithm in rfc793 with * an alpha of .875 (srtt = rtt/8 + srtt*7/8 in fixed * point). Adjust rtt to origin 0. */ delta = ((rtt - 1) << TCP_DELTA_SHIFT) - (tp->t_srtt >> (TCP_RTT_SHIFT - TCP_DELTA_SHIFT)); if ((tp->t_srtt += delta) <= 0) tp->t_srtt = 1; /* * We accumulate a smoothed rtt variance (actually, a * smoothed mean difference), then set the retransmit * timer to smoothed rtt + 4 times the smoothed variance. * rttvar is stored as fixed point with 4 bits after the * binary point (scaled by 16). The following is * equivalent to rfc793 smoothing with an alpha of .75 * (rttvar = rttvar*3/4 + |delta| / 4). This replaces * rfc793's wired-in beta. */ if (delta < 0) delta = -delta; delta -= tp->t_rttvar >> (TCP_RTTVAR_SHIFT - TCP_DELTA_SHIFT); if ((tp->t_rttvar += delta) <= 0) tp->t_rttvar = 1; if (tp->t_rttbest > tp->t_srtt + tp->t_rttvar) tp->t_rttbest = tp->t_srtt + tp->t_rttvar; } else { /* * No rtt measurement yet - use the unsmoothed rtt. * Set the variance to half the rtt (so our first * retransmit happens at 3*rtt). */ tp->t_srtt = rtt << TCP_RTT_SHIFT; tp->t_rttvar = rtt << (TCP_RTTVAR_SHIFT - 1); tp->t_rttbest = tp->t_srtt + tp->t_rttvar; } tp->t_rtttime = 0; tp->t_rxtshift = 0; /* * the retransmit should happen at rtt + 4 * rttvar. * Because of the way we do the smoothing, srtt and rttvar * will each average +1/2 tick of bias. When we compute * the retransmit timer, we want 1/2 tick of rounding and * 1 extra tick because of +-1/2 tick uncertainty in the * firing of the timer. The bias will give us exactly the * 1.5 tick we need. But, because the bias is * statistical, we have to test that we don't drop below * the minimum feasible timer (which is 2 ticks). */ TCPT_RANGESET(tp->t_rxtcur, TCP_REXMTVAL(tp), max(tp->t_rttmin, rtt + 2), TCPTV_REXMTMAX); /* * We received an ack for a packet that wasn't retransmitted; * it is probably safe to discard any error indications we've * received recently. This isn't quite right, but close enough * for now (a route might have failed after we sent a segment, * and the return path might not be symmetrical). */ tp->t_softerror = 0; } /* * Determine a reasonable value for maxseg size. * If the route is known, check route for mtu. * If none, use an mss that can be handled on the outgoing * interface without forcing IP to fragment; if bigger than * an mbuf cluster (MCLBYTES), round down to nearest multiple of MCLBYTES * to utilize large mbufs. If no route is found, route has no mtu, * or the destination isn't local, use a default, hopefully conservative * size (usually 512 or the default IP max size, but no more than the mtu * of the interface), as we can't discover anything about intervening * gateways or networks. We also initialize the congestion/slow start * window to be a single segment if the destination isn't local. * While looking at the routing entry, we also initialize other path-dependent * parameters from pre-set or cached values in the routing entry. * * Also take into account the space needed for options that we * send regularly. Make maxseg shorter by that amount to assure * that we can send maxseg amount of data even when the options * are present. Store the upper limit of the length of options plus * data in maxopd. * * In case of T/TCP, we call this routine during implicit connection * setup as well (offer = -1), to initialize maxseg from the cached * MSS of our peer. * * NOTE that this routine is only called when we process an incoming * segment. Outgoing SYN/ACK MSS settings are handled in tcp_mssopt(). */ void tcp_mss_update(struct tcpcb *tp, int offer, struct hc_metrics_lite *metricptr, int *mtuflags) { int mss; u_long maxmtu; struct inpcb *inp = tp->t_inpcb; struct hc_metrics_lite metrics; int origoffer = offer; #ifdef INET6 int isipv6 = ((inp->inp_vflag & INP_IPV6) != 0) ? 1 : 0; size_t min_protoh = isipv6 ? sizeof (struct ip6_hdr) + sizeof (struct tcphdr) : sizeof (struct tcpiphdr); #else const size_t min_protoh = sizeof(struct tcpiphdr); #endif INP_WLOCK_ASSERT(tp->t_inpcb); /* Initialize. */ #ifdef INET6 if (isipv6) { maxmtu = tcp_maxmtu6(&inp->inp_inc, mtuflags); tp->t_maxopd = tp->t_maxseg = V_tcp_v6mssdflt; } else #endif { maxmtu = tcp_maxmtu(&inp->inp_inc, mtuflags); tp->t_maxopd = tp->t_maxseg = V_tcp_mssdflt; } /* * No route to sender, stay with default mss and return. */ if (maxmtu == 0) { /* * In case we return early we need to initialize metrics * to a defined state as tcp_hc_get() would do for us * if there was no cache hit. */ if (metricptr != NULL) bzero(metricptr, sizeof(struct hc_metrics_lite)); return; } /* What have we got? */ switch (offer) { case 0: /* * Offer == 0 means that there was no MSS on the SYN * segment, in this case we use tcp_mssdflt as * already assigned to t_maxopd above. */ offer = tp->t_maxopd; break; case -1: /* * Offer == -1 means that we didn't receive SYN yet. */ /* FALLTHROUGH */ default: /* * Prevent DoS attack with too small MSS. Round up * to at least minmss. */ offer = max(offer, V_tcp_minmss); } /* * rmx information is now retrieved from tcp_hostcache. */ tcp_hc_get(&inp->inp_inc, &metrics); if (metricptr != NULL) bcopy(&metrics, metricptr, sizeof(struct hc_metrics_lite)); /* * If there's a discovered mtu int tcp hostcache, use it * else, use the link mtu. */ if (metrics.rmx_mtu) mss = min(metrics.rmx_mtu, maxmtu) - min_protoh; else { #ifdef INET6 if (isipv6) { mss = maxmtu - min_protoh; if (!V_path_mtu_discovery && !in6_localaddr(&inp->in6p_faddr)) mss = min(mss, V_tcp_v6mssdflt); } else #endif { mss = maxmtu - min_protoh; if (!V_path_mtu_discovery && !in_localaddr(inp->inp_faddr)) mss = min(mss, V_tcp_mssdflt); } /* * XXX - The above conditional (mss = maxmtu - min_protoh) * probably violates the TCP spec. * The problem is that, since we don't know the * other end's MSS, we are supposed to use a conservative * default. But, if we do that, then MTU discovery will * never actually take place, because the conservative * default is much less than the MTUs typically seen * on the Internet today. For the moment, we'll sweep * this under the carpet. * * The conservative default might not actually be a problem * if the only case this occurs is when sending an initial * SYN with options and data to a host we've never talked * to before. Then, they will reply with an MSS value which * will get recorded and the new parameters should get * recomputed. For Further Study. */ } mss = min(mss, offer); /* * Sanity check: make sure that maxopd will be large * enough to allow some data on segments even if the * all the option space is used (40bytes). Otherwise * funny things may happen in tcp_output. */ mss = max(mss, 64); /* * maxopd stores the maximum length of data AND options * in a segment; maxseg is the amount of data in a normal * segment. We need to store this value (maxopd) apart * from maxseg, because now every segment carries options * and thus we normally have somewhat less data in segments. */ tp->t_maxopd = mss; /* * origoffer==-1 indicates that no segments were received yet. * In this case we just guess. */ if ((tp->t_flags & (TF_REQ_TSTMP|TF_NOOPT)) == TF_REQ_TSTMP && (origoffer == -1 || (tp->t_flags & TF_RCVD_TSTMP) == TF_RCVD_TSTMP)) mss -= TCPOLEN_TSTAMP_APPA; #if (MCLBYTES & (MCLBYTES - 1)) == 0 if (mss > MCLBYTES) mss &= ~(MCLBYTES-1); #else if (mss > MCLBYTES) mss = mss / MCLBYTES * MCLBYTES; #endif tp->t_maxseg = mss; } void tcp_mss(struct tcpcb *tp, int offer) { int mss; u_long bufsize; struct inpcb *inp; struct socket *so; struct hc_metrics_lite metrics; int mtuflags = 0; KASSERT(tp != NULL, ("%s: tp == NULL", __func__)); tcp_mss_update(tp, offer, &metrics, &mtuflags); mss = tp->t_maxseg; inp = tp->t_inpcb; /* * If there's a pipesize, change the socket buffer to that size, * don't change if sb_hiwat is different than default (then it * has been changed on purpose with setsockopt). * Make the socket buffers an integral number of mss units; * if the mss is larger than the socket buffer, decrease the mss. */ so = inp->inp_socket; SOCKBUF_LOCK(&so->so_snd); if ((so->so_snd.sb_hiwat == tcp_sendspace) && metrics.rmx_sendpipe) bufsize = metrics.rmx_sendpipe; else bufsize = so->so_snd.sb_hiwat; if (bufsize < mss) mss = bufsize; else { bufsize = roundup(bufsize, mss); if (bufsize > sb_max) bufsize = sb_max; if (bufsize > so->so_snd.sb_hiwat) (void)sbreserve_locked(&so->so_snd, bufsize, so, NULL); } SOCKBUF_UNLOCK(&so->so_snd); tp->t_maxseg = mss; SOCKBUF_LOCK(&so->so_rcv); if ((so->so_rcv.sb_hiwat == tcp_recvspace) && metrics.rmx_recvpipe) bufsize = metrics.rmx_recvpipe; else bufsize = so->so_rcv.sb_hiwat; if (bufsize > mss) { bufsize = roundup(bufsize, mss); if (bufsize > sb_max) bufsize = sb_max; if (bufsize > so->so_rcv.sb_hiwat) (void)sbreserve_locked(&so->so_rcv, bufsize, so, NULL); } SOCKBUF_UNLOCK(&so->so_rcv); /* Check the interface for TSO capabilities. */ if (mtuflags & CSUM_TSO) tp->t_flags |= TF_TSO; } /* * Determine the MSS option to send on an outgoing SYN. */ int tcp_mssopt(struct in_conninfo *inc) { int mss = 0; u_long maxmtu = 0; u_long thcmtu = 0; size_t min_protoh; KASSERT(inc != NULL, ("tcp_mssopt with NULL in_conninfo pointer")); #ifdef INET6 if (inc->inc_flags & INC_ISIPV6) { mss = V_tcp_v6mssdflt; maxmtu = tcp_maxmtu6(inc, NULL); thcmtu = tcp_hc_getmtu(inc); /* IPv4 and IPv6 */ min_protoh = sizeof(struct ip6_hdr) + sizeof(struct tcphdr); } else #endif { mss = V_tcp_mssdflt; maxmtu = tcp_maxmtu(inc, NULL); thcmtu = tcp_hc_getmtu(inc); /* IPv4 and IPv6 */ min_protoh = sizeof(struct tcpiphdr); } if (maxmtu && thcmtu) mss = min(maxmtu, thcmtu) - min_protoh; else if (maxmtu || thcmtu) mss = max(maxmtu, thcmtu) - min_protoh; return (mss); } /* * On a partial ack arrives, force the retransmission of the * next unacknowledged segment. Do not clear tp->t_dupacks. * By setting snd_nxt to ti_ack, this forces retransmission timer to * be started again. */ static void tcp_newreno_partial_ack(struct tcpcb *tp, struct tcphdr *th) { tcp_seq onxt = tp->snd_nxt; u_long ocwnd = tp->snd_cwnd; INP_WLOCK_ASSERT(tp->t_inpcb); tcp_timer_activate(tp, TT_REXMT, 0); tp->t_rtttime = 0; tp->snd_nxt = th->th_ack; /* * Set snd_cwnd to one segment beyond acknowledged offset. * (tp->snd_una has not yet been updated when this function is called.) */ tp->snd_cwnd = tp->t_maxseg + BYTES_THIS_ACK(tp, th); tp->t_flags |= TF_ACKNOW; (void) tcp_output(tp); tp->snd_cwnd = ocwnd; if (SEQ_GT(onxt, tp->snd_nxt)) tp->snd_nxt = onxt; /* * Partial window deflation. Relies on fact that tp->snd_una * not updated yet. */ if (tp->snd_cwnd > BYTES_THIS_ACK(tp, th)) tp->snd_cwnd -= BYTES_THIS_ACK(tp, th); else tp->snd_cwnd = 0; tp->snd_cwnd += tp->t_maxseg; } Index: head/sys/sys/hhook.h =================================================================== --- head/sys/sys/hhook.h (revision 220559) +++ head/sys/sys/hhook.h (revision 220560) @@ -1,156 +1,156 @@ /*- * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by grants from the FreeBSD Foundation and Cisco University Research - * Program Fund at Community Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by grants from the FreeBSD Foundation and Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * A KPI modelled on the pfil framework for instantiating helper hook points * within the kernel for use by Khelp modules. Originally released as part of - * the NewTCP research project at Swinburne University's Centre for Advanced - * Internet Architectures, Melbourne, Australia, which was made possible in part - * by a grant from the Cisco University Research Program Fund at Community - * Foundation Silicon Valley. More details are available at: + * the NewTCP research project at Swinburne University of Technology's Centre + * for Advanced Internet Architectures, Melbourne, Australia, which was made + * possible in part by a grant from the Cisco University Research Program Fund + * at Community Foundation Silicon Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #ifndef _SYS_HHOOK_H_ #define _SYS_HHOOK_H_ /* XXXLAS: Is there a way around this? */ #include #include /* hhook_head flags. */ #define HHH_ISINVNET 0x00000001 /* Is the hook point in a vnet? */ /* Flags common to all register functions. */ #define HHOOK_WAITOK 0x00000001 /* Sleeping allowed. */ #define HHOOK_NOWAIT 0x00000002 /* Sleeping disallowed. */ /* Flags only relevant to hhook_head_register() and hhook_head_is_virtual(). */ #define HHOOK_HEADISINVNET 0x00000100 /* Public proxy for HHH_ISINVNET. */ /* Helper hook types. */ #define HHOOK_TYPE_TCP 1 struct helper; struct osd; /* Signature for helper hook functions. */ typedef int (*hhook_func_t)(int32_t hhook_type, int32_t hhook_id, void *udata, void *ctx_data, void *hdata, struct osd *hosd); /* * Information required to add/remove a helper hook function to/from a helper * hook point. */ struct hookinfo { hhook_func_t hook_func; struct helper *hook_helper; void *hook_udata; int32_t hook_id; int32_t hook_type; }; /* * Ideally this would be private but we need access to the hhh_nhooks member * variable in order to make the HHOOKS_RUN_IF() macro low impact. */ struct hhook_head { STAILQ_HEAD(hhook_list, hhook) hhh_hooks; struct rmlock hhh_lock; int32_t hhh_id; int32_t hhh_nhooks; int32_t hhh_type; uint32_t hhh_flags; volatile uint32_t hhh_refcount; LIST_ENTRY(hhook_head) hhh_next; }; /* Public KPI functions. */ void hhook_run_hooks(struct hhook_head *hhh, void *ctx_data, struct osd *hosd); int hhook_add_hook(struct hhook_head *hhh, struct hookinfo *hki, uint32_t flags); int hhook_add_hook_lookup(struct hookinfo *hki, uint32_t flags); int hhook_remove_hook(struct hhook_head *hhh, struct hookinfo *hki); int hhook_remove_hook_lookup(struct hookinfo *hki); int hhook_head_register(int32_t hhook_type, int32_t hhook_id, struct hhook_head **hhh, uint32_t flags); int hhook_head_deregister(struct hhook_head *hhh); int hhook_head_deregister_lookup(int32_t hhook_type, int32_t hhook_id); struct hhook_head * hhook_head_get(int32_t hhook_type, int32_t hhook_id); void hhook_head_release(struct hhook_head *hhh); uint32_t hhook_head_is_virtualised(struct hhook_head *hhh); uint32_t hhook_head_is_virtualised_lookup(int32_t hook_type, int32_t hook_id); /* * A wrapper around hhook_run_hooks() that only calls the function if at least * one helper hook function is registered for the specified helper hook point. */ #define HHOOKS_RUN_IF(hhh, ctx_data, hosd) do { \ if (hhh != NULL && hhh->hhh_nhooks > 0) \ hhook_run_hooks(hhh, ctx_data, hosd); \ } while (0) /* * WARNING: This macro should only be used in code paths that execute * infrequently, otherwise the refcounting overhead would be excessive. * * A similar wrapper to HHOOKS_RUN_IF() for situations where the caller prefers * not to lookup and store the appropriate hhook_head pointer themselves. */ #define HHOOKS_RUN_LOOKUP_IF(hhook_type, hhook_id, ctx_data, hosd) do { \ struct hhook_head *_hhh; \ \ _hhh = hhook_head_get(hhook_type, hhook_id); \ if (_hhh != NULL) { \ if (_hhh->hhh_nhooks > 0) \ hhook_run_hooks(_hhh, ctx_data, hosd); \ hhook_head_release(_hhh); \ } \ } while (0) #endif /* _SYS_HHOOK_H_ */ Index: head/sys/sys/khelp.h =================================================================== --- head/sys/sys/khelp.h (revision 220559) +++ head/sys/sys/khelp.h (revision 220560) @@ -1,76 +1,76 @@ /*- * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by grants from the FreeBSD Foundation and Cisco University Research - * Program Fund at Community Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by grants from the FreeBSD Foundation and Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * A KPI for managing kernel helper modules which perform useful functionality - * within the kernel. Originally released as part of the NewTCP research - * project at Swinburne University's Centre for Advanced Internet Architectures, - * Melbourne, Australia, which was made possible in part by a grant from the - * Cisco University Research Program Fund at Community Foundation Silicon - * Valley. More details are available at: + * within the kernel. Originally released as part of the NewTCP research project + * at Swinburne University of Technology's Centre for Advanced Internet + * Architectures, Melbourne, Australia, which was made possible in part by a + * grant from the Cisco University Research Program Fund at Community Foundation + * Silicon Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #ifndef _SYS_KHELP_H_ #define _SYS_KHELP_H_ struct helper; struct hookinfo; struct osd; /* Helper classes. */ #define HELPER_CLASS_TCP 0x00000001 /* Public KPI functions. */ int khelp_register_helper(struct helper *h); int khelp_deregister_helper(struct helper *h); int khelp_init_osd(uint32_t classes, struct osd *hosd); int khelp_destroy_osd(struct osd *hosd); void * khelp_get_osd(struct osd *hosd, int32_t id); int32_t khelp_get_id(char *hname); int khelp_add_hhook(struct hookinfo *hki, uint32_t flags); int khelp_remove_hhook(struct hookinfo *hki); #endif /* _SYS_KHELP_H_ */ Index: head/sys/sys/module_khelp.h =================================================================== --- head/sys/sys/module_khelp.h (revision 220559) +++ head/sys/sys/module_khelp.h (revision 220560) @@ -1,110 +1,110 @@ /*- * Copyright (c) 2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre - * for Advanced Internet Architectures, Swinburne University, made possible in - * part by grants from the FreeBSD Foundation and Cisco University Research - * Program Fund at Community Foundation Silicon Valley. + * for Advanced Internet Architectures, Swinburne University of Technology, made + * possible in part by grants from the FreeBSD Foundation and Cisco University + * Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _SYS_MODULE_KHELP_H_ #define _SYS_MODULE_KHELP_H_ /* XXXLAS: Needed for uma related typedefs. */ #include /* Helper flags. */ #define HELPER_NEEDS_OSD 0x0001 struct helper { int (*mod_init) (void); int (*mod_destroy) (void); #define HELPER_NAME_MAXLEN 16 char h_name[HELPER_NAME_MAXLEN]; uma_zone_t h_zone; struct hookinfo *h_hooks; uint32_t h_nhooks; uint32_t h_classes; int32_t h_id; volatile uint32_t h_refcount; uint16_t h_flags; TAILQ_ENTRY(helper) h_next; }; struct khelp_modevent_data { char name[HELPER_NAME_MAXLEN]; struct helper *helper; struct hookinfo *hooks; int nhooks; int uma_zsize; uma_ctor umactor; uma_dtor umadtor; }; #define KHELP_DECLARE_MOD(hname, hdata, hhooks, version) \ static struct khelp_modevent_data kmd_##hname = { \ .name = #hname, \ .helper = hdata \ .hooks = hhooks, \ .nhooks = sizeof(hhooks) / sizeof(hhooks[0]), \ }; \ static moduledata_t h_##hname = { \ .name = #hname, \ .evhand = khelp_modevent, \ .priv = &kmd_##hname \ }; \ DECLARE_MODULE(hname, h_##hname, SI_SUB_PROTO_IFATTACHDOMAIN, \ SI_ORDER_ANY); \ MODULE_VERSION(hname, version) #define KHELP_DECLARE_MOD_UMA(hname, hdata, hhooks, version, size, ctor, dtor) \ static struct khelp_modevent_data kmd_##hname = { \ .name = #hname, \ .helper = hdata, \ .hooks = hhooks, \ .nhooks = sizeof(hhooks) / sizeof(hhooks[0]), \ .uma_zsize = size, \ .umactor = ctor, \ .umadtor = dtor \ }; \ static moduledata_t h_##hname = { \ .name = #hname, \ .evhand = khelp_modevent, \ .priv = &kmd_##hname \ }; \ DECLARE_MODULE(hname, h_##hname, SI_SUB_PROTO_IFATTACHDOMAIN, \ SI_ORDER_ANY); \ MODULE_VERSION(hname, version) int khelp_modevent(module_t mod, int type, void *data); #endif /* _SYS_MODULE_KHELP_H_ */