[if_dwc] Add support for multi-descriptor packets in TX path

[gonzo]

Summary

Original if_dwc driver used m_defrag as an implementation shortcut but on 1000Mb
networks it affects performance. Implement multi-descriptor support for TX path.

Test Plan

Tested on RK3399-Firefly, patch adds ~15% of network throughput.

[manu]

Cool,
I've started something like that but it doesn't work yet and I don't think I'll have time to finish before 13.0
Give me a few days to review and test on other boards.

[ganbold]

Tested on NanoPI R4S.

root@nanopi-r4s:~ # iperf3 -c 192.168.1.8
Connecting to host 192.168.1.8, port 5201
[ 5] local 192.168.1.230 port 49248 connected to 192.168.1.8 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 77.6 MBytes 650 Mbits/sec 0 324 KBytes
[ 5] 1.00-2.00 sec 77.4 MBytes 650 Mbits/sec 0 324 KBytes
[ 5] 2.00-3.00 sec 77.9 MBytes 653 Mbits/sec 0 324 KBytes
[ 5] 3.00-4.00 sec 77.0 MBytes 647 Mbits/sec 0 324 KBytes
[ 5] 4.00-5.00 sec 75.5 MBytes 633 Mbits/sec 0 324 KBytes
[ 5] 5.00-6.00 sec 75.8 MBytes 636 Mbits/sec 0 324 KBytes
[ 5] 6.00-7.00 sec 76.6 MBytes 642 Mbits/sec 0 324 KBytes
[ 5] 7.00-8.00 sec 77.0 MBytes 647 Mbits/sec 0 324 KBytes
[ 5] 8.00-9.00 sec 77.4 MBytes 649 Mbits/sec 0 327 KBytes
[ 5] 9.00-10.00 sec 77.3 MBytes 649 Mbits/sec 0 327 KBytes

---

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 769 MBytes 645 Mbits/sec 0 sender
[ 5] 0.00-10.55 sec 769 MBytes 612 Mbits/sec receiver

iperf Done.
root@nanopi-r4s:~ # iperf3 -c 192.168.1.8 -R
Connecting to host 192.168.1.8, port 5201
Reverse mode, remote host 192.168.1.8 is sending
[ 5] local 192.168.1.230 port 28363 connected to 192.168.1.8 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 49.5 MBytes 415 Mbits/sec
[ 5] 1.00-2.00 sec 51.5 MBytes 432 Mbits/sec
[ 5] 2.00-3.00 sec 50.3 MBytes 422 Mbits/sec
[ 5] 3.00-4.00 sec 50.8 MBytes 427 Mbits/sec
[ 5] 4.00-5.00 sec 51.5 MBytes 432 Mbits/sec
[ 5] 5.00-6.00 sec 50.9 MBytes 427 Mbits/sec
[ 5] 6.00-7.00 sec 48.4 MBytes 406 Mbits/sec
[ 5] 7.00-8.00 sec 49.5 MBytes 415 Mbits/sec
[ 5] 8.00-9.00 sec 48.2 MBytes 404 Mbits/sec
[ 5] 9.00-10.00 sec 51.6 MBytes 433 Mbits/sec

---

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.55 sec 503 MBytes 400 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 502 MBytes 421 Mbits/sec receiver

iperf Done.
root@nanopi-r4s:~ # iperf3 -c 192.168.1.8 -u -b 900m
Connecting to host 192.168.1.8, port 5201
[ 5] local 192.168.1.230 port 17298 connected to 192.168.1.8 port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 50.5 MBytes 423 Mbits/sec 36242
[ 5] 1.00-2.00 sec 50.4 MBytes 422 Mbits/sec 36162
[ 5] 2.00-3.00 sec 50.4 MBytes 423 Mbits/sec 36178
[ 5] 3.00-4.00 sec 50.4 MBytes 423 Mbits/sec 36217
[ 5] 4.00-5.00 sec 50.4 MBytes 423 Mbits/sec 36195
[ 5] 5.00-6.00 sec 48.0 MBytes 403 Mbits/sec 34478
[ 5] 6.00-7.00 sec 42.2 MBytes 354 Mbits/sec 30306
[ 5] 7.00-8.00 sec 42.3 MBytes 355 Mbits/sec 30364
[ 5] 8.00-9.00 sec 42.4 MBytes 356 Mbits/sec 30459
[ 5] 9.00-10.00 sec 42.4 MBytes 356 Mbits/sec 30448

---

[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 469 MBytes 394 Mbits/sec 0.000 ms 0/337049 (0%) sender
[ 5] 0.00-10.03 sec 469 MBytes 393 Mbits/sec 0.040 ms 0/337048 (0%) receiver

iperf Done.
root@nanopi-r4s:~ # iperf3 -c 192.168.1.8 -u -b 900m -R
Connecting to host 192.168.1.8, port 5201
Reverse mode, remote host 192.168.1.8 is sending
[ 5] local 192.168.1.230 port 25924 connected to 192.168.1.8 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 83.9 MBytes 703 Mbits/sec 0.011 ms 16940/77172 (22%)
[ 5] 1.00-2.00 sec 90.3 MBytes 758 Mbits/sec 0.011 ms 12175/77051 (16%)
[ 5] 2.00-3.00 sec 84.7 MBytes 710 Mbits/sec 0.010 ms 16446/77259 (21%)
[ 5] 3.00-4.00 sec 96.3 MBytes 808 Mbits/sec 0.009 ms 7885/77057 (10%)
[ 5] 4.00-5.00 sec 95.7 MBytes 802 Mbits/sec 0.037 ms 8133/76839 (11%)
[ 5] 5.00-6.00 sec 83.5 MBytes 701 Mbits/sec 0.010 ms 17067/77068 (22%)
[ 5] 6.00-7.00 sec 86.6 MBytes 727 Mbits/sec 0.050 ms 15057/77267 (19%)
[ 5] 7.00-8.00 sec 96.3 MBytes 808 Mbits/sec 0.010 ms 7806/77001 (10%)
[ 5] 8.00-9.00 sec 96.4 MBytes 809 Mbits/sec 0.034 ms 7898/77121 (10%)
[ 5] 9.00-10.00 sec 96.3 MBytes 808 Mbits/sec 0.009 ms 7874/77064 (10%)

---

[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.02 sec 1.05 GBytes 900 Mbits/sec 0.000 ms 0/772035 (0%) sender
[ 5] 0.00-10.00 sec 910 MBytes 763 Mbits/sec 0.009 ms 117281/770899 (15%) receiver

iperf Done.

[gonzo]

Committed to main: https://reviews.freebsd.org/R10:e523262107865130e40fb19f7c3c571c8dd0b252