Change Details

The sample tests assume you are already in ngctl(8). This is a better formatted version of the testing.txt in original bug report ([[ https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278130 | 278130 ]]). First we need to create a bridge to use. We also want it to not shutdown when all hooks are disconnected since we will disconnect the control socket. Notice we are using the old `link0` here. It could just be `link` now. ``` mkpeer bridge b link0 name .:b br0 msg br0: setpersistent rmhook br0: link0 ``` This looks like you need `link0`, you do not. We could have used `rmhook .: b` instead. This is why you don't need `link` numbers, you can remove them from either side. Next that the bridge has no connections now (could also use `ls -l` but moving forward this limits output). ``` show br0: Name: br0 Type: bridge ID: 0000000f Num hooks: 0 ``` Now lets make an ng_eiface(4), these can be given to VNET jails although you will need to give it a MAC address for that which I don't show. Also connect the ng_eiface(4) to the ng_bridge(4) using only `link` but we will show that later. ``` mkpeer eiface e ether name .:e jail0 rmhook .: e connect br0: jail0: link ether ``` Show the ng_bridge(4) again, it should have picked `link0` as that is lowest available number: ``` show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 jail0 eiface 0000000b ether ``` Repeat for jail1 and jail2 so we have 3 ng_eiface(4) attached to our ng_bridge(4), this time specify an unused `link1` for the first (remember we are //mostly// retaining backward compatibility): ``` mkpeer eiface e ether name .:e jail1 rmhook jail1: ether connect jail1: br0: ether link1 mkpeer eiface e ether name .:e jail2 rmhook jail2: ether connect jail2: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link1 jail1 eiface 0000000c ether link0 jail0 eiface 0000000b ether ``` Next let's remove `jail1`, which we can see has `link1` but we don't need that information: ``` rmhook jail1: ether shutdown jail1: show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Lets make `jail3` now and first try connecting to `link2` which will fail, then let it auto assign: ``` mkpeer eiface e ether name .:e jail3 rmhook jail3: ether connect jail3: br0: ether link2 ngctl: send msg: File exists # <-- error returned from ngctl connect jail3: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link1 jail3 eiface 0000000e ether link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Hooks are unsorted linked list which is why we have `link0`, `link2`, and `link1` for local hooks. bhyve(8) uses ng_socket(4) but its create for you in the command. As long as you use `path=br0:` bhyve(8) will connect to this ng_bridge(4). Here is what I use ``` sh /usr/share/examples/bhyve/vmrun.sh -c 2 -m 4096M -t netgraph,path=bridge-lan: -t netgraph,path=bridge-jail: -d /dev/zvol/data/vmm/fbsd14 fbsd14 ``` Where `bridge-lan` is an ng_bridge(4) that makes use of an ng_ether(4), specifically my `re0` so that is has external and internal networking. Although I should mention this is flaky even with `uplink`. The releng/14 `re0` driver works great and host and bhyve guest can ping each other. The `stable/14` requires the `realtek-re-kmod` port and you can't ping between host and guest(s) or even one guest and another. But this is why I make one bridge with an ethernet and one purely internal.

The sample tests assume you are already in ngctl(8). This is a better formatted version of the testing.txt in original bug report ([[ https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278130 | 278130 ]]). First we need to create a bridge to use. We also want it to not shutdown when all hooks are disconnected since we will disconnect the control socket. Notice we are using the old `link0` here. It could just be `link` now. ``` mkpeer bridge b link0 name .:b br0 msg br0: setpersistent rmhook br0: link0 ``` This looks like you need `link0`, you do not. We could have used `rmhook .: b` instead. This is why you don't need `link` numbers, you can remove them from either side. Next show that the bridge currently has no connections (could also use `ls -l` but moving forward this limits output). ``` show br0: Name: br0 Type: bridge ID: 0000000f Num hooks: 0 ``` Now lets make an ng_eiface(4), these can be given to VNET jails although you will need to give it a MAC address for that which I don't show. Also connect the ng_eiface(4) to the ng_bridge(4) using only `link`. ``` mkpeer eiface e ether name .:e jail0 rmhook .: e connect br0: jail0: link ether ``` Show the ng_bridge(4) again, it should have picked `link0` as that is lowest available number: ``` show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 jail0 eiface 0000000b ether ``` Repeat for jail1 and jail2 so we have 3 ng_eiface(4) attached to our ng_bridge(4), this time specify an unused `link1` for the first (remember we are //mostly// retaining backward compatibility): ``` mkpeer eiface e ether name .:e jail1 rmhook jail1: ether connect jail1: br0: ether link1 mkpeer eiface e ether name .:e jail2 rmhook jail2: ether connect jail2: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link1 jail1 eiface 0000000c ether link0 jail0 eiface 0000000b ether ``` Next let's remove `jail1`, which we can see has `link1` but we don't need that information: ``` rmhook jail1: ether shutdown jail1: show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Lets make `jail3` now and first try connecting to `link2` which will fail, then let it auto assign: ``` mkpeer eiface e ether name .:e jail3 rmhook jail3: ether connect jail3: br0: ether link2 ngctl: send msg: File exists # <-- error returned from ngctl connect jail3: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link1 jail3 eiface 0000000e ether link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Hooks are unsorted linked list which is why we have `link0`, `link2`, and `link1` for local hooks. bhyve(8) uses ng_socket(4) but its create for you in the command. As long as you use `path=br0:` bhyve(8) will connect to this ng_bridge(4). Here is what I use ``` sh /usr/share/examples/bhyve/vmrun.sh -c 2 -m 4096M -t netgraph,path=bridge-lan: -t netgraph,path=bridge-jail: -d /dev/zvol/data/vmm/fbsd14 fbsd14 ``` Where `bridge-lan` is an ng_bridge(4) that makes use of an ng_ether(4), specifically my `re0` so that is has external and internal networking. Although I should mention this is flaky even with `uplink`. The releng/14 `re0` driver works great and host and bhyve guest can ping each other. The `stable/14` requires the `realtek-re-kmod` port and you can't ping between host and guest(s) or even one guest and another. But this is why I make one bridge with an ethernet and one purely internal.

The sample tests assume you are already in ngctl(8). This is a better formatted version of the testing.txt in original bug report ([[ https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278130 | 278130 ]]). First we need to create a bridge to use. We also want it to not shutdown when all hooks are disconnected since we will disconnect the control socket. Notice we are using the old `link0` here. It could just be `link` now. ``` mkpeer bridge b link0 name .:b br0 msg br0: setpersistent rmhook br0: link0 ``` This looks like you need `link0`, you do not. We could have used `rmhook .: b` instead. This is why you don't need `link` numbers, you can remove them from either side. Next show that the bridgee currently has no connections now (could also use `ls -l` but moving forward this limits output). ``` show br0: Name: br0 Type: bridge ID: 0000000f Num hooks: 0 ``` Now lets make an ng_eiface(4), these can be given to VNET jails although you will need to give it a MAC address for that which I don't show. Also connect the ng_eiface(4) to the ng_bridge(4) using only `link` but we will show that later. ``` mkpeer eiface e ether name .:e jail0 rmhook .: e connect br0: jail0: link ether ``` Show the ng_bridge(4) again, it should have picked `link0` as that is lowest available number: ``` show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 jail0 eiface 0000000b ether ``` Repeat for jail1 and jail2 so we have 3 ng_eiface(4) attached to our ng_bridge(4), this time specify an unused `link1` for the first (remember we are //mostly// retaining backward compatibility): ``` mkpeer eiface e ether name .:e jail1 rmhook jail1: ether connect jail1: br0: ether link1 mkpeer eiface e ether name .:e jail2 rmhook jail2: ether connect jail2: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link1 jail1 eiface 0000000c ether link0 jail0 eiface 0000000b ether ``` Next let's remove `jail1`, which we can see has `link1` but we don't need that information: ``` rmhook jail1: ether shutdown jail1: show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Lets make `jail3` now and first try connecting to `link2` which will fail, then let it auto assign: ``` mkpeer eiface e ether name .:e jail3 rmhook jail3: ether connect jail3: br0: ether link2 ngctl: send msg: File exists # <-- error returned from ngctl connect jail3: br0: ether link show br0: Name: br0 Type: bridge ID: 0000000a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link1 jail3 eiface 0000000e ether link2 jail2 eiface 0000000d ether link0 jail0 eiface 0000000b ether ``` Hooks are unsorted linked list which is why we have `link0`, `link2`, and `link1` for local hooks. bhyve(8) uses ng_socket(4) but its create for you in the command. As long as you use `path=br0:` bhyve(8) will connect to this ng_bridge(4). Here is what I use ``` sh /usr/share/examples/bhyve/vmrun.sh -c 2 -m 4096M -t netgraph,path=bridge-lan: -t netgraph,path=bridge-jail: -d /dev/zvol/data/vmm/fbsd14 fbsd14 ``` Where `bridge-lan` is an ng_bridge(4) that makes use of an ng_ether(4), specifically my `re0` so that is has external and internal networking. Although I should mention this is flaky even with `uplink`. The releng/14 `re0` driver works great and host and bhyve guest can ping each other. The `stable/14` requires the `realtek-re-kmod` port and you can't ping between host and guest(s) or even one guest and another. But this is why I make one bridge with an ethernet and one purely internal.