ixl: Permit 802.1ad frames to pass though the chip

[donner]

Summary

Intel X710 chips have an internal (on-chip) switch for various SDN purposes.
The internal switch used 0x88a8 to tag frames within the chip itself.
In order to keep the internal switch protected, the chip silently drops all packets with the currently used internal ethertype.

The ixl driver does already contain code to set the internally used ethertype, but this function is not called in any place.

This patch is a quick hack to change the internal switch_tag within the chip. It's only published to move things forward, to help people with run into the same problem (processing 802.1ad, or double tagged frames).

I'm unsure how move this into the normal ixl driver, but do not want to postphone this solution for a real world problem.

Test Plan

# tcpdump -eni ixl0 -c 2

nothing happens, no counters going up to tell about any incoming frame.

# sysctl -d dev.ixl.0.debug.switch_vlans
dev.ixl.0.debug.switch_vlans: HW Switch VLAN Configuration
# sysctl dev.ixl.0.debug.switch_vlans=0x9200
# dmesg | tail -1
ixl0: Setting switch config to switch_tag=9200, first_tag=0000, second_tag=0000
# tcpdump -eni ixl0 -c 2
23:49:16.356591 dc:39:6f:0a:10:78 > a0:23:9f:37:29:3f, ethertype 802.1Q-QinQ (0x88a8), length 78: vlan 2, p 0, ethertype 802.1Q, vlan 140, p 0, ethertype IPv4, 91.137.26.17 > 2.207.28.131: ICMP host 91.137.26.17 unreachable - admin prohibited filter, length 36
23:49:17.129581 00:1d:aa:d8:3d:99 > 00:07:b4:00:78:01, ethertype 802.1Q-QinQ (0x88a8), length 93: vlan 2, p 0, ethertype 802.1Q, vlan 140, p 0, ethertype IPv4, 5.102.161.204.33305 > 224.0.0.251.5353: 27179 PTR (QM)? 192.168.10.3.in-addr.arpa. (43)

[donner]

Patch does work with 12-STABLE, too. (removing the NEEDGIANT flag)

[donner]

Rebase to r366170.

[kp]

There are a few other minor style(9) nits, but it's entirely consistent with the rest of the file, so let's not argue about those.

Give Philip a day or two to take a look as well, he likes network drivers, but then I think it can go in.

It may be useful to document this sysctl in the ixl man page as well, if only to give users who run into this problem a fighting chance of figuring out that they can fix it.

sys/dev/ixl/ixl_pf_main.c

3519

return (status);.

[donner]

• fixing style(9)
• adding documentation for the hack

[philip]

I am okay with this in principle, modulo the documentation nitpicking, but I wonder if it wouldn't be better to change the *default* to something more sensible -- i.e., have 802.1ad work out of the box. One of the "local experimentation" types come to mind. Or set it to one of the types assigned to Intel!

http://standards-oui.ieee.org/ethertype/eth.txt

What do other operating systems do?

share/man/man4/ixl.4

235

I would suggest wording like "if you need 802.1ad support, set this to another ethertype".
Note that 0xffff is not "unused" it's allocated to some entity.

Perhaps suggest 0x88b5 or 0x88b6 instead. These are reserved for "local experimentation" in IEEE 802.

[brueffer]

share/man/man4/ixl.4

232

mdoc style: a new sentence should start on a new line.

233

This should probably be

Defaults to
.Dv 0x88a8 ,
which...

234

mdoc style: a new sentence should start on a new line.

235

Same as above, .Dv 0xffff

[philip]

Speaking nitpicking: I would also consistently capitalise Ethertype and VLAN as the IEEE does. :)

[donner]

In D24179#631321, @philip wrote:

I am okay with this in principle, modulo the documentation nitpicking, but I wonder if it wouldn't be better to change the *default* to something more sensible -- i.e., have 802.1ad work out of the box. One of the "local experimentation" types come to mind. Or set it to one of the types assigned to Intel!

http://standards-oui.ieee.org/ethertype/eth.txt

What do other operating systems do?

Linux is setting it to 0xffff unconditionally.

But I do not want to change this without discussion. The first step is to enable the interface at all. Then somebody can play with it. The internal bridge of the chip is most useful for virtualization, it can represent a vlan (or double tagged vlan) as an "physical" interface suited to be plugged into a VM using IO-VMM.

BTW: Linux is not exposing the interface. Hyper-V seems to use it actively.

[donner]

share/man/man4/ixl.4

235

Everybody out there seems to set it to 0xffff. Which should be the default in FreeBSD, too (sometimes later)

[donner]

• Fix man page style.

[kp]

Approved by: kp (mentor)

As you say, we can change the default later. We probably should, but it also makes sense to do that in a separate commit.

[brueffer]

share/man/man4/ixl.4

236

Just another minor nitpick, I think we capitalize LAN and VLAN everywhere. Looks good otherwise, thanks! Please bump the document date before you commit.

[donner]

• Bump document date and fix capitalization