mlx5ib: Support RAW Ethernet when RoCE is disabled in mlx5ib(4)

On some environments, such as certain SRIOV VF configurations, RoCE is
not supported for mlx5 Ethernet ports. Currently, the driver will not
open IB device on that port.

This is problematic, since we do want user-space RAW Ethernet (RAW_PACKET
QPs) functionality to remain in place. For that end, enhance the relevant
driver flows such that we do create a device instance in that case.

Linux commit:
ca5b91d63192ceaa41a6145f8c923debb64c71fa

Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 626cb01d4439b85eba9632a7738cddc64b6ad42d)