Page MenuHomeFreeBSD

fwget: rewrite in lua, attempt #1
Needs ReviewPublic

Authored by bz on Sep 1 2025, 6:56 PM.
Tags
None
Referenced Files
F153364182: D52325.diff
Mon, Apr 20, 5:45 PM
Unknown Object (File)
Tue, Apr 14, 12:56 PM
Unknown Object (File)
Mon, Apr 13, 7:13 AM
Unknown Object (File)
Sun, Apr 12, 9:46 PM
Unknown Object (File)
Tue, Apr 7, 3:24 AM
Unknown Object (File)
Mon, Apr 6, 6:05 AM
Unknown Object (File)
Mon, Mar 30, 1:23 PM
Unknown Object (File)
Sun, Mar 29, 4:22 PM
This revision needs review, but there are no reviewers specified.

Details

Reviewers
None
Summary

For now it tries to match -n -q -v behaviour from the shell script
version where possible. -v has minor alterations given we no longer
have individual shell scripts.

Added -D directory which is helpful while working on fwget to point it
to where to find the firmware table files given I was tired of env.

Added -d which outputs debug warnings which would otherwise be silent
like overlapping/duplicate firmware table entries (some may be false
positives); no primary key found; invalid lines, duplicate selectors, ...

The firmware table input files are modelled after the output of their
tool (pciconf -l or usbconfig -l dump_device_desc) and use the same
selectors. The "vendor id" is set to tbe the mandatory selector for
both busses. The firmware file(s) information is appended.

Selection starts on the entries with the most selectors (more
specific if I got the iterator right).

For now I kept the firmware tables split into individual files as
that may help maintainance (especially if auto-generated).

I should also help to sort certain things initially (now):

(a) pci wireless devices do not match subclass, and qca and mediatek

need individual checks for class as they use "old" or "misc" as
class for some (all?) chipsets.

(b) for pci video I currently modelled the files after the shell

scripts replacing globbing with masks BUT I also added the individual
records I extracted from the drivers.  There are some which I do not
fully understand how they fit together and others in the Intel case
which are entirely unhandled.  I left them for now to be clarified
or cleaned up.  Someone should look over them.  if we fix the current
shell version I'll try to keep this updated too.
The masked device entries likely should vanish at the end?

(c) The real motiviation for cleaning it up was so that I could add the

USB Bluetooth bits for Intel and Realtek and prepared qca and brcm
but the latter two need ports work first.
Test Plan

I run it across some of my systems (one including a lot of
wifi devices) before some last-minute changes.

I tried various command line combinations and tried to catch
bits but hey...

The only hard coded consumer in the tree seems to be the installer.

I am just putting this out here so that it's out.

The lua script or the formats chosen may be utterly useless.

I am very well aware of optimizations on how to build the
internal tables and do way more effective matching at the
end of the weekend but before I (or someone else) go(es)
there, let's have a look.

This is my first own lua scrpit ever and it's written like
perl but it was a fun experience beginning to learn yet
another language.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 66745
Build 63628: arc lint + arc unit

Event Timeline

bz requested review of this revision.Sep 1 2025, 6:56 PM

Oh and I have not style.9 cleaned it up yet. Where's the fun it that... So please don't go after that just yet.

I believe I'll add the BT bits in as a shell version for 15.0 as this is likely 16-CURRENT first?

usr.sbin/fwget/pci/pci_video_intel
143

Here's the marker from https://github.com/freebsd/freebsd-src/pull/1702 which I got pointed at today which seem to come to a similar conclusion.

The patch does not apply cleanly on main:

$ git arc patch D52325
Checking patch usr.sbin/fwget/usb/usb_wireless_ralink...
Checking patch usr.sbin/fwget/usb/usb_wireles_header...
Checking patch usr.sbin/fwget/usb/usb_ralink...
Checking patch usr.sbin/fwget/usb/usb_bluetooth_realtek...
Checking patch usr.sbin/fwget/usb/usb_bluetooth_qca...
Checking patch usr.sbin/fwget/usb/usb_bluetooth_intel...
Checking patch usr.sbin/fwget/usb/usb_bluetooth_header...
Checking patch usr.sbin/fwget/usb/usb_bluetooth_brcm...
Checking patch usr.sbin/fwget/usb/usb...
Checking patch usr.sbin/fwget/usb/Makefile...
Checking patch usr.sbin/fwget/pci/pci_wireless_realtek...
Checking patch usr.sbin/fwget/pci/pci_wireless_qca...
Checking patch usr.sbin/fwget/pci/pci_wireless_mediatek...
Checking patch usr.sbin/fwget/pci/pci_wireless_intel...
Checking patch usr.sbin/fwget/pci/pci_wireless_header...
Checking patch usr.sbin/fwget/pci/pci_video_intel...
Checking patch usr.sbin/fwget/pci/pci_video_header...
Checking patch usr.sbin/fwget/pci/pci_video_amd...
error: while searching for:
#-
# SPDX-License-Identifier: BSD-2-Clause
#
# Copyright 2023 Beckhoff Automation GmbH & Co. KG
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted providing that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

pci_video_amd()
{
	# A table listing the required firmware versions for each GPU series
	# is available at https://docs.kernel.org/gpu/amdgpu/driver-misc.html

	case "$1" in
		0x678*|0x679*)
			addpkg "gpu-firmware-amd-kmod-tahiti"
			;;
		0x680*|0x681*)
			addpkg "gpu-firmware-amd-kmod-pitcairn"
			;;
		0x660*|0x661*|0x662*|0x663*)
			addpkg "gpu-firmware-amd-kmod-oland"
			;;
		0x682*|0x683*)
			addpkg "gpu-firmware-amd-kmod-verde gpu-firmware-amd-kmod-si58"
			;;
		0x666*)
			addpkg "gpu-firmware-amd-kmod-hainan"
			;;
		0x13*)
			addpkg "gpu-firmware-amd-kmod-kaveri"
			;;
		0x664*|0x665*)
			addpkg "gpu-firmware-amd-kmod-bonaire"
			;;
		0x67a*|0x67b*)
			addpkg "gpu-firmware-amd-kmod-hawaii"
			;;
		0x983*)
			addpkg "gpu-firmware-amd-kmod-kabini"
			;;
		0x985*)
			addpkg "gpu-firmware-amd-kmod-mullins"
			;;
		0x690*)
			addpkg "gpu-firmware-amd-kmod-topaz"
			;;
		0x692*|0x693*)
			addpkg "gpu-firmware-amd-kmod-tonga"
			;;
		0x730*)
			addpkg "gpu-firmware-amd-kmod-fiji"
			;;
		0x987*)
			addpkg "gpu-firmware-amd-kmod-carrizo"
			;;
		0x98e4)
			addpkg "gpu-firmware-amd-kmod-stoney"
			;;
		0x67e*|0x67ff)
			addpkg "gpu-firmware-amd-kmod-polaris11"
			;;
		0x67c*|0x67d*|0x6fdf)
			addpkg "gpu-firmware-amd-kmod-polaris10"
			;;
		0x698*|0x699*)
			addpkg "gpu-firmware-amd-kmod-polaris12"
			;;
		0x694*)
			addpkg "gpu-firmware-amd-kmod-vegam"
			;;
		0x686*|0x687*)
			addpkg "gpu-firmware-amd-kmod-vega10"
			;;
		0x69a*)
			addpkg "gpu-firmware-amd-kmod-vega12"
			;;
		0x66a*)
			addpkg "gpu-firmware-amd-kmod-vega20"
			;;
		0x15d8)
			addpkg "gpu-firmware-amd-kmod-picasso"
			;;
		0x15d*)
			addpkg "gpu-firmware-amd-kmod-raven"
			;;
		0x738*|0x739*)
			addpkg "gpu-firmware-amd-kmod-arcturus"
			;;
		0x731*)
			addpkg "gpu-firmware-amd-kmod-navi10"
			;;
		0x734*)
			addpkg "gpu-firmware-amd-kmod-navi14"
			;;
		0x15e7|0x1638)
			addpkg "gpu-firmware-amd-kmod-renoir gpu-firmware-amd-kmod-green-sardine"
			;;
		0x1636|0x164c)
			addpkg "gpu-firmware-amd-kmod-renoir"
			;;
		0x736*)
			addpkg "gpu-firmware-amd-kmod-navi12"
			;;
		0x73a*|0x73b*)
			addpkg "gpu-firmware-amd-kmod-sienna-cichlid"
			;;
		0x163f)
			addpkg "gpu-firmware-amd-kmod-vangogh"
			;;
		0x164d|0x1681)
			addpkg "gpu-firmware-amd-kmod-yellow-carp"
			;;
		0x73c*|0x73d*)
			addpkg "gpu-firmware-amd-kmod-navy-flounder"
			;;
		0x73e*|0x73f*)
			addpkg "gpu-firmware-amd-kmod-dimgrey-cavefish"
			;;
		0x740*|0x741*)
			addpkg "gpu-firmware-amd-kmod-aldebaran"
			;;
		0x13fe|0x143f)
			addpkg "gpu-firmware-amd-kmod-cyan-skillfish2"
			;;
		0x742*|0x743*)
			addpkg "gpu-firmware-amd-kmod-beige-goby"
			;;
		0x744c)
			addpkg "gpu-firmware-amd-kmod-dcn-3-2-0"
			addpkg "gpu-firmware-amd-kmod-gc-11-0-0"
			addpkg "
error: patch failed: usr.sbin/fwget/pci/pci_video_amd:1
Checking patch usr.sbin/fwget/pci/pci_network_realtek...
Checking patch usr.sbin/fwget/pci/pci_network_qca...
Checking patch usr.sbin/fwget/pci/pci_network_mediatek...
error: while searching for:
#-
# SPDX-License-Identifier: BSD-2-Clause
#
# Copyright 2023 Bjoern A. Zeeb
# Copyright (c) 2024-2025 The FreeBSD Foundation
#
# Portions of this software were developed by Björn Zeeb
# under sponsorship from the FreeBSD Foundation.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#

pci_network_mediatek_mt76()
{

	case "$1" in
	# The case pattern table below is generated by a script also helping
	# to generate the flavor/firmware Makefile definitions for ports.
	# { sys/contrib/dev/mediatek/mt76/zzz_fw_ports_fwget.sh }
	### >>>

	0x0608)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x0616)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x0717)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x7611)	addpkg "wifi-firmware-mediatek-kmod-mt7615"; return 1 ;;
	0x7615)	addpkg "wifi-firmware-mediatek-kmod-mt7615"; return 1 ;;
	0x7663)	addpkg "wifi-firmware-mediatek-kmod-mt7615"; return 1 ;;
	0x7906)	addpkg "wifi-firmware-mediatek-kmod-mt7915"; return 1 ;;
	0x790a)	addpkg "wifi-firmware-mediatek-kmod-mt7915"; return 1 ;;
	0x7915)	addpkg "wifi-firmware-mediatek-kmod-mt7915"; return 1 ;;
	0x7916)	addpkg "wifi-firmware-mediatek-kmod-mt7915"; return 1 ;;
	0x7920)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x7922)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x7925)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x7961)	addpkg "wifi-firmware-mediatek-kmod-mt792x"; return 1 ;;
	0x7990)	addpkg "wifi-firmware-mediatek-kmod-mt7996"; return 1 ;;
	0x7991)	addpkg "wifi-firmware-mediatek-kmod-mt7996"; return 1 ;;
	0x7992)	addpkg "wifi-firmware-mediatek-kmod-mt7996"; return 1 ;;
	0x799a)	addpkg "wifi-firmware-mediatek-kmod-mt7996"; return 1 ;;

	### <<<
	esac
}

pci_network_mediatek()
{

	for _drv in mt76; do
		pci_network_mediatek_${_drv} "$1"
		case $? in
		1)	break ;;
		esac
	done
}

error: patch failed: usr.sbin/fwget/pci/pci_network_mediatek:1
error: removal patch leaves file contents
error: usr.sbin/fwget/pci/pci_network_mediatek: patch does not apply
Checking patch usr.sbin/fwget/pci/pci_network_intel...
Checking patch usr.sbin/fwget/pci/pci...
Checking patch usr.sbin/fwget/pci/Makefile.depend...
Checking patch usr.sbin/fwget/pci/Makefile...
Checking patch usr.sbin/fwget/fwget.sh...
Checking patch usr.sbin/fwget/fwget.lua...
Checking patch usr.sbin/fwget/fwget.8...
Checking patch usr.sbin/fwget/Makefile...
Checking patch ObsoleteFiles.inc...
error: while searching for:
#   xargs -n1 | sort | uniq -d;
# done

# 20250826: Remove a misspelled manual
OLD_FILES+=usr/share/man/man3/sysdecode_syscallnames.3.gz


error: patch failed: ObsoleteFiles.inc:51
Applied patch usr.sbin/fwget/usb/usb_wireless_ralink cleanly.
Applied patch usr.sbin/fwget/usb/usb_wireles_header cleanly.
Applied patch usr.sbin/fwget/usb/usb_ralink cleanly.
Applied patch usr.sbin/fwget/usb/usb_bluetooth_realtek cleanly.
Applied patch usr.sbin/fwget/usb/usb_bluetooth_qca cleanly.
Applied patch usr.sbin/fwget/usb/usb_bluetooth_intel cleanly.
Applied patch usr.sbin/fwget/usb/usb_bluetooth_header cleanly.
Applied patch usr.sbin/fwget/usb/usb_bluetooth_brcm cleanly.
Applied patch usr.sbin/fwget/usb/usb cleanly.
Applied patch usr.sbin/fwget/usb/Makefile cleanly.
Applied patch usr.sbin/fwget/pci/pci_wireless_realtek cleanly.
Applied patch usr.sbin/fwget/pci/pci_wireless_qca cleanly.
Applied patch usr.sbin/fwget/pci/pci_wireless_mediatek cleanly.
Applied patch usr.sbin/fwget/pci/pci_wireless_intel cleanly.
Applied patch usr.sbin/fwget/pci/pci_wireless_header cleanly.
Applied patch usr.sbin/fwget/pci/pci_video_intel cleanly.
Applied patch usr.sbin/fwget/pci/pci_video_header cleanly.
Applying patch usr.sbin/fwget/pci/pci_video_amd with 1 reject...
Rejected hunk #1.
Applied patch usr.sbin/fwget/pci/pci_network_realtek cleanly.
Applied patch usr.sbin/fwget/pci/pci_network_qca cleanly.
Applied patch usr.sbin/fwget/pci/pci_network_intel cleanly.
Applied patch usr.sbin/fwget/pci/pci cleanly.
Applied patch usr.sbin/fwget/pci/Makefile.depend cleanly.
Applied patch usr.sbin/fwget/pci/Makefile cleanly.
Applied patch usr.sbin/fwget/fwget.sh cleanly.
Applied patch usr.sbin/fwget/fwget.lua cleanly.
Applied patch usr.sbin/fwget/fwget.8 cleanly.
Applied patch usr.sbin/fwget/Makefile cleanly.
Applying patch ObsoleteFiles.inc with 1 reject...
Rejected hunk #1.

 Patch Failed!
Usage Exception: Unable to apply patch!

The patch does not apply cleanly on main:

Yes, it's 7 months old and didn't get much attention.
As said in email, I should rewrite some things and then really push to get it in.

In D52325#1291455, @bz wrote:

The patch does not apply cleanly on main:

Yes, it's 7 months old and didn't get much attention.
As said in email, I should rewrite some things and then really push to get it in.

Let me know if I can be of any help. I think especially the bluetooth bits should go in.