Paths

Table of Contentst

snd_hda: Implement automatic redirection between associations
Needs ReviewPublic
Actions

Authored by christos on Apr 29 2025, 1:13 PM.

Details

Reviewers

markj
emaste
dev_submerge.ch
mav
wulf
adrian

Summary

For audio to be redirected to the headphones/headset after plugging the
jack, or back to the speaker/internal mic when unplugging it, the
speaker and headphone pins need to be part of the same association
(i.e., the same PCM device). This patch makes it possible to redirect
audio even between different associations, which can reduce the need for
manual pin patching.

The idea is that we issue a devctl_notify() from within the jack
detection callback whenever a jack is (un-)plugged to redirect audio to
the appropriate device. Then the hda.conf devd script is responsible for
using virtual_oss to change the playback/recording device to whatever
snd_hda(4) selected. The reason for requiring virtual_oss is that it has
hot-swapping support, which is necessary for jack redirection.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Skipped

Unit

Tests Skipped

Build Status

Buildable 64373
Build 61257: arc lint + arc unit

Event Timeline

christos created this revision.Apr 29 2025, 1:13 PM

Herald added subscribers: ziaee, asomers, imp. · View Herald TranscriptApr 29 2025, 1:13 PM

christos requested review of this revision.Apr 29 2025, 1:13 PM

Harbormaster completed remote builds in B63775: Diff 154467.Apr 29 2025, 1:13 PM

This is a WIP and needs more testing.

A few things I know already need to be addressed:

We are currently choosing the first association that meets our search criteria. What if there are multiple such associations? I haven't had the chance to test this yet.
Should we touch hw.snd.default_unit? The answer is not obvious, since we're switching either the playback or the recording device, not both, and hw.snd.default_unit affects both.
The devd script relies on virtual_oss running with vdsp.ctl as the control device, because this is usually the default name people choose.

I'm also planning to port virtual_oss to base in the near future.

christos added inline comments.Apr 29 2025, 1:18 PM

sys/dev/sound/pci/hda/hdaa.c
6231	We're creating the PCM devices first so that the new code in `hdaa_presence_handler()` doesn't panic when trying to access `pdevinfo` in case it runs.

christos mentioned this in D49216: sound: Implement /dev/dsp as a router device.Apr 29 2025, 1:19 PM

meka_tilda.center added a subscriber: meka_tilda.center.Apr 29 2025, 1:43 PM

Rebase with 02d61f27585f52d422fc1e235ac6226e27145162 applied.

Harbormaster completed remote builds in B63807: Diff 154558.Apr 30 2025, 5:04 PM

christos added inline comments.May 5 2025, 9:29 AM

sbin/devd/hda.conf
3 ↗	(On Diff #154558)	Maybe it would make more sense to make this is a more generic "SND" system in case we want to extend this functionality outside snd_hda(4)?

mav added inline comments.May 5 2025, 5:50 PM

sys/dev/sound/pci/hda/hdaa.c
579	So if we have several associations of that direction, then disconnect will be produce the event for each of them other then the disconnected one? On the opposite side, what if the playback device we want to redirect to is on different HDA codec/function, for example on HDMI video card? Also this approach will work only for HDA devices, but not others, like USB. I don't know whether virtual_oss already has some logic for that case, but otherwise it looks like a very partial solution. And if the redirection decision is made by vitrual_oss, then to support more than two devices it would make sense to send just connect/disconnect events, rather than more (too much more) specific redirect.

christos added inline comments.May 5 2025, 6:47 PM

sys/dev/sound/pci/hda/hdaa.c
579	So if we have several associations of that direction, then disconnect will be produce the event for each of them other then the disconnected one? As it is currently, it will just switch to the first association that satisfies this condition and then `break`. It's mentioned in https://reviews.freebsd.org/D50070#1141951. On the opposite side, what if the playback device we want to redirect to is on different HDA codec/function, for example on HDMI video card? Do we want by default to auto-switch to the HDMI card if the cable is plugged? Also this approach will work only for HDA devices, but not others, like USB. I don't know whether virtual_oss already has some logic for that case, but otherwise it looks like a very partial solution. While I already mention in an inline comment above that we could generalize this outside HDA (i.e., using devd signals to hot-swap using virtual_oss), even in that case, HDA is what definitely needs a solution like this in order to avoid manually patching the code frequently. Is there a similar problem with USB? And if the redirection decision is made by vitrual_oss, then to support more than two devices it would make sense to send just connect/disconnect events, rather than more (too much more) specific redirect. I am not sure I follow. What do you mean?

mav added inline comments.May 5 2025, 6:58 PM

sys/dev/sound/pci/hda/hdaa.c
579	On the opposite side, what if the playback device we want to redirect to is on different HDA codec/function, for example on HDMI video card? Do we want by default to auto-switch to the HDMI card if the cable is plugged? More of opposite, to switch from HDMI to headphones on connection and back on disconnection. Also this approach will work only for HDA devices, but not others, like USB. I don't know whether virtual_oss already has some logic for that case, but otherwise it looks like a very partial solution. While I already mention in an inline comment above that we could generalize this outside HDA (i.e., using devd signals to hot-swap using virtual_oss), even in that case, HDA is what definitely needs a solution like this in order to avoid manually patching the code frequently. Is there a similar problem with USB? I don't know whether USB devices support pin sensing, sure there are some USB sounds cards with jacks, but if they do, it would require the same. If not, then just a connection of new USB headphones would be a good point to redirect there from HDMI monitor. And if the redirection decision is made by vitrual_oss, then to support more than two devices it would make sense to send just connect/disconnect events, rather than more (too much more) specific redirect. I am not sure I follow. What do you mean? I mean devd events should be like "dsp0 playback is connected to headphones, consider doing something", not specifically "redirect playback to dsp0".

guest-patmaddox added a subscriber: guest-patmaddox.May 6 2025, 2:29 AM

agh_riseup.net added a subscriber: agh_riseup.net.May 6 2025, 7:57 AM

christos added inline comments.May 8 2025, 3:00 PM

sys/dev/sound/pci/hda/hdaa.c
579	Do we want by default to auto-switch to the HDMI card if the cable is plugged? More of opposite, to switch from HDMI to headphones on connection and back on disconnection. So for headphone connection, the patch should work as expected because during connection we switch to the headphones anyway. The inverse however isn't true. Do you think it's worth the complexity we'll introduce if we do all that through sound(4) instead? I'm not completely sure to be honest. While I already mention in an inline comment above that we could generalize this outside HDA (i.e., using devd signals to hot-swap using virtual_oss), even in that case, HDA is what definitely needs a solution like this in order to avoid manually patching the code frequently. Is there a similar problem with USB? I don't know whether USB devices support pin sensing, sure there are some USB sounds cards with jacks, but if they do, it would require the same. If not, then just a connection of new USB headphones would be a good point to redirect there from HDMI monitor. USB audio doesn't seem to do jack sensing, so I don't think that should be a concern. I am not sure I follow. What do you mean? I mean devd events should be like "dsp0 playback is connected to headphones, consider doing something", not specifically "redirect playback to dsp0". No objection to that.

mav added inline comments.May 8 2025, 3:17 PM

sys/dev/sound/pci/hda/hdaa.c
579	Do we want by default to auto-switch to the HDMI card if the cable is plugged? More of opposite, to switch from HDMI to headphones on connection and back on disconnection. So for headphone connection, the patch should work as expected because during connection we switch to the headphones anyway. The inverse however isn't true. Do you think it's worth the complexity we'll introduce if we do all that through sound(4) instead? I'm not completely sure to be honest. I think it is essential that when I take off and unplug headphones sound should not go nowhere. I am not saying whether it should be sound(4), virtual_oss, devd or some other daemon, but something should be in charge of deciding what is the best sound device in case one we have now is disconnected. Your code may work in some trivial cases of laptops with only two audio devices on one CODEC, but HDMI in all cases I know is a separate CODEC, there are USB audio devices, etc.

christos added inline comments.May 11 2025, 4:20 PM

sys/dev/sound/pci/hda/hdaa.c
579	Thinking a bit more about it, I have a few things to point out: This patch is not really concerned with audio redirection in general, but only tries to solve the pin-patching issue, so that we can get jack redirection without manually patching HDA. However, one problem I can see with the current approach, is the following scenario. You are playing audio through, say, a USB device, but when you plug headphones to the HDA jack, sound will now redirect there automatically. But what's more is that if you unplug the headphones, the sound will redirect to the HDA speakers, instead of back to the USB device. To solve this, we can just execute the redirection code only if we're already using this HDA codec, otherwise we do nothing. With this solved, for the end-user, the functionality is almost identical to what we already do. Regarding further audio redirection as you mention, I'm a bit skeptical about introducing redirection heuristics across the whole sound system, because that requires us to hardcode a specific hierarchy for what we should be redirecting to, which for some users might not be desirable. For example, should the headphone jack come first, or HDMI (if plugged)? If both get unplugged, do we switch to HDA or other (e.g., USB) drivers, and if so, which one? Although I haven't really decided on this, I think it might be better to let the user manually swap to whatever device they want, as they already do.

adrian added inline comments.May 11 2025, 11:54 PM

sbin/devd/hda.conf
8 ↗	(On Diff #154558)	I'm not super plussed about having devd rules require a port installed :( Lemme poke you about this online to get a bit more context. And yes, I do like the idea of getting devd events in general for sound devices coming and going. Is this all doable right now?

christos added inline comments.May 11 2025, 11:58 PM

sbin/devd/hda.conf
8 ↗	(On Diff #154558)	I don't like using a port either, however virtual_oss will be ported to base anyway, so that shouldn't be a problem. Is this all doable right now? What do you mean?

adrian added inline comments.May 12 2025, 12:00 AM

sys/dev/sound/pci/hda/hdaa.c
579	Ah, sorry for not being clear! I see you add the redirect notification. Do we currently have jack plug/unplug devd notifications?

christos added inline comments.May 12 2025, 2:02 PM

sys/dev/sound/pci/hda/hdaa.c
579	I see you add the redirect notification. Do we currently have jack plug/unplug devd notifications? This patch adds exactly that, we do not currently have notifications for anything else.

mav added inline comments.May 12 2025, 2:09 PM

sys/dev/sound/pci/hda/hdaa.c
579	I see you add the redirect notification. Do we currently have jack plug/unplug devd notifications? This patch adds exactly that, we do not currently have notifications for anything else. This patch is limited specifically to a redirection within one audio function. It is not a generic plug/unplug notification.

christos added inline comments.May 12 2025, 2:11 PM

sys/dev/sound/pci/hda/hdaa.c
579	I detailed some concerns with inter-codec switching in a comment above. I'm still open to discussion and feedback though. :-)

mav added inline comments.May 12 2025, 2:24 PM

sys/dev/sound/pci/hda/hdaa.c
579	I've explained my vision of it couple times. I am getting tired of the discussion. I can not stop you from narrower solution, but I don't like it. Speaking about the skepticism, my Android phone has internal speaker, Bluetooth audio with several headphones and a car, plus USB audio with the same car as part of android auto. And most of them I use practically every day. And "surprise", in most cases it just works in reasonable order. I've never checked what happen if I connect headphones after connecting to car's USB, i.e. whether the priorities are static or just latest connected gets the sound, but if I ever have to find out and the answer is wrong, I can always fall back to manual selection in the UI, but I rarely had to.

christos added inline comments.May 12 2025, 2:30 PM

sys/dev/sound/pci/hda/hdaa.c
579	I will experiment with a few ideas.

adrian added inline comments.May 12 2025, 2:47 PM

sys/dev/sound/pci/hda/hdaa.c
579	I will experiment with a few ideas. Yeah, let's have a think of the events we want to generate first. I really, REALLY would love to see the jack detect events published. How about we start with that?

Run redirection code only if we are already using the current codec, otherwise we'll be switching automatically to the headphones even when using a different device (e.g., USB).
Rename "HDA" devd system to "SND" to make it more general.
Rename "REDIR" devd subsystem to "CONN", according to mav@'s suggestion.

This is still a WIP, so do not take this as a finalized version.

One thing I'd like to note is that I do not object to @mav's suggestion of implementing general automatic audio switching, but this particular patch is trying to address the issue with manual pin-patching inside a single HDA codec, not audio switching as a whole.

Also because virtual_oss is not yet in base, I'm considering @jrtc27's and @adrian's suggestion (on IRC) to move the devd script into the port's directoy, and install it from there in /usr/local/etc/devd/, since the script currently uses a program that's in /usr/local/.