After looking at some kernel stack traces, I found that this guy was responsible: https://www.kernel.org/doc/html/latest/sound/soc/dapm.html
When the PCM device is closed the following happens in the kernel:
snd_pcm_release() -> soc_pcm_close() -> delay of `pmdown_time` -> close_delayed_work()
The last function powers down the audio subsystem, causing the hum. To circumvent this behavior, I simply set `pmdown_time` to -1:
echo -1 > /sys/devices/platform/soc@01c00000/1c22c00.codec/cdc/pmdown_time
Now, once the sound subsystem is powered on, it stays powered on. Not ideal but it seems to work.