Thank you @hexdump . Great tips!. and great repositories too!.
The fact that those PCM2704 chips always come without an enclosure holds me back slightly. Still, I will try one of them, but the ones with 4 buttons, since the aim is to be able to use the box without monitor, mouse or keyboard. Plug the box, connect a midi keyboard, some heaphones or speakers to the usb DAC and start playing, using those buttons jut to switch the instrument.
The sonaremin looks cool, but it is beyond what I intended.
I followed your instructions to build sfizz. With buster I had to change the name of a component in apt install, but it eventually compiled. I connected the midi keyboard with it using a2jmidid and sfizz with jack normally. I tried a sfz file and it seemed to play nicely but I noticed that one of the cores was reaching 100% while the rest were much lower. As I started adding some simultaneous notes, the sound started crackling and that one core stayed at 100%. With fluidsynth the load was minimal all along. I guess fluidsynth is a more mature development and at the moment, the sf2 format is good enough for me.
By the way, I reverted nodm to lightdm-autologin-greeter, which does not do the weird restarting of xwindows upon shutdown, but it still does not really shutdown the box, leaving it at the logo. I guess that is a limitation of the qplus itself.