I did something similar with pulseaudio about 15 years ago, had an old thinkpad running Debian, and then multicast activated on my source. Worked surprisingly well!
I use the 2S and the 3 over Bluetooth (Windows 10 and Linux (Pipewire/pulseaudio)) and they generally work well.
But on Linux I sometimes get lagging sound on my bt headphones, when I use the mouse heavily, and on Windows I get a rubber band effect on the mouse when I use a headset for calls, but not music. ymmv
Spent some time reading through matrix specification; my opinion is that it's overkill in some ways and under-delivers in some parts of its specs for my usecase, enough to justify writing a whole new protocol from scratch
With that being said, we're working on implementing a matrix compatibility layer!