Swift just needs to gain cmake bidirectional support, like, SPM should be able to compile CMake projects and vice-versa, or be able to use vcpkg. Once that's done, I could very much see myself switching to it for a lot of things.
I know, and that's a good thing. That's probably one of the biggest blockers for it's wider adoption. I hope that CMake can eventually get similar abilities for calling into SPM or just outright handling Swift package manifests. That might be too much to ask though.