I'm a bit confused as to why X11/wayland would be a huge issue here? The Mesa docs do say X11-only, but they're referring to running the driver on macOS (hence the XQuartz reference), where Wayland basically doesn't exist.
In any case, I don't think Asahi/M1 has proper KWin or Mutter support yet. It's still going to take a while before you get a truly smooth desktop Linux experience on those devices, but some hardware acceleration is definitely better than none!