I'm in heat stasis, so for now just some unsorted lines...
I've build emacs28 with X11 support on riscv64 before, but that means I have to build the xsets at home (like every other day) instead of being able to update from daily builds (which currently do not include the xsets).
If someone could nudge the riscv64 port people to include xsets in daily builds, I'd use them instead of trying nonstandard dance moves.
But indepently: If there is a build option that claims to disable x11
in a build, it please should work.