Initiate a new (X11) desktop session on a remote host

I would like to initiate a new (X11) desktop session on a remote host (let’s say for ‘exporting’ the session with x11vnc). At first, this would be as easy as it is on the local host:

$ xinit /usr/bin/startxfce4 -- :4


X.Org X Server 1.20.6
X Protocol Version 11, Revision 0
Build Operating System:  5.0.6-200.fc29.x86_64 
Current Operating System: Linux 5.5.10-200.fc31.x86_64 #1 SMP Wed Mar 18 14:21:38 UTC 2020 x86_64
Kernel command line: BOOT_IMAGE=(hd0,msdos3)/vmlinuz-5.5.10-200.fc31.x86_64 root=/dev/mapper/vg_sdd-root ro rd.luks.uuid=luks-a9eba963-ff2e-499b-a298-0659c5e29359 rhgb quiet resume=/dev/disk/by-uuid/73299393-6bdb-4230-a1ca-096e5ee32396 splash=silent quiet video.use_native_backlight=1
Build Date: 25 November 2019  12:00:00AM
Build ID: xorg-x11-server 1.20.6-1.fc31 
Current version of pixman: 0.38.4
        Before reporting problems, check
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/home/tpasch/.local/share/xorg/Xorg.4.log", Time: Fri Apr  3 19:38:52 2020
(==) Using config directory: "/etc/X11/xorg.conf.d"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
Fatal server error:
(EE) parse_vt_settings: Cannot open /dev/tty0 (Permission denied)
Please consult the Fedora Project support 
 for help. 
(EE) Please also check the log file at "/home/tpasch/.local/share/xorg/Xorg.4.log" for additional information.
(EE) Server terminated with error (1). Closing log file.
xinit: giving up
xinit: unable to connect to X server: Connection refused
xinit: server error

I guess one of the reason for that is that sddm is (also) running on the remote host. It there easy way to get this running?

Kind regards,