Connect via sftp via Nautilus to Fedora CoreOS?

If I have Fedora CoreOS installed, I can connect via usual SSH on the command line, but when I try to use sftp by entering this in Nautilus, it just seems to load forever:

sftp://core@device/

On other devices with SSH this works and I know the given username and host name is valid (also tried it with the IP).

I also checked the journalctl logs on the server and client side and nothing is being logged there.

Client: Fedora Workstation 34
Server: Fedora CoreOS 34.20210427.3.0