Steam Link controllers don't work in games when connected to GNOME remotely

Hi everyone, I’m running Fedora 41 with Gnome Remote Services

I start a RDP session with GNOME remote services and then try to run a game with Steam Link on Android connected to the Fedora host.
In that situation controllers, be it physical or touch controls work fine within the Steam Big picture mode but once I launch a game it just won’t respond to controller input. If I use the Steam button to open the overlay, it opens and the controller works within the overlay.

The behavior is not observed when connecting with Steam Link to a non remote session, be it a gnome or a kde plasma session.

Weird thing is it was working fine yesterday when I tested it with gnome remote services

Is there anything I can do to make it work as expected?