Using Yubikey for SSH, always asking for password (GNOME)

Same story here with YubiKey 5.

I’ve noticed that if the key is plugged into the computer before turning it on then I have to restart pcscd to get it to work.

If the YubiKey is not plugged in before booting and I plug it in when I’m already at my desktop the SSH key appears just fine.

My guess is that since the YubiKey is plugged in when the computer starts and the pcscd is a system service vs gpg-agent being a user service, pcscd starts earlier, sees the YubiKey ant takes exclusive access of it. Later scdaemon starts and can’t access it. Restarting pcscd releases the YubiKey and scdaemon can take control of it so the SSH key shows up.

I don’t know how openSUSE does it but I didn’t have this problem on Tumbleweed about a month ago when I was still using it on the same computer with the same YubiKey and the same configuration (except I had KDE instead of Gnome).

Is there a way to make pcscd not take exclusive access or somehow ignore YubiKeys?