Yubikey ykman fails disabling OTP (works after reboot)

I recently started using a Yubikey PIV functionality to do ssh logins.
After a few accidental touches on the key spewing out OTP strings on my terminal I realized I could disable OTP. When I did that however the key stopped working until I re-enabled it.

My initial conclusion was that there was something wrong with the Yubikey which caused PIV to break when disabling OTP and I stated as much here. Then I opened a ticket with Yubico.

During troubleshooting with Yubico I tried triggering the issue from within a Windows VM - but using the tooling there everything worked as intended. I then tried ykman on the Fedora 38 host again and was able to break functionality - until I rebooted.

When I described that to Yubico I got the following response:

If a reboot is resolving the issue with your Fedora machine then the issue is local with the operating system and not the YubiKey. The logic here is that if the issue is with the YubiKey or our software, disabling the OTP would break the PIV functionality even after the reboot. This is because reboot of the machine nor re-insertion of the YubiKey would looks the same to the YubiKey firmware. Meaning that a restart of the operating system is not rebooting or making any modification to the YubiKey firmware. I would recommend reaching out to the Fedora community (Communication in Fedora :: Fedora Docs) to ask on why the issue is occurring on your Fedora machine. We wish this news was better.

Hence this post.

I know to little about ykman, pcscd, libykcs11 and ssh-agent interoperation to know how to troubleshoot further. Is there anyone else here sharing my experience?

In the following example I toggled OTP in the “wrong” direction - enabled it instead of disabling it, but the effect is the same, whenever OTP has been enabled/disabled PIV stops working:

[user@laptop ~]$ ykman config usb --list
FIDO U2F
FIDO2
OATH
PIV
OpenPGP
[user@laptop ~]$ ssh-add -s /usr/lib64/libykcs11.so.2
Enter passphrase for PKCS#11:
Card added: /usr/lib64/libykcs11.so.2
[user@laptop ~]$ ssh server.dummy.domain.se
Last login: Tue May 23 09:06:07 2023 from xxx
[user@server ~]$ # success
[user@server ~]$ logout
Shared connection to server.dummy.domain.se closed.
[user@laptop ~]$ ykman config usb --enable OTP
USB configuration changes:
  Enable OTP
  The YubiKey will reboot
Proceed? [y/N]: y
[user@laptop ~]$ ykman config usb --list
OTP
FIDO U2F
FIDO2
OATH
PIV
OpenPGP
[user@laptop ~]$ ssh server.dummy.domain.se
sign_and_send_pubkey: signing failed for ECDSA "Public key for PIV Authentication" from agent: agent refused operation
user@server.dummy.domain.se''s password:

[user@laptop ~]$ systemctl restart pcscd
[user@laptop ~]$ ssh-add -D
All identities removed.
[user@laptop ~]$ ssh-add -s /usr/lib64/libykcs11.so.2
Enter passphrase for PKCS#11:
Card added: /usr/lib64/libykcs11.so.2
[user@laptop ~]$ ssh server.dummy.domain.se
sign_and_send_pubkey: signing failed for ECDSA "Public key for PIV Authentication" from agent: agent refused operation
user@server.dummy.domain.se''s password:

[user@laptop ~]$ ykman config usb --disable OTP
USB configuration changes:
  Disable OTP
  The YubiKey will reboot
Proceed? [y/N]: y
[user@laptop ~]$ ssh server.dummy.domain.se
sign_and_send_pubkey: signing failed for ECDSA "Public key for PIV Authentication" from agent: agent refused operation
user@server.dummy.domain.se''s password:

[user@laptop ~]$ ssh-add -D
All identities removed.
[user@laptop ~]$ ssh-add -s /usr/lib64/libykcs11.so.2
Enter passphrase for PKCS#11:
Card added: /usr/lib64/libykcs11.so.2
[user@laptop ~]$ ssh server.dummy.domain.se
sign_and_send_pubkey: signing failed for ECDSA "Public key for PIV Authentication" from agent: agent refused operation
user@server.dummy.domain.se''s password:

[user@laptop ~]$ ykman config usb --list
FIDO U2F
FIDO2
OATH
PIV
OpenPGP
[user@laptop ~]$ systemctl restart pcscd
[user@laptop ~]$ ssh-add -D
All identities removed.
[user@laptop ~]$ # remove yubikey from usb
[user@laptop ~]$ # plug back in
[user@laptop ~]$ ssh-add -s /usr/lib64/libykcs11.so.2
Enter passphrase for PKCS#11:
Card added: /usr/lib64/libykcs11.so.2
[user@laptop ~]$ ykman config usb --list
FIDO U2F
FIDO2
OATH
PIV
OpenPGP
[user@laptop ~]$ ssh server.dummy.domain.se
sign_and_send_pubkey: signing failed for ECDSA "Public key for PIV Authentication" from agent: agent refused operation
user@server.dummy.domain.se''s password:

[user@laptop ~]$ reboot
# after reboot
[user@laptop ~]$ ssh-add -s /usr/lib64/libykcs11.so.2
Enter passphrase for PKCS#11: 
Card added: /usr/lib64/libykcs11.so.2
[user@laptop ~]$ ssh server.dummy.domain.se
Last login: Mon May 29 08:54:45 2023 from xxx
[user@server ~]$ # successful login again

[user@laptop ~]$ ykman --version
YubiKey Manager (ykman) version: 5.1.0
[user@laptop ~]$ cat /etc/fedora-release 
Fedora release 38 (Thirty Eight)