Gpg --card-status fails

I do have a yubikey and a nitrokey. They both show the same error when I issue

$ gpg --card-status
gpg: selecting card failed: No such device
gpg: OpenPGP card not available: No such device

if I restart pcscd the issue disappears and gpg outputs
the correct card info.

But if eject and reinsert the key it again fails.

The log shows the following error after inserting the key:

eb 12 14:28:10 oreo kernel: usb 1-2: new full-speed USB device number 7 using xhci_hcd
Feb 12 14:28:10 oreo kernel: usb 1-2: New USB device found, idVendor=1050, idProduct=0407, bcdDevice= 5.71
Feb 12 14:28:10 oreo kernel: usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Feb 12 14:28:10 oreo kernel: usb 1-2: Product: YubiKey OTP+FIDO+CCID
Feb 12 14:28:10 oreo kernel: usb 1-2: Manufacturer: Yubico
Feb 12 14:28:10 oreo kernel: input: Yubico YubiKey OTP+FIDO+CCID as /devices/pci0000:00/0000:00:08.1/0000:c1:00.3/usb1/1-2/1-2:1.0/0003:1050:0407.000A/input/input18
Feb 12 14:28:10 oreo kernel: hid-generic 0003:1050:0407.000A: input,hidraw0: USB HID v1.10 Keyboard [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:c1:00.3-2/input0
Feb 12 14:28:10 oreo kernel: hid-generic 0003:1050:0407.000B: hiddev96,hidraw1: USB HID v1.10 Device [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:c1:00.3-2/input1
Feb 12 14:28:11 oreo kernel: ucsi_acpi USBC000:00: unknown error 256
Feb 12 14:28:11 oreo kernel: ucsi_acpi USBC000:00: GET_CABLE_PROPERTY failed (-5)