ADB does not require extra groups on Fedora.
Make sure to enable USB debugging on your phone.
If the issue persists, revoke USB debugging authorizations.
If you have installed USBGuard, temporarily stop it.
Also try a different USB cable and USB port.