For Fedora 40, `nvidia-smi` does not work even after re-installing multiple times

I know there are multiple Topic regarding nvidia drivers but I checked them and tried their stuff, still doesn’t work. I’m on Fedora 40 and I’ve been trying to set up gpu for pytorch-cuda. I tried many thing but it didn’t worked. At some point I was able to run nvcc --version, but nvidia-smi always gave me this output:
nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
I’ve tried installing from command line as well downloading the .run file from nvidia’s site.
I’m trying to install Cuda 12.1 and Nvidia drivers.

First, I never recommend installing nvidia drivers nor cuda from any location other than the rpmfusion repo.

Guidance on doing that is available here.

The current release of cuda for fedora is 12.4 and is installed as shown with those links

Drivers from other sites must be removed before installing the ones from rpmfusion to avoid conflicts.

Re-installed as per the instruction still same error.
polaris@fedora:~$ rpm -qa | grep -i nvidia


polaris@fedora:~$ nvidia-smi
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

That shows most of the packages are installed but does not show the driver is loaded.
It also does not show the nvidia-gpu-firmware package which is mandatory for use of almost all nvidia gpus.
Please reinstall that package with sudo dnf install nvidia-gpu-firmware

After reinstalling the firmware package (and rebooting) please post the output of
lsmod | grep -E "nvidia|nouveau" and mokutil --sb-state

Preformatted text can be done by placing ``` on the lines preceding and following the text or by highlighting the text then clicking on the </> button on the toolbar.