In Fedora 39, when running VLC, the option to use VA-API for hardware acceleration is unavailable, as shown in the screenshot below. This happens even if proprietary codecs are installed.
This is a notable change from prior releases (where VLC wasn’t available in Fedora, but could be installed from third-party repositories).
You can discuss this issue here.
Since Fedora 39, VLC is in Fedora main repositories. However, the current VLC version 3.0 requires an old FFmpeg (version 4) in order for VA-API support to work. Fedora 39 contains FFmpeg version 6. For this reason, VLC 3.0 is unable to leverage VA-API support.
This problem should be resolved once VLC 4.0 is released.
Bugzilla report: #2254999
If you require VA-API acceleration, the simplest workaround is to install VLC from Flathub instead of VLC from Fedora RPM repositories. VLC from Flathub pulls a matching version of FFMpeg which allows VA-API acceleration.
In GNOME Software, you should see a drop-down option to select the installation source, if you have Flathub repository enabled. You can enable repositories in app settings.
Or you can use these commands:
flatpak remote-modify --enable flathub
flatpak install flathub org.videolan.VLC