Handbrake and hardware H.265 encoding through AMD VCN

Accordingly to RadeonFeature AMD VCE/VCN hardware encoding should be supported for recent AMD GPUs.
After I had upgraded my system to a Ryzen 5700G I thought I would been able to use the GPU to encode videos to h.265 with Handbrake, but it doesn’t work… whenever I choose the appropriate preset in Handbrake it always falls back to software h.264 encoding.

Does h.265 encoder patent limitations in Fedora also applies to hardware capabilities? Or is it rpmfusion’s Handbrake built without VCE/VCN support?
Is there a way I can check if VCE/VCN works out of Handbrake?

I’d really like to be able to encode my videos without dual booting into Windows.

Hi, I tried to install HandBrake from Flathub and use hardware preset H.265 VCN 1080p and H.265 VCN 2160p 4K, it’s works. I played also the result files with VCL and check the properties it mentioned VCN.

But I’m not sure weather it’s really use GPU or fallback during the encoding. But from documentation said:

Support for the AMD VCE encoder is enabled in preferences on the video tab. If your system is not supported, the option will be disabled.

And the option are available on my installation.

Well, I did try to use the Flathub version, but it behaves the same: whenever I choose H.265 VCN it falls back to H.264 software… and I know my Ryzen 5700G supports H.265 VCN because with the same Handbrake version under Windows it works.
Have you installed any AMD driver?

Checking if Handbrake is really using the GPU is quite straightforward: the same video I’m trying to encode requires over 40 minutes in H.265 software, but less than 2 minutes using H.265 VCN under Windows…

I never install any AMD driver, just use what available from Fedora. When mention it’s slow, yes it also slow here, but sorry I can’t compare it with Windows.

Btw it’s never fallback to H.264 if I choose video encoder to H.265. The result still showing H.265, but I could be wrong. The only configurations I changed only hardware preset and video encoder. Maybe you forget to change the video encoder and only change the hardware preset (about fallback to H.264 since it’s the default configuration for Hanbrake).