Scrubbing through x264 files freezes playback

No matter whether it’s the Videos app, mpv or VLC, whenever I open any one of my x264 Videos attempting to jump around in the video to different sections, it freezes.
The video plays when just watching it in its entirety. It’s whenever you want to jump in the video that it just freezes for several minutes before continuing playback. Even if it does continue, often (especially when you attempt to jump more than once) it will not resume the video but only the audio playback (after having thought about it for minutes).
It’s been an issue for a while, I finally managed to get it fixed (after installing libavcodec I believe). However it is now broken again and I cannot seem to get it to work at all anymore.
There isn’t any issue with the AV1 files I tested.

Broken File Info

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L5.1
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 1 frame
Codec ID : V_MPEG4/ISO/AVC
Duration : 1 h 38 min
Width : 2 560 pixels
Height : 1 440 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Default : No
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Working File Info

Video
ID : 1
Format : AV1
Format/Info : AOMedia Video 1
Format profile : Main@L5.0
Codec ID : V_AV1
Duration : 1 h 39 min
Width : 2 560 pixels
Height : 1 440 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 0)
Bit depth : 8 bits
Default : No
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

If any additional information is required, please ask.