No you don’t!
For example this will rebase you to F34 beta without having to uninstall any rpmfusion packages
rpm-ostree rebase fedora:fedora/34/x86_64/silverblue --uninstall rpmfusion-free-release-33-1.noarch --install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-branched.noarch.rpm --uninstall rpmfusion-nonfree-release-33-1.noarch --install https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-branched.noarch.rpm