Installing Local Test Kernel on Fedora CoreOS

Hi,

We are running Fedora Core on an appliance and chose it because it was immutable. However, we would like to modify the kernel to remove some unused things and add some drivers.

We were able to build the kernel using the documentation here, but run into problems trying to install the kernel using rpm-ostree

Building a custom kernel/Source RPM - Fedora Project Wiki

The build completes and the output looks okay, but when I try to install, I get dependency conflicts. We are building locally and copying the rpms, we do not have a repository for rpm-ostree to pull from.

I’ve tried a few variations, all failing, either with dependency conflicts or “rpm not found errors.”

sudo rpm-ostree override remove kernel kernel-core kernel-modules kernel-modules-extra --replace ./kernel*.rpm
sudo rpm-ostree override remove kernel kernel-core kernel-modules --replace ./kernel*.rpm
sudo rpm-ostree override remove kernel kernel-core kernel-modules --replace kernel*.rpm
sudo rpm-ostree install kernel-*.rpm
sudo rpm-ostree install kernel-*.rpm --replace
sudo rpm-ostree override install kernel-*.rpm
sudo rpm-ostree override replace kernel-*.rpm
sudo rpm-ostree override replace *.rpm
sudo rpm-ostree override replace --experimental *.rpm
sudo rpm-ostree override replace --experimental kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra

I tried:

sudo rpm-ostree usroverlay --hotfix
sudo rpm -Uvh kernel-*.rpm

Which installed completely, then I edited the /boot/ostree/entries to point to the new kernel, but that fails to boot with ostree saying /sysroot doesn’t match.

The closest I get is with the following, which has a lot of dependency conflicts.

$ sudo rpm-ostree override replace ./kernel*.rpm
Inactive base replacements:
  kernel-devel-6.11.3-200.foobar.fc40.x86_64
  kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64
  kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64
  kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64
  kernel-tools-6.11.3-200.foobar.fc40.x86_64
  kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64
  kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64
  kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64
  kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64
Checking out tree 0bb1317... done
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2024-03-12T11:45:42Z solvables: 3
rpm-md repo 'updates' (cached); generated: 2024-11-05T04:35:00Z solvables: 30687
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'updates-archive' (cached); generated: 2024-11-05T05:27:06Z solvables: 53738
Resolving dependencies... done
error: Could not depsolve transaction; 3 problems detected:
 Problem 1: conflicting requests
  - package kernel-devel-matched-6.11.3-200.fc40.x86_64 from updates-archive requires kernel-core = 6.11.3-200.fc40, but none of the providers can be installed
  - package kernel-devel-matched-6.11.5-200.fc40.x86_64 from updates requires kernel-core = 6.11.5-200.fc40, but none of the providers can be installed
  - package kernel-devel-matched-6.11.5-200.fc40.x86_64 from updates-archive requires kernel-core = 6.11.5-200.fc40, but none of the providers can be installed

I’m building on an fc40 machine, when I try to build the kernel on Fedora Core machine it fails the instantiating vmlinuz, with an “lvm module not found” error.

Any help would be greatly appreciated.

Just an update, I was explicit about --remove and --install and got it down to one dependency conflict, however, I’m still not able to resolve that last one.

$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
Inactive requests:
  kernel-devel-uname-r (already provided by kernel-devel-6.11.3-200.foobar.fc40.x86_64)
  kernel-modules-extra (already provided by kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64)
  kernel-modules-internal (already provided by kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64)
  kernel-devel-matched (already provided by kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64)
Checking out tree 0bb1317... done
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2024-03-12T11:45:42Z solvables: 3
rpm-md repo 'updates' (cached); generated: 2024-11-05T04:35:00Z solvables: 30687
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'updates-archive' (cached); generated: 2024-11-05T05:27:06Z solvables: 53738
Resolving dependencies... done
error: Could not depsolve transaction; 1 problem detected:
 Problem: cannot install both kernel-devel-6.11.3-200.foobar.fc40.x86_64 from @commandline and kernel-devel-6.11.3-200.fc40.x86_64 from updates-archive
  - conflicting requests


$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --remove=kernel-devel-6.11.3-200.fc40.x86_64 --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
error: Package "kernel-devel-6.11.3-200.fc40.x86_64" not found


$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --remove=kernel-devel-6.11
.3-200.fc40 --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
error: Package "kernel-devel-6.11.3-200.fc40" not found


$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --remove=kernel-devel-6.11
.3-200 --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
error: Package "kernel-devel-6.11.3-200" not found

try to reset all your overrides first and then try your command:

sudo rpm-ostree override reset --all

Thanks for the suggestion, unfortunately I get stuck with the same problem.

$ sudo rpm-ostree override reset --all
Checking out tree 0bb1317... done
Resolving dependencies... done
No change.


$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --remove=kernel-devel-6.11.3-200 --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
error: Package "kernel-devel-6.11.3-200" not found


$ sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --remove=kernel-modules --remove=kernel-modules-core --install=kernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --install=kernel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-uki-virt-addons-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-modules-6.11.3-200.foobar.fc40.x86_64.rpm   --install=kernel-tools-libs-6.11.3-200.foobar.fc40.x86_64.rpm   kernel-6.11.3-200.foobar.fc40.x86_64.rpm
Inactive requests:
  kernel-devel-uname-r (already provided by kernel-devel-6.11.3-200.foobar.fc40.x86_64)
  kernel-modules-extra (already provided by kernel-modules-extra-6.11.3-200.foobar.fc40.x86_64)
  kernel-modules-internal (already provided by kernel-modules-internal-6.11.3-200.foobar.fc40.x86_64)
  kernel-devel-matched (already provided by kernel-devel-matched-6.11.3-200.foobar.fc40.x86_64)
Checking out tree 0bb1317... done
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2024-03-12T11:45:42Z solvables: 3
rpm-md repo 'updates' (cached); generated: 2024-11-05T04:35:00Z solvables: 30687
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'updates-archive' (cached); generated: 2024-11-05T05:27:06Z solvables: 53738
Resolving dependencies... done
error: Could not depsolve transaction; 1 problem detected:
 Problem: cannot install both kernel-devel-6.11.3-200.foobar.fc40.x86_64 from @commandline and kernel-devel-6.11.3-200.fc40.x86_64 from updates-archive
  - conflicting requests

I’ve tried --remove=kernel-devel, --remove=kernel-devel-matched, etc., but can’t get that to resolve.

Do you really need kernel-devel, the uki ones, etc? You should only install the ones that you need (i.e. the ones you removed) from the RPMs, not all of them.

I installed a fresh image, so there were no overlays or other modifications to the system. I limited my installs and replace to just what was shown with rpm -qa | grep kernel, but I still get the following error:

sudo rpm-ostree override replace --remove=kernel --remove=kernel-core --install=k
ernel-core-6.11.3-200.foobar.fc40.x86_64.rpm --remove=kernel-modules --install=kernel-modules-6.11.3-200.foobar.fc40.x86
_64.rpm --remove=kernel-modules-core --install=kernel-modules-core-6.11.3-200.foobar.fc40.x86_64.rpm kernel-6.11.3-200.f
oobar.fc40.x86_64.rpm
Checking out tree a095055... done
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2024-03-12T11:45:42Z solvables: 3
rpm-md repo 'updates' (cached); generated: 2024-11-11T05:04:26Z solvables: 30859
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'updates-archive' (cached); generated: 2024-11-11T05:41:42Z solvables: 55277
Resolving dependencies... done
error: Base packages not marked to be removed: kernel, kernel-core, kernel-modules, kernel-modules-core

Any thoughts?

If the package names don’t change, you need rpm-ostree override replace *.rpm.