How to install Kubernetes on Silverblue?

I would like to switch to Silverblue for the Fedora 35 release but my different tests to do a clean install of Kubernetes failed and I didn’t find any resources about it. The goal of the install is local development.

How do you install Kubernetes on Silverblue ? Any pointers would be appreciated.

Hello @tguimbert ,
I would say https://kevinboone.me/podman_deploy.html?i=1 is possibly a place for some info that seems directed at using Openshift for local dev. Also https://k3s.io/ is lightweight kubernetes (related Mag article at https://fedoramagazine.org/kubernetes-on-fedora-iot-with-k3s/). IoT is a rpm-ostree based OS so instructions will be applicable if slightly outdated. Also, at RH there is this about Nofe.js dev using podman, etc… at https://developers.redhat.com/blog/2019/09/13/develop-with-node-js-in-a-container-on-red-hat-enterprise-linux. And I forgot this https://www.okd.io/minishift/. And if this isn’t already confusing enough check this link out https://github.com/containers/quadlet

Hello @jakfrost .
Thanks for all these resources. I already had some of them and k3s seemed to be the best candidate. But I hoped to find to some resources for things like minikube or kind that have features that help developers.

I will also use podman but I also want to test my manifest and helm charts, so I need a local Kubernetes installation. I don’t how Openshift works but I will read about it with your links. Thanks!

minikube works on silverblue, install moby-engine using rpm-ostree install moby-engine after that reboot your system. After reboot, download minikube binary and place it into your ~/.local/bin directory and change chmod to be able to execute it, and after that start the minikube using minikube start --vm-driver=docker and it will work without issues

2 Likes

Kind works for me very well at least for development and testing.

https://kind.sigs.k8s.io/

Also has a driver to use podman (rootless) as container runtime (experimental).

Be sure to enable iptables modules, first, all of them. more info here:
https://kind.sigs.k8s.io/docs/user/rootless/

And also if you have BTRFS enabled, you have to create a custom cluster-config.yaml in order to mount the host volume to the nodes, else will fail on creation.

More info here: kind doesn't work on btrfs · Issue #1416 · kubernetes-sigs/kind · GitHub

Here my configuration for a single node cluster called development:

cluster-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: development
networking:
nodes:
- role: control-plane
  # https://github.com/kubernetes-sigs/kind/issues/1416#issuecomment-600438973
  # https://kind.sigs.k8s.io/docs/user/configuration/#extra-mounts	  
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
  # To enable Ingress this port-mapping is required
  # https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings
  extraPortMappings:  
  - containerPort: 80 
    hostPort: 8080
    protocol: TCP
  - containerPort: 443
    hostPort: 4443
    protocol: TCP
  extraMounts:
    - hostPath: /dev/nvme0n1p3
      containerPath: /dev/nvme0n1p3
      propagation: HostToContainer

Then create the cluster with your cluster-config.yaml file:

kind create cluster --config=cluster-config.yaml

That should do it :sparkles:

ps. What’s best: you don’t need to install any additional stuff to get your cluster up and running, just the kind binary and that’s it. no rpm-ostree, no nothing… :eyes:

3 Likes

Thank you all for your answers.

I tested @xurzua answer and it works ! I think I will go with it as it is the cleanest install I came across.

Excited to switch to Silverblue for the next release :smiley:

1 Like

The latest minikube (v1.23.2 at the time of this writing) works great with the kvm2 driver on Silverblue if you install virt-manager libvirt-client packages in an rpm-ostree overlay.

Then from there you need to add your user to be able to run virtual machines in polkit:

Create the file /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules and place the following inside, replacing the string USERNAME with the actual username you want to use:

polkit.addRule(function(action, subject) {
        if (action.id == "org.libvirt.unix.manage" &&
            subject.user == "USERNAME") {
                return polkit.Result.YES;
                polkit.log("action=" + action);
                polkit.log("subject=" + subject);
        }
});

Then ensure the libvirtd service is started, you should be able to use minikube as per normal operation:

$ minikube start
😄  minikube v1.23.2 on Fedora 34
✨  Automatically selected the kvm2 driver
💾  Downloading driver docker-machine-driver-kvm2:
    > docker-machine-driver-kvm2....: 65 B / 65 B [----------] 100.00% ? p/s 0s
    > docker-machine-driver-kvm2: 11.40 MiB / 11.40 MiB [-] 100.00% ? p/s 100ms
💿  Downloading VM boot image ...
    > minikube-v1.23.1.iso.sha256: 65 B / 65 B [-------------] 100.00% ? p/s 0s
    > minikube-v1.23.1.iso: 225.22 MiB / 225.22 MiB  100.00% 121.96 MiB p/s 2.0
👍  Starting control plane node minikube in cluster minikube
💾  Downloading Kubernetes v1.22.2 preload ...
    > preloaded-images-k8s-v13-v1...: 511.84 MiB / 511.84 MiB  100.00% 114.21 M
🔥  Creating kvm2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass

❗  /var/home/admiller/bin/kubectl is version 1.20.1-5-g76a04fc, which may have incompatibilites with Kubernetes 1.22.2.
    ▪ Want kubectl v1.22.2? Try 'minikube kubectl -- get pods -A'
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

$ minikube kubectl -- get pods -A
    > kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubectl: 44.73 MiB / 44.73 MiB [-----------] 100.00% 147.37 MiB p/s 500ms
NAMESPACE     NAME                               READY   STATUS    RESTARTS   AGE
kube-system   coredns-78fcd69978-5z7rt           1/1     Running   0          15s
kube-system   etcd-minikube                      1/1     Running   0          25s
kube-system   kube-apiserver-minikube            1/1     Running   0          29s
kube-system   kube-controller-manager-minikube   1/1     Running   0          25s
kube-system   kube-proxy-hngrc                   1/1     Running   0          15s
kube-system   kube-scheduler-minikube            1/1     Running   0          25s
kube-system   storage-provisioner                1/1     Running   0          24s

$ grep VERSION= /etc/os-release
VERSION="34.20211012.0 (Silverblue)"
REDHAT_BUGZILLA_PRODUCT_VERSION=34
REDHAT_SUPPORT_PRODUCT_VERSION=34
OSTREE_VERSION='34.20211012.0'

1 Like