I would like to switch to Silverblue for the Fedora 35 release but my different tests to do a clean install of Kubernetes failed and I didn’t find any resources about it. The goal of the install is local development.
How do you install Kubernetes on Silverblue ? Any pointers would be appreciated.
Hello @tguimbert ,
I would say https://kevinboone.me/podman_deploy.html?i=1 is possibly a place for some info that seems directed at using Openshift for local dev. Also https://k3s.io/ is lightweight kubernetes (related Mag article at https://fedoramagazine.org/kubernetes-on-fedora-iot-with-k3s/). IoT is a rpm-ostree based OS so instructions will be applicable if slightly outdated. Also, at RH there is this about Nofe.js dev using podman, etc… at https://developers.redhat.com/blog/2019/09/13/develop-with-node-js-in-a-container-on-red-hat-enterprise-linux. And I forgot this https://www.okd.io/minishift/. And if this isn’t already confusing enough check this link out https://github.com/containers/quadlet
Hello @jakfrost .
Thanks for all these resources. I already had some of them and k3s seemed to be the best candidate. But I hoped to find to some resources for things like minikube or kind that have features that help developers.
I will also use podman but I also want to test my manifest and helm charts, so I need a local Kubernetes installation. I don’t how Openshift works but I will read about it with your links. Thanks!
minikube works on silverblue, install moby-engine using rpm-ostree install moby-engine after that reboot your system. After reboot, download minikube binary and place it into your ~/.local/bin directory and change chmod to be able to execute it, and after that start the minikube using minikube start --vm-driver=docker and it will work without issues
Kind works for me very well at least for development and testing.
Also has a driver to use podman (rootless) as container runtime (experimental).
Be sure to enable iptables modules, first, all of them. more info here:
https://kind.sigs.k8s.io/docs/user/rootless/
And also if you have BTRFS enabled, you have to create a custom cluster-config.yaml in order to mount the host volume to the nodes, else will fail on creation.
More info here: kind doesn't work on btrfs · Issue #1416 · kubernetes-sigs/kind · GitHub
Here my configuration for a single node cluster called development:
cluster-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: development
networking:
nodes:
- role: control-plane
# https://github.com/kubernetes-sigs/kind/issues/1416#issuecomment-600438973
# https://kind.sigs.k8s.io/docs/user/configuration/#extra-mounts
kubeadmConfigPatches:
- |
kind: JoinConfiguration
# To enable Ingress this port-mapping is required
# https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings
extraPortMappings:
- containerPort: 80
hostPort: 8080
protocol: TCP
- containerPort: 443
hostPort: 4443
protocol: TCP
extraMounts:
- hostPath: /dev/nvme0n1p3
containerPath: /dev/nvme0n1p3
propagation: HostToContainer
Then create the cluster with your cluster-config.yaml file:
kind create cluster --config=cluster-config.yaml
That should do it 
ps. What’s best: you don’t need to install any additional stuff to get your cluster up and running, just the kind binary and that’s it. no rpm-ostree, no nothing… 
Thank you all for your answers.
I tested @xurzua answer and it works ! I think I will go with it as it is the cleanest install I came across.
Excited to switch to Silverblue for the next release 
The latest minikube (v1.23.2 at the time of this writing) works great with the kvm2 driver on Silverblue if you install virt-manager libvirt-client packages in an rpm-ostree overlay.
Then from there you need to add your user to be able to run virtual machines in polkit:
Create the file /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules and place the following inside, replacing the string USERNAME with the actual username you want to use:
polkit.addRule(function(action, subject) {
if (action.id == "org.libvirt.unix.manage" &&
subject.user == "USERNAME") {
return polkit.Result.YES;
polkit.log("action=" + action);
polkit.log("subject=" + subject);
}
});
Then ensure the libvirtd service is started, you should be able to use minikube as per normal operation:
$ minikube start
😄 minikube v1.23.2 on Fedora 34
✨ Automatically selected the kvm2 driver
💾 Downloading driver docker-machine-driver-kvm2:
> docker-machine-driver-kvm2....: 65 B / 65 B [----------] 100.00% ? p/s 0s
> docker-machine-driver-kvm2: 11.40 MiB / 11.40 MiB [-] 100.00% ? p/s 100ms
💿 Downloading VM boot image ...
> minikube-v1.23.1.iso.sha256: 65 B / 65 B [-------------] 100.00% ? p/s 0s
> minikube-v1.23.1.iso: 225.22 MiB / 225.22 MiB 100.00% 121.96 MiB p/s 2.0
👍 Starting control plane node minikube in cluster minikube
💾 Downloading Kubernetes v1.22.2 preload ...
> preloaded-images-k8s-v13-v1...: 511.84 MiB / 511.84 MiB 100.00% 114.21 M
🔥 Creating kvm2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
❗ /var/home/admiller/bin/kubectl is version 1.20.1-5-g76a04fc, which may have incompatibilites with Kubernetes 1.22.2.
▪ Want kubectl v1.22.2? Try 'minikube kubectl -- get pods -A'
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
$ minikube kubectl -- get pods -A
> kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubectl: 44.73 MiB / 44.73 MiB [-----------] 100.00% 147.37 MiB p/s 500ms
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-78fcd69978-5z7rt 1/1 Running 0 15s
kube-system etcd-minikube 1/1 Running 0 25s
kube-system kube-apiserver-minikube 1/1 Running 0 29s
kube-system kube-controller-manager-minikube 1/1 Running 0 25s
kube-system kube-proxy-hngrc 1/1 Running 0 15s
kube-system kube-scheduler-minikube 1/1 Running 0 25s
kube-system storage-provisioner 1/1 Running 0 24s
$ grep VERSION= /etc/os-release
VERSION="34.20211012.0 (Silverblue)"
REDHAT_BUGZILLA_PRODUCT_VERSION=34
REDHAT_SUPPORT_PRODUCT_VERSION=34
OSTREE_VERSION='34.20211012.0'