I’m running k0s on Fedora CoreOS, and any time I shutdown the server, I have to reinstall the service. It survives a soft reboot fine. There is a service file that gets installed into /etc/systemd/system/ but is wiped at restart. What’s the proper way to handle this? I could write the file there through the ignition file, but would that cause a problem on first boot before I have installed k0s? Thanks for any guidance.
Are you running a “live” system? Live systems run from memory and all changes are lost when the system is powered off.
I guess if you let us know how you are running in (maybe point to a docs page that you are following) that would help.
Ah, maybe that’s what I’m doing. Still trying to figure out how this works.
I’m running it on kubevirt, and basing my setup off this doc. Provisioning Fedora CoreOS on KubeVirt :: Fedora Docs
Here is my butane file.
variant: fcos
version: 1.4.0
storage:
disks:
- device: /dev/vdb
wipe_table: false
partitions:
- number: 1
label: var
size_mib: 0
filesystems:
- path: /var
device: /dev/disk/by-partlabel/var
format: xfs
with_mount_unit: true
files:
- path: /etc/NetworkManager/system-connections/enp1s0.nmconnection
mode: 0600
contents:
inline: |
[connection]
id=enp1s0
type=ethernet
interface-name=enp1s0
[ipv4]
address1=10.3.3.20/24,10.3.3.1
dns=8.8.8.8;
dns-search=
may-fail=false
method=manual
- path: /etc/hostname
mode: 0644
contents:
inline: k8s-node1
passwd:
users:
- name: core
groups:
- sudo
ssh_authorized_keys:
- mykey
This points to your entire system being reprovisioned from scratch when you shut it down. Are any other changes in /etc
preserved? If you want to keep this logic then you need to declare all configuration changes that you want in /etc
in you Ignition config.
I think that’s what’s happening. Pretty sure nothing is preserved. I guess I thought that was just how this all worked. I’d be happy to have it installed instead, just not sure how to get there from here. I started with ISO installs on Proxmox, but my challenge was getting the Ignition file into it for initial setup. Trying to avoid running a web server to host the config, so I was having to create an ignition ISO to mount on the VM every time I changed a configuration. It was kind of a hassle, so when I discovered I could launch the kubevirt image and have the ignition config stored in a k8s secret, it suddenly made things super easy for me to get going. Turns out not so easy to keep things going.
Ahh. You know we are learning more and more about kubevirt ourselves as we go through this. What I’ve found out over time is that a “containerdisk” in kubevirt land isn’t something that’s persistent. i.e. it’s kind of like containers where when you stop a container and start it again anything that wasn’t in a Volume is gone. That’s what you are hitting here.
You can try to start a virtual machine type and use the data importer stuff to get the container imported into an actual PV (i.e. the OS will be persistent). Try with something like this:
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: fcos
spec:
runStrategy: Always
dataVolumeTemplates:
- metadata:
name: fcos-os-disk-volume
spec:
storage:
volumeMode: Block
resources:
requests:
storage: 10Gi
accessModes:
- ReadWriteOnce
source:
registry:
url: "docker://quay.io/fedora/fedora-coreos-kubevirt:stable"
template:
spec:
domain:
devices:
disks:
- disk:
bus: virtio
name: fcos-os-disk
- disk:
bus: virtio
name: cloudinitdisk
rng: {}
resources:
requests:
memory: 2048M
volumes:
- dataVolume:
name: fcos-os-disk-volume
name: fcos-os-disk
- name: cloudinitdisk
cloudInitConfigDrive:
secretRef:
name: ignition-payload
Depending on your cluster you may need to set storageClassName: name
under dataVolumeTemplates.spec.storage
.
Look at the .service file and find the missing files and copy and paste into a folder where they belong.
I opened a PR to enhance the docs: provisioning/kubevirt: add persistent disk example by dustymabe · Pull Request #580 · coreos/fedora-coreos-docs · GitHub
Super. Thanks so much! I’m going to give this a try.