Ignition: partition layout for CoreOS

Project Discussion
1

I’m confused about the partitons Ignition/CoreOS requires and how to configure them.

When not configuring partitions the following are created (in that order):
/boot
/boot/efi
(mbr boot partition)
/sysroot

My first try was setting up those partitions with the root filesystem (/sysroot) on a raid1 which resulted in:

systemd[1]: Starting Ignition OSTree: Grow root filesystem...
coreos-growpart[707]: cat: /sys/devices/virtual/block/md127/partition: No such file or directory
systemd[1]: ignition-ostree-growfs.service: Main process exited, code=exited, status=1/FAILURE

(ignition-ostree-growfs.service)

raid.yaml 
variant: fcos
version: 1.0.0
storage:
  disks:
    - device: /dev/vda
      wipe_table: true 
      partitions:
        - label: boot
          number: 1
          size_mib: 400
          start_mib: 0
          type_guid: BC13C2FF-59E6-4262-A352-B275FD6F7172
        - label: EFI-SYSTEM
          number: 0
          size_mib: 50
          type_guid: C12A7328-F81F-11D2-BA4B-00A0C93EC93B
        - label: bios
          number: 0
          size_mib: 1
          type_guid: 21686148-6449-6E6F-744E-656564454649
        - label: root-r1
          number: 0
          size_mib: 50000
          type_guid: A19D880F-05FC-4D3B-A006-743F0F84911E
    - device: /dev/vdb
      wipe_table: true 
      partitions:
        - label: root-r2
          number: 1
          size_mib: 50000
          type_guid: A19D880F-05FC-4D3B-A006-743F0F84911E
  raid:
    - name: rootfs
      level: raid1
      devices:
      - /dev/disk/by-partlabel/root-r1
      - /dev/disk/by-partlabel/root-r2
  filesystems:
    - path: /boot
      device: /dev/disk/by-partlabel/boot
      format: ext4
      label: boot
    - path: /boot/efi
      device: /dev/disk/by-partlabel/EFI-SYSTEM
      format: vfat
      label: EFI-SYSTEM
    - path: /sysroot
      device: /dev/md/rootfs
      format: ext4
      label: root

ok, raid doesn’t work. Let’s try a normal partition.

systemd[1]: Starting Mount OSTree /var...
ignition-ostree-mount-var[745]: /sysroot//ostree/boot.1/fedora-coreos/3d0f96f51f3cb5047c0e8d4158d80ad5f5af74446d7ee3baecdcba9c7c23f1b9/0 is not a symlink
systemd[1]: ignition-ostree-mount-var.service: Main process exited, code=exited, status=1/FAILURE

(ignition-ostree-mount-var.service)

plain.yaml 
variant: fcos
version: 1.0.0
storage:
  disks:
    - device: /dev/vda
      wipe_table: true 
      partitions:
        - label: boot
          number: 1
          size_mib: 400
          start_mib: 0
          type_guid: BC13C2FF-59E6-4262-A352-B275FD6F7172
        - label: EFI-SYSTEM
          number: 0
          size_mib: 50
          type_guid: C12A7328-F81F-11D2-BA4B-00A0C93EC93B
        - label: bios
          number: 0
          size_mib: 1
          type_guid: 21686148-6449-6E6F-744E-656564454649
        - label: root-r1
          number: 0
          size_mib: 50000
          type_guid: A19D880F-05FC-4D3B-A006-743F0F84911E
  filesystems:
    - path: /boot
      device: /dev/disk/by-partlabel/boot
      format: ext4
      label: boot
    - path: /boot/efi
      device: /dev/disk/by-partlabel/EFI-SYSTEM
      format: vfat
      label: EFI-SYSTEM
    - path: /sysroot
      device: /dev/disk/by-partlabel/root-r1
      format: ext4
      label: root

How do I have to configure Ignition to not fail on creating/using my partitions? And what partitions are required/expected?

2

You are trying to re-configure main OS partitions (rootfs, bootfs, ESP) which at the moment is not supported on FCOS (similar statement for complex storage setup, like RAID).

This is currently an active area of brainstorming among developers, but no concrete details at this point. The issue is tracked and discussed at https://github.com/coreos/fedora-coreos-tracker/issues/94.

3

Is this now working?

Because trying this, I still can’t boot.
First I tried this:

storage:
  disks:
    - device: /dev/sda
      wipe_table: true
      partitions:
        - number: 1
          label: swap
          size_mib: 8192
        - number: 2
          label: boot
          size_mib: 1024
        - number:
          label: root
          size_mib: 51200
        - number: 4
          label: var
  filesystems:
    - device: /dev/disk/by-partlabel/swap
      format: swap
      wipe_filesystem: true
      with_mount_unit: true
    - path: /boot
      device: /dev/disk/by-partlabel/boot
      format: ext4
      wipe_filesystem: true
      with_mount_unit: true
    - path: /var
      device: /dev/disk/by-partlabel/var
      format: btrfs
      wipe_filesystem: true
      with_mount_unit: true
    - path: /
      device: /dev/disk/by-partlabel/root
      format: xfs
      wipe_filesystem: true
      with_mount_unit: true

Then I tried to leave root, boot, EFI &co unconfigured.

storage:
storage:
  disks:
    - device: /dev/sda
      wipe_table: false
      partitions:
        - label: swap
          size_mib: 8192
          size_mib: 1024
        - label: root
        - label: var
          size_mib: 51200
  filesystems:
    - device: /dev/disk/by-partlabel/swap
      format: swap
      with_mount_unit: true
    - path: /var
      device: /dev/disk/by-partlabel/var
      format: btrfs
      with_mount_unit: true
4

Fedora CoreOS does support this now. In your filesystems block, you need to set the label of the root filesystem to root, so the support code knows that you’re replacing the root filesystem.

Also, in your second example, you’re actually creating a second root partition after the default one. Because of the way Ignition matches partitions, you need to add partition: 4 to the root partition to get it to reuse the existing partition.

It’s possible to replace the entire partition table as you’re trying to do in your first example, but it’s better to leave the existing partitions alone unless there’s a reason to change them. I’d recommend something closer to your second example. Something like this should work (untested):

storage:
  disks:
    - device: /dev/sda
      wipe_table: false
      partitions:
        - label: root
          partition: 4
          size_mib: 8192
          resize: true
        - label: swap
          size_mib: 1024
        - label: var
  filesystems:
    - device: /dev/disk/by-partlabel/swap
      format: swap
      with_mount_unit: true
    - device: /dev/disk/by-partlabel/root
      format: xfs
      label: root
      wipe_filesystem: true
    - path: /var
      device: /dev/disk/by-partlabel/var
      format: btrfs
      with_mount_unit: true
      label: var
1 Like