I’ve been working on setting up Fedora CoreOS on my server and running into persistent issues that I haven’t been able to resolve. I’m hoping someone here can help me identify what I’m doing wrong and guide me in the right direction.
What I’m Trying to Achieve
I’m aiming to install Fedora CoreOS with the following setup:
- RAID1 Configuration for Boot and Root:
- Mirrored boot, EFI, and root partitions across two SSDs for redundancy.
- Configuring RAID1 for
/bootand/andEFI.
- Separate
/varPartition:
/varis configured on an NVMe drive (/dev/nvme0n1).- It is encrypted with LUKS and will eventually be formatted with ZFS after the installation with the /var partition from the root drive moved to the NVME drive after installation.
- Encrypted LUKS Containers for Additional Drives:
- I plan on setting up the LUKS encryption on some additional drives (NAS-1, NAS-2, NAS-3) that I will arrange in a RAID-Z configuration after installation with ZFS.
The partitions are defined in my Ignition configuration file, which I’ve attached to this post for reference.
The Problem I’m Facing
RAID Assembly Fails During Installation:
The installer logs show errors like:
sdb4: Process '/usr/sbin/mdadm -If sdb4 --path pci-0000:01:00.1-ata-2.0' failed with exit code 1.
sdb3: Process '/usr/sbin/mdadm -If sdb3 --path pci-0000:01:00.1-ata-2.0' failed with exit code 1.
sda4: Process '/usr/sbin/mdadm -If sda4 --path pci-0000:01:00.1-ata-1.0' failed with exit code 1.
sda3: Process '/usr/sbin/mdadm -If sda3 --path pci-0000:01:00.1-ata-1.0' failed with exit code 1.
Then the installer times out waiting for those devices that never get properly made:
Ignition failed: failed to create filesystems: failed to wait on filesystems devs: device unit dev-disk-by\x2dlabel-boot.device timeout
Some things I’ve tried
-
I ran my butane file through the checker with the strict option and I’ve used ignition-validate to confirm that there aren’t any errors with formatting.
-
I’ve adjusted my ignition configuration in so many different ways that I’ve lost count
-
I’ve also read the Fedora CoreOS and Ignition documentation up and down many multiple times but there has to be something I’m missing here.
What I Need Help With
Correctly Configuring RAID in Ignition:
- Is there something I’m missing in my Ignition file that’s causing the RAID setup to fail?
Here is my Butane configuration file (with mild redactions)
variant: fcos
version: 1.5.0
# Authentication
passwd:
users:
- name: core
ssh_authorized_keys:
- ssh-ed25519 SSH_KEYS_HERE
# Specify boot device is mirrored between two disks
#boot_device:
# Mirrored boot disk on boot disk SSD
#mirror:
#devices:
# Boot Disk 1
#- /dev/disk/by-id/ata-CT250MX500SSD1_2307E6AD8730
# Boot Disk 2
#- /dev/disk/by-id/ata-CT250MX500SSD1_2307E6AD876F
storage:
disks:
# Boot Disk 1
- device: /dev/disk/by-id/ata-CT250MX500SSD1_2307E6AD8730
wipe_table: true
partitions:
# EFI partition 1
- label: esp-1
size_mib: 1024
#type_guid: "c12a7328-f81f-11d2-ba4b-00a0c93ec93b"
# Boot partition 1
- label: boot-1
size_mib: 1024
# Root partition 1
- label: root-1
# Boot Disk 2
- device: /dev/disk/by-id/ata-CT250MX500SSD1_2307E6AD876F
wipe_table: true
partitions:
# EFI parition 2
- label: esp-2
size_mib: 1024
#type_guid: "c12a7328-f81f-11d2-ba4b-00a0c93ec93b"
# Boot partition 2
- label: boot-2
size_mib: 1024
# Root parition 2
- label: root-2
# NVME Fast Storage used for /var in CoreOS
- device: /dev/disk/by-id/nvme-WD_Red_SN700_1000GB_220443800077
wipe_table: true
partitions:
- label: var
# Seagate NAS Drive
- device: /dev/disk/by-id/ata-ST4000DM004-2CV104_ZFN0TVG2
wipe_table: true
partitions:
- label: NAS-1
# WD NAS Drive 1
- device: /dev/disk/by-id/ata-WDC_WD40EFPX-68C6CN0_WD-WX62D4312RJ2
wipe_table: true
partitions:
- label: NAS-2
# WD NAS Drive 2
- device: /dev/disk/by-id/ata-WDC_WD40EFPX-68C6CN0_WD-WXH2D430RSK5
wipe_table: true
partitions:
- label: NAS-3
# WD NVR Drive
- device: /dev/disk/by-id/ata-WDC_WD22PURZ-85B4ZY0_WD-WX42AC2M58ZR
wipe_table: true
partitions:
- label: NVR-1
raid:
# Setup RAID device for boot partition
- name: md-boot
level: raid1
devices:
- /dev/disk/by-partlabel/boot-1
- /dev/disk/by-partlabel/boot-2
# Setup RAID device for root partition
- name: md-root
level: raid1
devices:
- /dev/disk/by-partlabel/root-1
- /dev/disk/by-partlabel/root-2
luks:
- name: root
device: /dev/md/md-root
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
#label: root
- name: var
device: /dev/disk/by-partlabel/var
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
label: var
- name: NAS-1
device: /dev/disk/by-partlabel/NAS-1
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
label: NAS-1
- name: NAS-2
device: /dev/disk/by-partlabel/NAS-2
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
label: NAS-2
- name: NAS-3
device: /dev/disk/by-partlabel/NAS-3
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
label: NAS-3
- name: NVR-1
device: /dev/disk/by-partlabel/NVR-1
clevis:
custom:
needs_network: false
pin: tpm2
config: '{"pcr_bank":"sha256","pcr_ids":"7"}'
wipe_volume: true
label: NVR-1
filesystems:
# Primary EFI partition
- path: /boot/efi
device: /dev/disk/by-partlabel/esp-1
format: vfat
wipe_filesystem: true
label: esp-1
with_mount_unit: true
mount_options: ["umask=0077"]
# Secondary EFI partition for redundancy
- path: /boot/efi2
device: /dev/disk/by-partlabel/esp-2
format: vfat
wipe_filesystem: true
label: esp-2
with_mount_unit: true
mount_options: ["umask=0077"]
# Boot partition
- path: /boot
device: /dev/md/md-boot
format: ext4
wipe_filesystem: true
label: boot
with_mount_unit: true
# Root parition
- path: /
device: /dev/mapper/root
format: ext4
wipe_filesystem: true
label: root
with_mount_unit: true
files:
# Define Hostname as CoreOS_Server
- path: /etc/hostname
mode: 0644
contents:
inline: CoreOS_Server
# Define Keyboard Layout as US keymap
- path: /etc/vconsole.conf
mode: 0644
contents:
inline: KEYMAP=us
# Configure Swap on ZRAM as 8GB using lz4 compression
- path: /etc/systemd/zram-generator.conf
mode: 0644
contents:
inline: |
# This config file enables a /dev/zram0 device with the default settings
[zram0]
# Set the fraction of total memory for ZRAM (e.g., 50% of total RAM)
# Adjust to fit your memory requirements
zram-size = ram / 2
# Optionally set the compression algorithm (e.g., zstd, lz4)
compression-algorithm = lz4
# Define swap priority (higher number has higher priority)
swap-priority = 100
# Max zram device limit to avoid too large allocation, useful for systems with lots of RAM
max-zram-size = 8192M
# Configure Grub password
grub:
users:
- name: root
# Specify grub password hash for grub user 'root'
password_hash: grub.pbkdf2.sha512.GRUB_HASH
I’ve already spent an embarrassingly long amount of time on this, so any advice or guidance would be greatly appreciated!
Thanks for your help!
!
