I could use some advice regarding CoreOS

Hi,

I have a few questions about CoreOS and its suitability for my needs. I currently have a Debian server in need of desperate upgrade running several containers ( currrently in docker). I want to switch to Fedora , either Server or CoreOS.

I’m more drawn to CoreOS, for my simple use case it seem perfect . I don’t really change the base OS much beyond utilising Wireguard for remote access , everything else runs in containers.

I have tested CoreOs in a VM and I’ve started adapting my containers to run using Podman. I’m now testing on an old laptop to perfect my setup before switching my Debian install to Fedora.

This is the basic Butane file I use

variant: fcos
version: 1.6.0
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-ed25519 MY_KEY
      password_hash: My_Password_HASH
storage:
  files:
    - path: /etc/sysctl.d/90-ipv4-ip-forward.conf
      mode: 0644
      contents:
        inline: |
          net.ipv4.ip_forward = 1

    - path: /etc/wireguard/wg0.conf
      mode: 0600
      contents:
        inline: |
          [Interface]
          Address = 10.100.0.1/24
          PrivateKey = my_private_key
          ListenPort = 51821

          PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -s 10.100.0.0/24 -o enp1s0 -j MASQUERADE
          PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -s 10.100.0.0/24 -o enp1s0 -j MASQUERADE

          [Peer]
          PublicKey = my_public_key
          PresharedKey = my pre_shared_key
          AllowedIPs = 10.100.0.2/32
systemd:
  units:
    - name: wg-quick@wg0.service
      enabled: true

Everything works as expected, however it may be just the laptop I’m using but boot to prompt takes maybe 4 or 5 minutes is this normal ?

It displays this for quite a long time,

>>Checking Media Presence...........
>>Media Present..........
>>START PXE over IPv4 on MAC: 00-00-00-00-00-00.

before the boot text starts and eventually I get to the prompt.

also from a fresh install with the above butane and nothing but cockpit installed I have these SELinux errors are they related to the delayed boot times?:



SELinux is preventing ln from create access on the lnk_file boot.mount. For complete SELinux messages run: sealert -l c018328d-e1c0-4ea6-aef7-9c6c4e3ac102

SELinux is preventing cat from getattr access on the file /run/systemd/generator/boot.mount. For complete SELinux messages run: sealert -l 1ab09bce-09c4-4ac5-902d-466844817f59

SELinux is preventing coreos-boot-mou from 'write, open' accesses on the file /run/systemd/generator/boot.mount. For complete SELinux messages run: sealert -l 080241b4-218a-4362-a429-dff62ea83a66

failed to retrieve rpm info for path '/run/systemd/generator/boot.mount':

SELinux is preventing coreos-boot-mou from create access on the file boot.mount. For complete SELinux messages run: sealert -l b8547893-777c-41d1-bdf1-36237d1f56ce

SELinux is preventing coreos-boot-mou from add_name access on the directory boot.mount. For complete SELinux messages run: sealert -l be0e17b3-a253-4dac-b325-7631559cced3

SELinux is preventing coreos-boot-mou from write access on the directory generator. For complete SELinux messages run: sealert -l dbd56e78-a5d5-41a5-bebd-d66abd76b7cc

Apologies for any ignorance regarding SELinux, its always been somewhat an Achilles heel of mine.

I also don’t think I quite understand what BootC and bootable CoreOS images etc.. means. will this mean at some point I will need to transition across to a different setup? or as I read somewhere this should not affect the end user ! I like the idea of set and forget for the most part , but If Fedora CoreOS is in a transition period then maybe I should consider Server instead.

If updates are handled by Zincati within a maintenance window will this be effected in any way if I enable “linger” for a user to start user-run Podman containers.

Cockpit also shows an error:

is this normal also?

I currently use Fedora Kinoite for my desktop which is why I was drawn to CoreOS’s atomic nature and the use of toolbx and Podman containers by default.

thanks in advance for any advice and tips given.

it seems my boot delay was related to my boot order somehow being messed up, so I’ve manged to fix this now and it boots super fast as expected.

I still could use some advice on the above mentioned issues.

1 Like

This is weird, but it might be a bug in Cockpit trying to fetch this URL directly instead of relying on rpm-ostree. Can you file a bug for cockpit?

For the SELinux issues, can you copy/paste the exact log entries from the audit log here or in an issue on the tracker?

Regarding bootc, the plan is that the transition happens without interactions for most users. The transition is already in progress.

You have a lot of questions in a single post so if you can number them that would help (and feel free to re-ask the ones I missed).

Hum … I did see that in the past, there is an option in the bios saying not to check media
then, PXE boot is faster

1 Like

Thanks for the reply and the advice. I will file a bug for this issue for cockpit.

As for the SELinux errors

Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing ln from create access on the lnk_file boot.mount.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing ln from create access on the lnk_file boot.mount. For complete SELinux messages run: sealert -l c018328d-e1c0-4ea6-aef7-9c6c4e3ac102
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing cat from getattr access on the file /run/systemd/generator/boot.mount.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing cat from getattr access on the file /run/systemd/generator/boot.mount. For complete SELinux messages run: sealert -l 1ab09bce-09c4-4ac5-902d-466844817f59
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from 'write, open' accesses on the file /run/systemd/generator/boot.mount.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from 'write, open' accesses on the file /run/systemd/generator/boot.mount. For complete SELinux messages run: sealert -l 080241b4-218a-4362-a429-dff62ea83a66
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from create access on the file boot.mount.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from create access on the file boot.mount. For complete SELinux messages run: sealert -l b8547893-777c-41d1-bdf1-36237d1f56ce
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from add_name access on the directory boot.mount.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from add_name access on the directory boot.mount. For complete SELinux messages run: sealert -l be0e17b3-a253-4dac-b325-7631559cced3
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from write access on the directory generator.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from write access on the directory generator. For complete SELinux messages run: sealert -l dbd56e78-a5d5-41a5-bebd-d66abd76b7cc
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from write access on the directory generator.
Dec 03 13:06:29 localhost.localdomain setroubleshoot[2192]: SELinux is preventing coreos-boot-mou from write access on the directory generator. For complete SELinux messages run: sealert -l dbd56e78-a5d5-41a5-bebd-d66abd76b7cc

These were the “SELinux is preventing” errors I could find in journalctl. Grepping the audit log or using ausearch, did not show me anything for SELinux.

As for bootc, so I shouldn’t notice any difference in how I interact with or if needed how I reinstall CoreOS going forward?

thanks again for your help

For SELinux, you need to search for AVC entries in the journal or audit logs.

For bootc, one of the difference right now is that bootc does not support package layering. How did you install cockpit? If you layered packages then that one of the things where it’s not yet fully decided how that will look like with bootc.

woops,

sorry for the ignorance, I really need to learn SELinux so I can resolve these issues myself. If you know of a SELinux for idiots guide I’m open for advice.

type=AVC msg=audit(1764766706.169:113): avc:  denied  { read } for  pid=1313 comm="agetty" name="22_clhm_enp1s0.issue" dev="nvme0n1p4" ino=939524880 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:object_r:NetworkManager_dispatcher_console_var_run_t:s0 tclass=file permissive=0
type=AVC msg=audit(1764767185.247:189): avc:  denied  { write } for  pid=2134 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3223 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764767185.268:190): avc:  denied  { write } for  pid=2184 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3223 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764767185.268:191): avc:  denied  { add_name } for  pid=2184 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764767185.268:192): avc:  denied  { create } for  pid=2184 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764767185.268:193): avc:  denied  { write open } for  pid=2184 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3242 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764767185.269:194): avc:  denied  { getattr } for  pid=2184 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3242 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764767185.273:195): avc:  denied  { create } for  pid=2186 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1
type=AVC msg=audit(1764853234.011:186): avc:  denied  { write } for  pid=1929 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3179 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764853234.030:187): avc:  denied  { write } for  pid=1979 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3179 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764853234.030:188): avc:  denied  { add_name } for  pid=1979 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764853234.030:189): avc:  denied  { create } for  pid=1979 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764853234.030:190): avc:  denied  { write open } for  pid=1979 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3198 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764853234.031:191): avc:  denied  { getattr } for  pid=1979 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3198 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764853234.034:192): avc:  denied  { create } for  pid=1981 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1
type=AVC msg=audit(1764858255.760:632): avc:  denied  { write } for  pid=2993 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3483 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858255.780:633): avc:  denied  { write } for  pid=3044 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3483 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858255.780:634): avc:  denied  { add_name } for  pid=3044 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858255.780:635): avc:  denied  { create } for  pid=3044 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858255.780:636): avc:  denied  { write open } for  pid=3044 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3509 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858255.781:637): avc:  denied  { getattr } for  pid=3044 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3509 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858255.784:638): avc:  denied  { create } for  pid=3046 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1
type=AVC msg=audit(1764858306.594:706): avc:  denied  { write } for  pid=3136 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3514 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858306.614:707): avc:  denied  { write } for  pid=3186 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3514 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858306.614:708): avc:  denied  { add_name } for  pid=3186 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858306.614:709): avc:  denied  { create } for  pid=3186 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858306.614:710): avc:  denied  { write open } for  pid=3186 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3540 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858306.615:711): avc:  denied  { getattr } for  pid=3186 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3540 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858306.618:712): avc:  denied  { create } for  pid=3188 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1
type=AVC msg=audit(1764858562.976:818): avc:  denied  { write } for  pid=3381 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3581 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858562.996:819): avc:  denied  { write } for  pid=3431 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3581 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858562.996:820): avc:  denied  { add_name } for  pid=3431 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764858562.996:821): avc:  denied  { create } for  pid=3431 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858562.996:822): avc:  denied  { write open } for  pid=3431 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3607 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858562.997:823): avc:  denied  { getattr } for  pid=3431 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3607 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764858563.000:824): avc:  denied  { create } for  pid=3433 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1
type=AVC msg=audit(1764859099.496:1001): avc:  denied  { write } for  pid=3972 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3727 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764859099.517:1002): avc:  denied  { write } for  pid=4022 comm="coreos-boot-mou" name="generator" dev="tmpfs" ino=3727 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764859099.517:1003): avc:  denied  { add_name } for  pid=4022 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1764859099.517:1004): avc:  denied  { create } for  pid=4022 comm="coreos-boot-mou" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764859099.517:1005): avc:  denied  { write open } for  pid=4022 comm="coreos-boot-mou" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3753 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764859099.518:1006): avc:  denied  { getattr } for  pid=4022 comm="cat" path="/run/systemd/generator/boot.mount" dev="tmpfs" ino=3753 scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=file permissive=1
type=AVC msg=audit(1764859099.521:1007): avc:  denied  { create } for  pid=4024 comm="ln" name="boot.mount" scontext=system_u:system_r:coreos_boot_mount_generator_t:s0 tcontext=system_u:object_r:systemd_generator_unit_file_t:s0 tclass=lnk_file permissive=1

these are the AVC entries in the audit.log

I did install cockpit and firewalld with layered packages , if this changes how else could you install system level packages going forward?

So anything with permissive=1 on the line was not actually denied, but is only logged. Some of those we are aware of. See: [rawhide][branched] SELinux AVC denials causes the afterburn write to fail in cloud platforms · Issue #1784 · coreos/fedora-coreos-tracker · GitHub

Can you file an issue in the tracker for the ones that have permissive=0 and that were effectively denied? Please include the full list. Thanks


One option would be another form of package layering. Another option would be to use sysexts, etc. It’s not finalized yet.

1 Like