Unable to Upgrade to 35.20211029.3.0

Hello,

My Fcos instances are failing to upgrade to 35.20211029.3.0 from 34.20211031.3.0. rpm-ostree status gives me the following:

State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; update staged: 35.20211029.3.0; reboot pending due to update strategy
Deployments:
  fedora:fedora/x86_64/coreos/stable
                   Version: 35.20211029.3.0 (2021-11-17T23:45:08Z)
                BaseCommit: ccdac241ed4b83b55e4d72e864923679c00a15804fbea7d1e638e67c2a08dd5c
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
                      Diff: 422 upgraded, 7 removed, 7 added
           LayeredPackages: kubectl-1.19.0 qemu-guest-agent tuned cri-o kubeadm-1.19.0 kubelet-1.19.0
            EnabledModules: cri-o:1.19

● fedora:fedora/x86_64/coreos/stable
                   Version: 34.20211031.3.0 (2021-11-08T23:55:31Z)
                BaseCommit: 056a1508fad0eecb740cf10a949d2292e11084195f1defbf455f418a5991daa2
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: kubectl-1.19.0 qemu-guest-agent tuned cri-o kubeadm-1.19.0 kubelet-1.19.0
            EnabledModules: cri-o:1.19

Note, I have the cri-o module enabled and installed. This was added using the rpm-ostree install command but since a change was made (I think in 34.20210904.3.0), I had to re-enable and re-add using the experimental option in the rpm-ostree command. (See cri-o-no-longer-available-through-rpm-ostree.

So rpm-ostree status shows that everything is ready to go, but when I reboot, the system is back on 34.20211031.3.0 and Zincati goes through it’s check again.

Here are the logs after reboot for rpm-ostree:

Dec 02 13:01:54 fcos-01 systemd[1]: rpm-ostree-countme.timer: Deactivated successfully.
Dec 02 13:01:54 fcos-01 systemd[1]: Stopped Weekly rpm-ostree Count Me timer.
Dec 02 13:02:04 fcos-01 systemd-tmpfiles[762]: /usr/lib/tmpfiles.d/rpm-ostree-1-autovar.conf:1: Duplicate line for path "/var/lib/alternatives", ignoring.
Dec 02 13:02:04 fcos-01 systemd[1]: Started Weekly rpm-ostree Count Me timer.
Dec 02 13:02:05 fcos-01 systemd[1]: Starting rpm-ostree System Management Daemon...
Dec 02 13:02:05 fcos-01 rpm-ostree[849]: Reading config file '/etc/rpm-ostreed.conf'
Dec 02 13:02:06 fcos-01 rpm-ostree[849]: In idle state; will auto-exit in 61 seconds
Dec 02 13:02:06 fcos-01 systemd[1]: Started rpm-ostree System Management Daemon.
Dec 02 13:02:06 fcos-01 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=rpm-ostreed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 02 13:02:07 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] registering as the update driver for rpm-ostree
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Locked sysroot
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Initiated txn Deploy for client(dbus:1.14 unit:zincati.service uid:977): /org/projectatomic/rpmostree1/fedora_coreos
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Process [pid: 2121 uid: 977 unit: zincati.service] connected to transaction progress
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Unlocked sysroot
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: Process [pid: 2121 uid: 977 unit: zincati.service] disconnected from transaction progress
Dec 02 13:02:08 fcos-01 rpm-ostree[849]: In idle state; will auto-exit in 61 seconds
Dec 02 13:02:11 fcos-01 rpm-ostree[849]: Locked sysroot
Dec 02 13:02:11 fcos-01 rpm-ostree[849]: Initiated txn Deploy for client(dbus:1.21 unit:zincati.service uid:977): /org/projectatomic/rpmostree1/fedora_coreos
Dec 02 13:02:11 fcos-01 rpm-ostree[849]: Process [pid: 3296 uid: 977 unit: zincati.service] connected to transaction progress
Dec 02 13:02:11 fcos-01 rpm-ostree[849]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Dec 02 13:02:12 fcos-01 rpm-ostree[849]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.76.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.0) libssh/0.9.6/openssl/zlib nghttp2/1.43.0)
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-f35&arch=x86_64
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/487513ec7d95eefbd61c05b5d85a7fb3ded655479e95c2ce4752d695097134d9-primary.xml.zck
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/119c67c4e7be7ce53fa185400ef87a63040a46ad881576cd31cd80c08e57af1f-filelists.xml.zck
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/efb712305e244eb653738ef92db36a469f9f2ea0b53ad8a3dd6390c38e4d7128-comps-Everything.x86_64.xml.zck
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/725d2584edd6b688f762112171f848089a8ba5d1671cced81ddc92bc43af2d43-updateinfo.xml.zck
Dec 02 13:02:17 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/725d2584edd6b688f762112171f848089a8ba5d1671cced81ddc92bc43af2d43-updateinfo.xml.zck
Dec 02 13:02:18 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/119c67c4e7be7ce53fa185400ef87a63040a46ad881576cd31cd80c08e57af1f-filelists.xml.zck
Dec 02 13:02:18 fcos-01 rpm-ostree[849]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/487513ec7d95eefbd61c05b5d85a7fb3ded655479e95c2ce4752d695097134d9-primary.xml.zck
Dec 02 13:02:18 fcos-01 rpm-ostree[849]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Dec 02 13:02:18 fcos-01 rpm-ostree[849]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f35&arch=x86_64
Dec 02 13:02:19 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/repomd.xml
Dec 02 13:02:19 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/123dcf0a121e8e84f9e79b8544be5daa8ebe7a6127dac15b3a93eac16e6b7acf-primary.xml.zck
Dec 02 13:02:19 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/1b8a90fbb3c0639cfb514d80aa946535f9edeaff4c93d8dde27b036042ba7ffb-filelists.xml.zck
Dec 02 13:02:19 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/90eeb6cbf34a216e27e7ffff54daee2ecc2cb88ffd4dde54f5ff418ab4eca388-modules.yaml.gz
Dec 02 13:02:19 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/123dcf0a121e8e84f9e79b8544be5daa8ebe7a6127dac15b3a93eac16e6b7acf-primary.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/1b8a90fbb3c0639cfb514d80aa946535f9edeaff4c93d8dde27b036042ba7ffb-filelists.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/1dd82fc766b81b170d1768f2a0f597098399db37b2198fd9f2034620c3f20197-updateinfo.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: http://mirror.genesisadaptive.com/fedora/linux/updates/35/Modular/x86_64/repodata/1dd82fc766b81b170d1768f2a0f597098399db37b2198fd9f2034620c3f20197-updateinfo.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/repomd.xml
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/636ef6c64f681bfb587ca727e707ebfe5ce1d701ce2238ff113fef4746a511ce-primary.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/92fb5e77d0563d4eb5b2feea7871f763032e0f45e33a3d53421a586926807adb-filelists.xml.zck
Dec 02 13:02:20 fcos-01 rpm-ostree[849]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/636ef6c64f681bfb587ca727e707ebfe5ce1d701ce2238ff113fef4746a511ce-primary.xml.zck
Dec 02 13:02:21 fcos-01 rpm-ostree[849]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/92fb5e77d0563d4eb5b2feea7871f763032e0f45e33a3d53421a586926807adb-filelists.xml.zck
Dec 02 13:02:33 fcos-01 rpm-ostree[849]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'updates', 'fedora', 'updates-modular', 'fedora-modular', 'kubernetes', 'updates-archive'] solvables: 93513
Dec 02 13:02:33 fcos-01 rpm-ostree[849]: Executed %post for conntrack-tools in 153 ms
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: Executed %post for kernel-tools in 92 ms
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: Executed %post for tuned in 81 ms
Dec 02 13:02:34 fcos-01 rpm-ostree(qemu-guest-agent.post)[5660]: Created symlink /etc/systemd/system/dev-virtio\x2dports-org.qemu.guest_agent.0.device.wants/qemu-guest-agent.service → /usr/lib/systemd/system/qemu-guest-agent.service.
Dec 02 13:02:34 fcos-01 rpm-ostree(qemu-guest-agent.post)[5660]: Unit /usr/lib/systemd/system/qemu-guest-agent.service is added as a dependency to a non-existent unit dev-virtio\x2dports-org.qemu.guest_agent.0.device.
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: Executed %post for qemu-guest-agent in 79 ms
Dec 02 13:02:34 fcos-01 rpm-ostree(cri-o.post)[5675]: ln: failed to create symbolic link '{_unitdir}/cri-o.service': No such file or directory
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: Executed %post for cri-o in 96 ms
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: Executed %posttrans for tuned in 55 ms
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: No files matched %transfiletriggerin(lib) for glibc-common
Dec 02 13:02:34 fcos-01 rpm-ostree[849]: No files matched %transfiletriggerin(lib64) for glibc-common
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 583 ms; 14717 matched files
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 51 ms; 824 matched files
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 35 ms; 26 matched files
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 41 ms; 70 matched files
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: sanitycheck(/usr/bin/true) successful
Dec 02 13:02:35 fcos-01 rpm-ostree[849]: Regenerating rpmdb for target
Dec 02 13:02:40 fcos-01 rpm-ostree[849]: Wrote commit: 0026bf23e93d34faa3516835b3cda0914b5f43e8c10657fa50d496808dd78b7c; New objects: meta:5 content:1 totaling 17.0 MB)
Dec 02 13:02:42 fcos-01 rpm-ostree[849]: note: Deploying commit 0026bf23e93d34faa3516835b3cda0914b5f43e8c10657fa50d496808dd78b7c which contains content in /var/lib that will be ignored.
Dec 02 13:02:43 fcos-01 rpm-ostree[849]: Created new deployment /ostree/deploy/fedora-coreos/deploy/0026bf23e93d34faa3516835b3cda0914b5f43e8c10657fa50d496808dd78b7c.0
Dec 02 13:02:47 fcos-01 rpm-ostree[849]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Dec 02 13:02:50 fcos-01 rpm-ostree[849]: Unlocked sysroot
Dec 02 13:02:50 fcos-01 rpm-ostree[849]: Process [pid: 3296 uid: 977 unit: zincati.service] disconnected from transaction progress
Dec 02 13:02:50 fcos-01 rpm-ostree[849]: In idle state; will auto-exit in 61 seconds

And here are the logs from Zincati:

Dec 02 13:01:54 fcos-01 systemd[1]: Stopping Zincati Update Agent...
Dec 02 13:01:55 fcos-01 systemd[1]: zincati.service: Deactivated successfully.
Dec 02 13:01:55 fcos-01 systemd[1]: Stopped Zincati Update Agent.
-- Boot 1f570aaf16574352aae5ebf16f0d3262 --
Dec 02 13:02:05 fcos-01 systemd[1]: Starting Zincati Update Agent...
Dec 02 13:02:05 fcos-01 zincati[821]: [INFO  zincati::cli::agent] starting update agent (zincati 0.0.23)
Dec 02 13:02:07 fcos-01 zincati[821]: [INFO  zincati::cincinnati] Cincinnati service: https://updates.coreos.fedoraproject.org
Dec 02 13:02:07 fcos-01 zincati[821]: [INFO  zincati::cli::agent] agent running on node '650bcef949bf4390a8f756d74c0b84a8', in update group 'default'
Dec 02 13:02:07 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] registering as the update driver for rpm-ostree
Dec 02 13:02:09 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] found 1 other finalized deployment
Dec 02 13:02:09 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] deployment 34.20210904.3.0 (45e0f7e030762d03f5fb4584463cc0b81a320e641ef5e8>
Dec 02 13:02:09 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] initialization complete, auto-updates logic enabled
Dec 02 13:02:09 fcos-01 zincati[821]: [INFO  zincati::strategy] update strategy: periodic, total schedule length 60 minutes; next window at 6:0 on Sa>
Dec 02 13:02:09 fcos-01 systemd[1]: Started Zincati Update Agent.
Dec 02 13:02:09 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] reached steady state, periodically polling for updates
Dec 02 13:02:10 fcos-01 zincati[821]: [INFO  zincati::cincinnati] current release detected as not a dead-end
Dec 02 13:02:10 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] target release '35.20211029.3.0' selected, proceeding to stage it
Dec 02 13:02:51 fcos-01 zincati[821]: [INFO  zincati::update_agent::actor] update staged: 35.20211029.3.0

Nothing stands out to me as to what prevented it from updating. Can anyone help me get the new update rolled out?

Also, on a side note, I could not gather logs from journalctl for rpm-ostree with the --unit option (e.g. journalctl -u rpm-ostree -S “5 minutes ago”). I had to grep for it.

Help is greatly appreciated!
Thanks
Ray

Seems CRI-O module is not yet ready for F35

Strange, as the package exists in the repo

Hey @rayray221. How are you performing the update? Are you by any chance manually rebooting your nodes rather than letting zincati initiate the reboot? If you manually reboot the node you will boot into the same deployment you were originally booted into.

More context in: RFE: Allow manual update checks and reboots · Issue #498 · coreos/zincati · GitHub

Hello Dusty,

Ahh, I did not realize that a reboot alone would not trigger an update. Regardless, I have also tried:

sudo rpm-ostree upgrade --bypass-driver

Followed by:

sudo systemctl reboot

I actually have a schedule for Zincati open for 6am EST tomorrow morning and can let you know the results if that helps as well.

Thanks
Ray

Hello @dustymabe

Thanks, the scheduled update window in Zincati worked and my systems are now running 35.20211029.3.0.

Question though, I would have thought that:

sudo rpm-ostree upgrade --bypass-driver

Followed by:

sudo systemctl reboot

Would have worked. Assuming it does not, is there any way to manually execute an upgrade?

Thanks again, I really appreciate all the help you have been providing me. :smile:

Ray

I would have thought so too, Maybe the update had already been staged by Zincati before you ran sudo rpm-ostree upgrade --bypass-driver?

Possible…maybe I’ll try this again with the next update to see if I get the same results…

Thanks Dusty

Finally found some time to futz with this again; still cannot manually upgrading. An update was currently staged by Zincati so I ran:

sudo rpm-ostree cleanup -p

Which removed the staged update. Then I ran

sudo rpm-ostree upgrade --bypass-driver

After a reboot, I still have a pending upgrade. :man_shrugging:

Guessing it’s not possible to manually upgrade if a schedule in Zincati exists.

Thanks
Ray

To me this sounds like the finalization isn’t succeeding during the shutdown. Do you by chance have any logs that you can share?

Additionally I can’t remember if adding a --reboot will cause the finalization to happen immediately or not, but you can at least try it:

sudo rpm-ostree upgrade --bypass-driver --reboot

Here is the first part of the rpm-ostree logs from the time I was trying this…(had to break it into two post b/c of character limit.)

Jan 07 12:55:06 fcos-01 rpm-ostree[1902518]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 12:55:08 fcos-01 rpm-ostree[1902518]: In idle state; will auto-exit in 60 seconds
Jan 07 12:55:08 fcos-01 rpm-ostree[1902518]: client(id:cli dbus:1.399 unit:session-4.scope uid:0) added; new total=1
Jan 07 12:55:13 fcos-01 rpm-ostree[1902518]: client(id:cli dbus:1.399 unit:session-4.scope uid:0) vanished; remaining=0
Jan 07 12:55:13 fcos-01 rpm-ostree[1902518]: In idle state; will auto-exit in 64 seconds
Jan 07 12:57:48 fcos-01 rpm-ostree[881]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 12:57:50 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 60 seconds
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Locked sysroot
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Initiated txn Deploy for client(dbus:1.15 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Process [pid: 2285 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Unlocked sysroot
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: Process [pid: 2285 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 12:57:51 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 63 seconds
Jan 07 12:57:54 fcos-01 rpm-ostree[881]: Locked sysroot
Jan 07 12:57:54 fcos-01 rpm-ostree[881]: Initiated txn Deploy for client(dbus:1.21 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 12:57:54 fcos-01 rpm-ostree[881]: Process [pid: 3381 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 12:57:55 fcos-01 rpm-ostree[881]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 12:57:56 fcos-01 rpm-ostree[881]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 12:57:56 fcos-01 rpm-ostree[881]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Jan 07 12:57:56 fcos-01 rpm-ostree[881]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f35&arch=x86_64
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://mirror.siena.edu/fedora/linux/updates/35/Modular/x86_64/repodata/repomd.xml
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://mirror.siena.edu/fedora/linux/updates/35/Modular/x86_64/repodata/cb1011863b680c60eea33388e9fb93550aa9cfa8920394a47d6a77feb5ae86f6-primary.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://mirror.siena.edu/fedora/linux/updates/35/Modular/x86_64/repodata/c2ec609492a55c46093b32fbafb4c184eb6e43ce5da0d7eac1475d1d5f6a2b03-filelists.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://mirror.siena.edu/fedora/linux/updates/35/Modular/x86_64/repodata/5313f3b6f5b9b4985a8773c96589537c72346aa80e4008b0f376a03f0c682c63-modules.yaml.gz
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://mirror.siena.edu/fedora/linux/updates/35/Modular/x86_64/repodata/976ab4169243d86798199100dbc54f51986d863d4b8bc35e7d23d069a3258485-updateinfo.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-f35&arch=x86_64
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/c5e61bc71c92503b4268680a286e749b7688cfc255ade2a69b8f2c6988f7d23f-primary.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/d62bd11b0a299a670bde53339fddb9e41348e371ae2a2580269d8d97225b476f-filelists.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/09d5d783b8fb62a6dc21e4f2c55f9918506aabb551582756e35f27f087c26319-comps-Everything.x86_64.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/5b65104ec80fff5a08c3ddf55542fb35a7ad3e4518d50f3a31f0ded33fb19576-updateinfo.xml.zck
Jan 07 12:57:57 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/5b65104ec80fff5a08c3ddf55542fb35a7ad3e4518d50f3a31f0ded33fb19576-updateinfo.xml.zck
Jan 07 12:57:58 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/c5e61bc71c92503b4268680a286e749b7688cfc255ade2a69b8f2c6988f7d23f-primary.xml.zck
Jan 07 12:57:58 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/repodata/d62bd11b0a299a670bde53339fddb9e41348e371ae2a2580269d8d97225b476f-filelists.xml.zck
Jan 07 12:57:59 fcos-01 rpm-ostree[881]: Downloading: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-35-x86_64
Jan 07 12:57:59 fcos-01 rpm-ostree[881]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/repomd.xml
Jan 07 12:58:00 fcos-01 rpm-ostree[881]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/3c9ee343d92e434be5f4bbc3a76541c499a3642cb4bdec7629bafa34540d2e22-primary.xml.zck
Jan 07 12:58:00 fcos-01 rpm-ostree[881]: Downloading: https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/repodata/59311d5b139a5b24a4f8a3e2ea9048400577256a6588271d6e903b6ff10b9635-filelists.xml.zck
Jan 07 12:58:06 fcos-01 rpm-ostree[881]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %post for conntrack-tools in 144 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %post for kernel-tools in 93 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %post for tuned in 107 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %post for qemu-guest-agent in 111 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %post for cri-o in 89 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: Executed %posttrans for tuned in 46 ms
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 12:58:07 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 550 ms; 14795 matched files
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 42 ms; 824 matched files
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 40 ms; 26 matched files
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 36 ms; 70 matched files
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: sanitycheck(/usr/bin/true) successful
Jan 07 12:58:08 fcos-01 rpm-ostree[881]: Regenerating rpmdb for target
Jan 07 12:58:12 fcos-01 rpm-ostree[881]: Wrote commit: 8e2b0ce410b08960bc05c261fd851c990d1c872b170cd933cb9705b65b7494f0; New objects: meta:5 content:1 totaling 16.8 MB)
Jan 07 12:58:14 fcos-01 rpm-ostree[881]: note: Deploying commit 8e2b0ce410b08960bc05c261fd851c990d1c872b170cd933cb9705b65b7494f0 which contains content in /var/lib that will be ignored.
Jan 07 12:58:16 fcos-01 rpm-ostree[881]: Created new deployment /ostree/deploy/fedora-coreos/deploy/8e2b0ce410b08960bc05c261fd851c990d1c872b170cd933cb9705b65b7494f0.0
Jan 07 12:58:20 fcos-01 rpm-ostree[881]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 12:58:24 fcos-01 rpm-ostree[881]: Unlocked sysroot
Jan 07 12:58:24 fcos-01 rpm-ostree[881]: Process [pid: 3381 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 12:58:24 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 61 seconds
Jan 07 12:58:24 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.66 unit:session-2.scope uid:0) added; new total=1
Jan 07 12:58:28 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.66 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 12:58:28 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 63 seconds
Jan 07 12:59:01 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.70 unit:session-2.scope uid:0) added; new total=1
Jan 07 12:59:02 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.70 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 12:59:02 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 61 seconds
Jan 07 12:59:09 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.73 unit:session-2.scope uid:0) added; new total=1
Jan 07 12:59:10 fcos-01 rpm-ostree[881]: Locked sysroot
Jan 07 12:59:10 fcos-01 rpm-ostree[881]: Initiated txn Cleanup for client(id:cli dbus:1.73 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 12:59:10 fcos-01 rpm-ostree[881]: Process [pid: 10200 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 12:59:13 fcos-01 rpm-ostree[881]: Txn Cleanup on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 12:59:16 fcos-01 rpm-ostree[881]: Unlocked sysroot
Jan 07 12:59:16 fcos-01 rpm-ostree[881]: Process [pid: 10200 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 12:59:17 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.73 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 12:59:17 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 62 seconds
Jan 07 12:59:22 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.75 unit:session-2.scope uid:0) added; new total=1
Jan 07 12:59:23 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.75 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 12:59:23 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 61 seconds
Jan 07 12:59:26 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.77 unit:session-2.scope uid:0) added; new total=1
Jan 07 12:59:27 fcos-01 rpm-ostree[881]: Locked sysroot
Jan 07 12:59:27 fcos-01 rpm-ostree[881]: Initiated txn Upgrade for client(id:cli dbus:1.77 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 12:59:27 fcos-01 rpm-ostree[881]: Process [pid: 12803 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 12:59:29 fcos-01 rpm-ostree[881]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 12:59:29 fcos-01 rpm-ostree[881]: 2 metadata, 0 content objects fetched; 788 B transferred in 1 seconds; 0 bytes content written
Jan 07 12:59:31 fcos-01 rpm-ostree[881]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 12:59:31 fcos-01 rpm-ostree[881]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-f35&arch=x86_64
Jan 07 12:59:31 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python3-libs-3.10.1-2.fc35.x86_64.rpm
Jan 07 12:59:31 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python3-3.10.1-2.fc35.x86_64.rpm
Jan 07 12:59:31 fcos-01 rpm-ostree[881]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python-unversioned-command-3.10.1-2.fc35.noarch.rpm
Jan 07 12:59:33 fcos-01 rpm-ostree[881]: Imported 3 pkgs
Jan 07 12:59:33 fcos-01 rpm-ostree[881]: Executed %post for conntrack-tools in 105 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %post for kernel-tools in 89 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %post for tuned in 108 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %post for qemu-guest-agent in 94 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %post for cri-o in 118 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %posttrans for tuned in 48 ms
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 240 ms; 14795 matched files
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 45 ms; 824 matched files
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 39 ms; 26 matched files
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 40 ms; 70 matched files
Jan 07 12:59:34 fcos-01 rpm-ostree[881]: sanitycheck(/usr/bin/true) successful
Jan 07 12:59:35 fcos-01 rpm-ostree[881]: Regenerating rpmdb for target
Jan 07 12:59:39 fcos-01 rpm-ostree[881]: Wrote commit: 167a9671082037334a528465ff524c899dcbc7dffee27e4bb4db76aaac84214d; New objects: meta:108 content:1 totaling 16.8 MB)
Jan 07 12:59:41 fcos-01 rpm-ostree[881]: note: Deploying commit 167a9671082037334a528465ff524c899dcbc7dffee27e4bb4db76aaac84214d which contains content in /var/lib that will be ignored.
Jan 07 12:59:42 fcos-01 rpm-ostree[881]: Created new deployment /ostree/deploy/fedora-coreos/deploy/167a9671082037334a528465ff524c899dcbc7dffee27e4bb4db76aaac84214d.0
Jan 07 12:59:46 fcos-01 rpm-ostree[881]: Txn Upgrade on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 12:59:52 fcos-01 rpm-ostree[881]: Unlocked sysroot
Jan 07 12:59:52 fcos-01 rpm-ostree[881]: Process [pid: 12803 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 12:59:54 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.77 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 12:59:54 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 61 seconds
Jan 07 12:59:58 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.79 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:00:01 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.79 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:00:01 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 64 seconds
Jan 07 13:00:20 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.81 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:00:23 fcos-01 rpm-ostree[881]: client(id:cli dbus:1.81 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:00:23 fcos-01 rpm-ostree[881]: In idle state; will auto-exit in 64 seconds
Jan 07 13:01:06 fcos-01 rpm-ostree[842]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 13:01:08 fcos-01 rpm-ostree[842]: In idle state; will auto-exit in 64 seconds
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Locked sysroot
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Initiated txn Deploy for client(dbus:1.15 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Process [pid: 2361 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Unlocked sysroot
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: Process [pid: 2361 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:01:09 fcos-01 rpm-ostree[842]: In idle state; will auto-exit in 63 seconds
Jan 07 13:01:12 fcos-01 rpm-ostree[842]: Locked sysroot
Jan 07 13:01:12 fcos-01 rpm-ostree[842]: Initiated txn Deploy for client(dbus:1.21 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:01:12 fcos-01 rpm-ostree[842]: Process [pid: 3371 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:01:13 fcos-01 rpm-ostree[842]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 13:01:14 fcos-01 rpm-ostree[842]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 13:01:16 fcos-01 rpm-ostree[842]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %post for conntrack-tools in 144 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %post for kernel-tools in 163 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %post for tuned in 129 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %post for qemu-guest-agent in 67 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %post for cri-o in 123 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %posttrans for tuned in 56 ms
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 226 ms; 14795 matched files
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:01:17 fcos-01 rpm-ostree[842]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:01:18 fcos-01 rpm-ostree[842]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 92 ms; 824 matched files
Jan 07 13:01:18 fcos-01 rpm-ostree[842]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 54 ms; 26 matched files
Jan 07 13:01:18 fcos-01 rpm-ostree[842]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 106 ms; 70 matched files
Jan 07 13:01:18 fcos-01 rpm-ostree[842]: sanitycheck(/usr/bin/true) successful
Jan 07 13:01:18 fcos-01 rpm-ostree[842]: Regenerating rpmdb for target
Jan 07 13:01:20 fcos-01 rpm-ostree[842]: client(id:cli dbus:1.48 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:01:23 fcos-01 rpm-ostree[842]: Wrote commit: 3fd8eb92d3c93570badd80ab4e50e3333d1db6ddd1b16485538874800b4dfaff; New objects: meta:5 content:1 totaling 16.8 MB)
Jan 07 13:01:24 fcos-01 rpm-ostree[842]: Process [pid: 4820 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:01:26 fcos-01 rpm-ostree[842]: Process [pid: 4820 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:01:26 fcos-01 rpm-ostree[842]: client(id:cli dbus:1.48 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:01:27 fcos-01 rpm-ostree[842]: note: Deploying commit 3fd8eb92d3c93570badd80ab4e50e3333d1db6ddd1b16485538874800b4dfaff which contains content in /var/lib that will be ignored.
Jan 07 13:01:29 fcos-01 rpm-ostree[842]: Created new deployment /ostree/deploy/fedora-coreos/deploy/3fd8eb92d3c93570badd80ab4e50e3333d1db6ddd1b16485538874800b4dfaff.0
Jan 07 13:01:32 fcos-01 rpm-ostree[842]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:01:35 fcos-01 rpm-ostree[842]: Unlocked sysroot
Jan 07 13:01:35 fcos-01 rpm-ostree[842]: Process [pid: 3371 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:01:35 fcos-01 rpm-ostree[842]: In idle state; will auto-exit in 60 seconds
Jan 07 13:01:39 fcos-01 rpm-ostree[842]: client(id:cli dbus:1.69 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:01:40 fcos-01 rpm-ostree[842]: client(id:cli dbus:1.69 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:01:40 fcos-01 rpm-ostree[842]: In idle state; will auto-exit in 63 seconds
Jan 07 13:04:20 fcos-01 rpm-ostree[11069]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 13:04:21 fcos-01 rpm-ostree[11069]: In idle state; will auto-exit in 64 seconds
Jan 07 13:04:21 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.72 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:04:22 fcos-01 rpm-ostree[11069]: Locked sysroot
Jan 07 13:04:22 fcos-01 rpm-ostree[11069]: Initiated txn Cleanup for client(id:cli dbus:1.72 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:04:22 fcos-01 rpm-ostree[11069]: Process [pid: 11064 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:04:23 fcos-01 rpm-ostree[11069]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 13:04:25 fcos-01 rpm-ostree[11069]: Txn Cleanup on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:04:26 fcos-01 rpm-ostree[11069]: Unlocked sysroot
Jan 07 13:04:26 fcos-01 rpm-ostree[11069]: Process [pid: 11064 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:04:27 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.72 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:04:27 fcos-01 rpm-ostree[11069]: In idle state; will auto-exit in 64 seconds
Jan 07 13:04:29 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.74 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:04:30 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.74 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:04:30 fcos-01 rpm-ostree[11069]: In idle state; will auto-exit in 63 seconds
Jan 07 13:04:33 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.76 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:04:34 fcos-01 rpm-ostree[11069]: Locked sysroot
Jan 07 13:04:34 fcos-01 rpm-ostree[11069]: Initiated txn Upgrade for client(id:cli dbus:1.76 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:04:34 fcos-01 rpm-ostree[11069]: Process [pid: 14619 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:04:35 fcos-01 rpm-ostree[11069]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 13:04:35 fcos-01 rpm-ostree[11069]: 2 metadata, 0 content objects fetched; 788 B transferred in 0 seconds; 0 bytes content written
Jan 07 13:04:37 fcos-01 rpm-ostree[11069]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:04:37 fcos-01 rpm-ostree[11069]: Downloading: https://mirrors.fedoraproject.org/metalink?repo=updates-released-f35&arch=x86_64
Jan 07 13:04:38 fcos-01 rpm-ostree[11069]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python3-libs-3.10.1-2.fc35.x86_64.rpm
Jan 07 13:04:38 fcos-01 rpm-ostree[11069]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python3-3.10.1-2.fc35.x86_64.rpm
Jan 07 13:04:38 fcos-01 rpm-ostree[11069]: Downloading: http://cooler.dbz/fedora/linux/updates/35/Everything/x86_64/Packages/p/python-unversioned-command-3.10.1-2.fc35.noarch.rpm
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Imported 3 pkgs
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %post for conntrack-tools in 114 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %post for kernel-tools in 83 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %post for tuned in 104 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %post for qemu-guest-agent in 81 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %post for cri-o in 93 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %posttrans for tuned in 53 ms
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 180 ms; 14795 matched files
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:04:40 fcos-01 rpm-ostree[11069]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:04:41 fcos-01 rpm-ostree[11069]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 49 ms; 824 matched files
Jan 07 13:04:41 fcos-01 rpm-ostree[11069]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 49 ms; 26 matched files
Jan 07 13:04:41 fcos-01 rpm-ostree[11069]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 38 ms; 70 matched files
Jan 07 13:04:41 fcos-01 rpm-ostree[11069]: sanitycheck(/usr/bin/true) successful
Jan 07 13:04:41 fcos-01 rpm-ostree[11069]: Regenerating rpmdb for target
Jan 07 13:04:45 fcos-01 rpm-ostree[11069]: Wrote commit: 14b4de60823d568f222b79a5b5f6b35a6460c60c013782a1e7b91b1bff11d29b; New objects: meta:108 content:1 totaling 16.8 MB)
Jan 07 13:04:47 fcos-01 rpm-ostree[11069]: note: Deploying commit 14b4de60823d568f222b79a5b5f6b35a6460c60c013782a1e7b91b1bff11d29b which contains content in /var/lib that will be ignored.
Jan 07 13:04:48 fcos-01 rpm-ostree[11069]: Created new deployment /ostree/deploy/fedora-coreos/deploy/14b4de60823d568f222b79a5b5f6b35a6460c60c013782a1e7b91b1bff11d29b.0
Jan 07 13:04:52 fcos-01 rpm-ostree[11069]: Txn Upgrade on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:04:55 fcos-01 rpm-ostree[11069]: Unlocked sysroot
Jan 07 13:04:55 fcos-01 rpm-ostree[11069]: Process [pid: 14619 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:04:57 fcos-01 rpm-ostree[11069]: client(id:cli dbus:1.76 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:04:57 fcos-01 rpm-ostree[11069]: In idle state; will auto-exit in 64 seconds
Jan 07 13:05:28 fcos-01 rpm-ostree[859]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 13:05:30 fcos-01 rpm-ostree[859]: In idle state; will auto-exit in 62 seconds
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Locked sysroot
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Initiated txn Deploy for client(dbus:1.15 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Process [pid: 2354 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Unlocked sysroot
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: Process [pid: 2354 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:05:31 fcos-01 rpm-ostree[859]: In idle state; will auto-exit in 62 seconds
Jan 07 13:05:33 fcos-01 rpm-ostree[859]: Locked sysroot
Jan 07 13:05:33 fcos-01 rpm-ostree[859]: Initiated txn Deploy for client(dbus:1.21 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:05:33 fcos-01 rpm-ostree[859]: Process [pid: 3364 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:05:34 fcos-01 rpm-ostree[859]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 13:05:35 fcos-01 rpm-ostree[859]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 13:05:37 fcos-01 rpm-ostree[859]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:05:37 fcos-01 rpm-ostree[859]: Executed %post for conntrack-tools in 224 ms
Jan 07 13:05:37 fcos-01 rpm-ostree[859]: Executed %post for kernel-tools in 127 ms
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %post for tuned in 115 ms
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %post for qemu-guest-agent in 119 ms
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %post for cri-o in 145 ms
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %posttrans for tuned in 63 ms
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 188 ms; 14795 matched files
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 133 ms; 824 matched files
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 83 ms; 26 matched files
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 63 ms; 70 matched files
Jan 07 13:05:38 fcos-01 rpm-ostree[859]: sanitycheck(/usr/bin/true) successful
Jan 07 13:05:39 fcos-01 rpm-ostree[859]: Regenerating rpmdb for target
Jan 07 13:05:43 fcos-01 rpm-ostree[859]: Wrote commit: f5c9b892d3ad2f9e7fc44463134ae7fd82856cb1c0732fcb5b0bd44b5685cf2d; New objects: meta:5 content:1 totaling 16.8 MB)
Jan 07 13:05:45 fcos-01 rpm-ostree[859]: note: Deploying commit f5c9b892d3ad2f9e7fc44463134ae7fd82856cb1c0732fcb5b0bd44b5685cf2d which contains content in /var/lib that will be ignored.
Jan 07 13:05:46 fcos-01 rpm-ostree[859]: Created new deployment /ostree/deploy/fedora-coreos/deploy/f5c9b892d3ad2f9e7fc44463134ae7fd82856cb1c0732fcb5b0bd44b5685cf2d.0
Jan 07 13:05:50 fcos-01 rpm-ostree[859]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:05:53 fcos-01 rpm-ostree[859]: Unlocked sysroot
Jan 07 13:05:53 fcos-01 rpm-ostree[859]: client(id:cli dbus:1.48 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:05:53 fcos-01 rpm-ostree[859]: Process [pid: 3364 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:05:57 fcos-01 rpm-ostree[859]: client(id:cli dbus:1.48 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:05:57 fcos-01 rpm-ostree[859]: In idle state; will auto-exit in 60 seconds
Jan 07 13:06:57 fcos-01 rpm-ostree[859]: In idle state; will auto-exit in 60 seconds

Part 2

Jan 07 13:10:11 fcos-01 rpm-ostree[11054]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 13:10:13 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 64 seconds
Jan 07 13:10:13 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.70 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:10:13 fcos-01 rpm-ostree[11054]: Locked sysroot
Jan 07 13:10:13 fcos-01 rpm-ostree[11054]: Initiated txn PkgChange for client(id:cli dbus:1.70 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:10:14 fcos-01 rpm-ostree[11054]: Process [pid: 11049 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:10:14 fcos-01 rpm-ostree[11054]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 13:10:15 fcos-01 rpm-ostree[11054]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_coreos failed: Tweaking SELinux timestamps: Analyzing /usr/etc/selinux content: Cancelled
Jan 07 13:10:15 fcos-01 rpm-ostree[11054]: Unlocked sysroot
Jan 07 13:10:15 fcos-01 rpm-ostree[11054]: Process [pid: 11049 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:10:15 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.70 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:10:15 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 64 seconds
Jan 07 13:10:17 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.72 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:10:18 fcos-01 rpm-ostree[11054]: Locked sysroot
Jan 07 13:10:18 fcos-01 rpm-ostree[11054]: Initiated txn PkgChange for client(id:cli dbus:1.72 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:10:18 fcos-01 rpm-ostree[11054]: Process [pid: 12591 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_coreos failed: Packages not found: kubeadm-1.20
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: Unlocked sysroot
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: Process [pid: 12591 uid: 0 unit: (unknown)] disconnected from transaction progress
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.72 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:10:20 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 62 seconds
Jan 07 13:10:43 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.74 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:10:44 fcos-01 rpm-ostree[11054]: Locked sysroot
Jan 07 13:10:44 fcos-01 rpm-ostree[11054]: Initiated txn PkgChange for client(id:cli dbus:1.74 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:10:44 fcos-01 rpm-ostree[11054]: Process [pid: 13272 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:10:46 fcos-01 rpm-ostree[11054]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:10:47 fcos-01 rpm-ostree[11054]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_coreos failed: Could not depsolve transaction; 1 problem detected:
Jan 07 13:10:47 fcos-01 rpm-ostree[11054]: Unlocked sysroot
Jan 07 13:10:47 fcos-01 rpm-ostree[11054]: Process [pid: 13272 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:10:47 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.74 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:10:47 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 60 seconds
Jan 07 13:11:04 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.76 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:11:06 fcos-01 rpm-ostree[11054]: Locked sysroot
Jan 07 13:11:06 fcos-01 rpm-ostree[11054]: Initiated txn PkgChange for client(id:cli dbus:1.76 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:11:06 fcos-01 rpm-ostree[11054]: Process [pid: 13926 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:11:07 fcos-01 rpm-ostree[11054]: Ignoring pkgcache-only request in presence of module requests
Jan 07 13:11:08 fcos-01 rpm-ostree[11054]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:11:08 fcos-01 rpm-ostree[11054]: Executed %post for conntrack-tools in 90 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %post for kernel-tools in 97 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %post for tuned in 88 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %post for qemu-guest-agent in 90 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %post for cri-o in 90 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %posttrans for tuned in 49 ms
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 206 ms; 14793 matched files
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 43 ms; 824 matched files
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 41 ms; 26 matched files
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 39 ms; 70 matched files
Jan 07 13:11:09 fcos-01 rpm-ostree[11054]: sanitycheck(/usr/bin/true) successful
Jan 07 13:11:10 fcos-01 rpm-ostree[11054]: Regenerating rpmdb for target
Jan 07 13:11:14 fcos-01 rpm-ostree[11054]: Wrote commit: c00e58386fdfe9eacf17c03f01c50de77712b88a861bdd52c9910ff2893d28d8; New objects: meta:15 content:1 totaling 16.8 MB)
Jan 07 13:11:15 fcos-01 rpm-ostree[11054]: note: Deploying commit c00e58386fdfe9eacf17c03f01c50de77712b88a861bdd52c9910ff2893d28d8 which contains content in /var/lib that will be ignored.
Jan 07 13:11:17 fcos-01 rpm-ostree[11054]: Created new deployment /ostree/deploy/fedora-coreos/deploy/c00e58386fdfe9eacf17c03f01c50de77712b88a861bdd52c9910ff2893d28d8.0
Jan 07 13:11:19 fcos-01 rpm-ostree[11054]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:11:24 fcos-01 rpm-ostree[11054]: Unlocked sysroot
Jan 07 13:11:24 fcos-01 rpm-ostree[11054]: Process [pid: 13926 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:11:26 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.76 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:11:26 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 60 seconds
Jan 07 13:11:30 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.78 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:11:32 fcos-01 rpm-ostree[11054]: Locked sysroot
Jan 07 13:11:32 fcos-01 rpm-ostree[11054]: Initiated txn PkgChange for client(id:cli dbus:1.78 unit:session-2.scope uid:0): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:11:32 fcos-01 rpm-ostree[11054]: Process [pid: 17445 uid: 0 unit: session-2.scope] connected to transaction progress
Jan 07 13:11:34 fcos-01 rpm-ostree[11054]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:11:34 fcos-01 rpm-ostree[11054]: Downloading: https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/Packages/af3fcc9988dac9fda003fd43cf9a7578eef3b38271506b9885f06f42214ecf4f-kubeadm-1.21.8-0.x86_64.rpm
Jan 07 13:11:36 fcos-01 rpm-ostree[11054]: Imported 1 pkg
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %post for conntrack-tools in 114 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %post for kernel-tools in 103 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %post for tuned in 93 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %post for qemu-guest-agent in 87 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %post for cri-o in 90 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %posttrans for tuned in 49 ms
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 216 ms; 14795 matched files
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:11:37 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 47 ms; 824 matched files
Jan 07 13:11:38 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 40 ms; 26 matched files
Jan 07 13:11:38 fcos-01 rpm-ostree[11054]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 42 ms; 70 matched files
Jan 07 13:11:38 fcos-01 rpm-ostree[11054]: sanitycheck(/usr/bin/true) successful
Jan 07 13:11:38 fcos-01 rpm-ostree[11054]: Regenerating rpmdb for target
Jan 07 13:11:42 fcos-01 rpm-ostree[11054]: Wrote commit: 173c31dbc911c6302981c7b57556a3df7c0b50c785a131d782f324f8f6b7450c; New objects: meta:12 content:1 totaling 16.8 MB)
Jan 07 13:11:44 fcos-01 rpm-ostree[11054]: note: Deploying commit 173c31dbc911c6302981c7b57556a3df7c0b50c785a131d782f324f8f6b7450c which contains content in /var/lib that will be ignored.
Jan 07 13:11:46 fcos-01 rpm-ostree[11054]: Created new deployment /ostree/deploy/fedora-coreos/deploy/173c31dbc911c6302981c7b57556a3df7c0b50c785a131d782f324f8f6b7450c.0
Jan 07 13:11:48 fcos-01 rpm-ostree[11054]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:11:55 fcos-01 rpm-ostree[11054]: Unlocked sysroot
Jan 07 13:11:55 fcos-01 rpm-ostree[11054]: Process [pid: 17445 uid: 0 unit: session-2.scope] disconnected from transaction progress
Jan 07 13:11:58 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.78 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:11:58 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 64 seconds
Jan 07 13:12:01 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.80 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:12:04 fcos-01 rpm-ostree[11054]: client(id:cli dbus:1.80 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:12:04 fcos-01 rpm-ostree[11054]: In idle state; will auto-exit in 64 seconds
Jan 07 13:13:19 fcos-01 rpm-ostree[863]: Reading config file '/etc/rpm-ostreed.conf'
Jan 07 13:13:21 fcos-01 rpm-ostree[863]: In idle state; will auto-exit in 63 seconds
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Locked sysroot
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Initiated txn Deploy for client(dbus:1.15 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Process [pid: 2367 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Unlocked sysroot
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: Process [pid: 2367 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:13:22 fcos-01 rpm-ostree[863]: In idle state; will auto-exit in 60 seconds
Jan 07 13:13:24 fcos-01 rpm-ostree[863]: Locked sysroot
Jan 07 13:13:24 fcos-01 rpm-ostree[863]: Initiated txn Deploy for client(dbus:1.21 unit:zincati.service uid:978): /org/projectatomic/rpmostree1/fedora_coreos
Jan 07 13:13:24 fcos-01 rpm-ostree[863]: Process [pid: 3377 uid: 978 unit: zincati.service] connected to transaction progress
Jan 07 13:13:26 fcos-01 rpm-ostree[863]: libostree pull from 'fedora' for fedora/x86_64/coreos/stable complete
Jan 07 13:13:27 fcos-01 rpm-ostree[863]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 07 13:13:29 fcos-01 rpm-ostree[863]: Preparing pkg txn; enabled repos: ['fedora-cisco-openh264', 'fedora-modular', 'updates-modular', 'updates', 'fedora', 'kubernetes', 'updates-archive'] solvables: 103995
Jan 07 13:13:29 fcos-01 rpm-ostree[863]: Executed %post for conntrack-tools in 123 ms
Jan 07 13:13:29 fcos-01 rpm-ostree[863]: Executed %post for kernel-tools in 116 ms
Jan 07 13:13:29 fcos-01 rpm-ostree[863]: Executed %post for tuned in 108 ms
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %post for qemu-guest-agent in 117 ms
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %post for cri-o in 158 ms
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %posttrans for tuned in 60 ms
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: No files matched %transfiletriggerin(lib) for glibc-common
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: No files matched %transfiletriggerin(lib64) for glibc-common
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 242 ms; 14795 matched files
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %transfiletriggerin(shared-mime-info) for usr/share/mime in 66 ms; 824 matched files
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 57 ms; 26 matched files
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 84 ms; 70 matched files
Jan 07 13:13:30 fcos-01 rpm-ostree[863]: sanitycheck(/usr/bin/true) successful
Jan 07 13:13:31 fcos-01 rpm-ostree[863]: Regenerating rpmdb for target
Jan 07 13:13:35 fcos-01 rpm-ostree[863]: Wrote commit: eb470b633bc58674ea3d44f24ecf2ec42a2abfff4902539b429589715e12b4e6; New objects: meta:6 content:1 totaling 16.8 MB)
Jan 07 13:13:37 fcos-01 rpm-ostree[863]: note: Deploying commit eb470b633bc58674ea3d44f24ecf2ec42a2abfff4902539b429589715e12b4e6 which contains content in /var/lib that will be ignored.
Jan 07 13:13:39 fcos-01 rpm-ostree[863]: Created new deployment /ostree/deploy/fedora-coreos/deploy/eb470b633bc58674ea3d44f24ecf2ec42a2abfff4902539b429589715e12b4e6.0
Jan 07 13:13:42 fcos-01 rpm-ostree[863]: Txn Deploy on /org/projectatomic/rpmostree1/fedora_coreos successful
Jan 07 13:13:45 fcos-01 rpm-ostree[863]: Unlocked sysroot
Jan 07 13:13:45 fcos-01 rpm-ostree[863]: Process [pid: 3377 uid: 978 unit: zincati.service] disconnected from transaction progress
Jan 07 13:13:45 fcos-01 rpm-ostree[863]: In idle state; will auto-exit in 62 seconds
Jan 07 13:13:58 fcos-01 rpm-ostree[863]: client(id:cli dbus:1.68 unit:session-2.scope uid:0) added; new total=1
Jan 07 13:14:00 fcos-01 rpm-ostree[863]: client(id:cli dbus:1.68 unit:session-2.scope uid:0) vanished; remaining=0
Jan 07 13:14:00 fcos-01 rpm-ostree[863]: In idle state; will auto-exit in 60 seconds
Jan 07 13:15:00 fcos-01 rpm-ostree[863]: In idle state; will auto-exit in 63 seconds

If you think other logs would help, let me know.

I was not aware of the --reboot parameter, I’ll give that a shot next time around.

Appreciate your help!
Ray

Hey @rayray221 - we kind of need the logs (all logs) from the reboot to see if something failed on the way down. If you can upload it somewhere like https://paste.centos.org/.

What does sudo rpm-ostree status --verbose show?

Hello @dustymabe, with the new release, here is what sudo rpm-ostree status --verbose shows:

State: idle
AutomaticUpdatesDriver: Zincati (zincati.service)
  DriverState: active; update staged: 35.20220103.3.0; reboot pending due to update strategy
Deployments:
  fedora:fedora/x86_64/coreos/stable
                   Version: 35.20220103.3.0 (2022-01-17T15:41:41Z)
                BaseCommit: 83f5d39fbf539a1e5edd21747899480e11710b351d25d5790e15e1930f51e2b9
                            └─ fedora-coreos-pool (2022-01-16T21:39:56Z)
                    Commit: 3a0293981a360ea954dd2fa711eb31d3384ca26dba1a8169bca157444db551c8
                            ├─ fedora-cisco-openh264 (2021-09-21T18:07:30Z)
                            ├─ fedora-modular (2021-10-26T05:08:36Z)
                            ├─ updates-modular (2022-01-06T01:57:22Z)
                            ├─ updates (2022-01-19T01:42:17Z)
                            ├─ fedora (2021-10-26T05:31:27Z)
                            ├─ kubernetes ((invalid timestamp))
                            └─ updates-archive (2022-01-19T02:49:26Z)
                    Staged: yes
                 StateRoot: fedora-coreos
              GPGSignature: 1 signature
                            Signature made Mon Jan 17 15:48:45 2022 using RSA key ID DB4639719867C58F
                            Good signature from "Fedora <fedora-35-primary@fedoraproject.org>"
                  Upgraded: bind-libs 32:9.16.23-1.fc35 -> 32:9.16.24-1.fc35
                            bind-license 32:9.16.23-1.fc35 -> 32:9.16.24-1.fc35
                            bind-utils 32:9.16.23-1.fc35 -> 32:9.16.24-1.fc35
                            ca-certificates 2021.2.50-3.fc35 -> 2021.2.52-1.0.fc35
                            console-login-helper-messages 0.21.2-2.fc35 -> 0.21.2-3.fc35
                            console-login-helper-messages-issuegen 0.21.2-2.fc35 -> 0.21.2-3.fc35
                            console-login-helper-messages-motdgen 0.21.2-2.fc35 -> 0.21.2-3.fc35
                            console-login-helper-messages-profile 0.21.2-2.fc35 -> 0.21.2-3.fc35
                            coreos-installer 0.12.0-1.fc35 -> 0.12.0-2.fc35
                            coreos-installer-bootinfra 0.12.0-1.fc35 -> 0.12.0-2.fc35
                            crun 1.3-1.fc35 -> 1.4-1.fc35
                            fwupd 1.7.2-1.fc35 -> 1.7.3-1.fc35
                            gnupg2 2.3.3-2.fc35 -> 2.3.4-1.fc35
                            libxcrypt 4.4.26-4.fc35 -> 4.4.27-1.fc35
                            libxcrypt-compat 4.4.26-4.fc35 -> 4.4.27-1.fc35
                            libzstd 1.5.0-2.fc35 -> 1.5.1-3.fc35
                            linux-firmware 20211027-126.fc35 -> 20211216-127.fc35
                            linux-firmware-whence 20211027-126.fc35 -> 20211216-127.fc35
                            podman 3:3.4.2-1.fc35 -> 3:3.4.4-1.fc35
                            podman-plugins 3:3.4.2-1.fc35 -> 3:3.4.4-1.fc35
                            python-unversioned-command 3.10.1-2.fc35 -> 3.10.1-3.fc35
                            python3 3.10.1-2.fc35 -> 3.10.1-3.fc35
                            python3-libs 3.10.1-2.fc35 -> 3.10.1-3.fc35
                            selinux-policy 35.6-1.fc35 -> 35.7-1.fc35
                            selinux-policy-targeted 35.6-1.fc35 -> 35.7-1.fc35
           LayeredPackages: cri-o kubeadm-1.21.8 kubectl-1.21.8 kubelet-1.21.8 qemu-guest-agent tuned
            EnabledModules: cri-o:1.21

● fedora:fedora/x86_64/coreos/stable
                   Version: 35.20211215.3.0 (2022-01-04T18:57:51Z)
                BaseCommit: 30c82ee684674b9a552ffee709501f981f35f36408085f089686e43b09aeca1b
                            └─ fedora-coreos-pool (2022-01-03T21:39:36Z)
                    Commit: e9a708b7bcb2bebe08909fb572c033706a85b47d3c3c00da4ed342f59e8566ca
                            ├─ fedora-cisco-openh264 (2021-09-21T18:07:30Z)
                            ├─ fedora-modular (2021-10-26T05:08:36Z)
                            ├─ updates-modular (2022-01-06T01:57:22Z)
                            ├─ updates (2022-01-08T01:04:36Z)
                            ├─ fedora (2021-10-26T05:31:27Z)
                            ├─ kubernetes ((invalid timestamp))
                            └─ updates-archive (2022-01-08T01:31:50Z)
                 StateRoot: fedora-coreos
              GPGSignature: 1 signature
                            Signature made Tue Jan  4 19:06:54 2022 using RSA key ID DB4639719867C58F
                            Good signature from "Fedora <fedora-35-primary@fedoraproject.org>"
           LayeredPackages: cri-o kubeadm-1.21.8 kubectl-1.21.8 kubelet-1.21.8 qemu-guest-agent tuned
            EnabledModules: cri-o:1.21

  fedora:fedora/x86_64/coreos/stable
                   Version: 35.20211215.3.0 (2022-01-04T18:57:51Z)
                BaseCommit: 30c82ee684674b9a552ffee709501f981f35f36408085f089686e43b09aeca1b
                            └─ fedora-coreos-pool (2022-01-03T21:39:36Z)
                    Commit: 892a1566daecae54b2347df1fcedaf429999845c93f43f72c1e923dadebd9294
                            ├─ fedora-cisco-openh264 (2021-09-21T18:07:30Z)
                            ├─ fedora-modular (2021-10-26T05:08:36Z)
                            ├─ updates-modular (2022-01-06T01:57:22Z)
                            ├─ updates (2022-01-08T01:04:36Z)
                            ├─ fedora (2021-10-26T05:31:27Z)
                            ├─ kubernetes ((invalid timestamp))
                            └─ updates-archive (2022-01-08T01:31:50Z)
                 StateRoot: fedora-coreos
              GPGSignature: 1 signature
                            Signature made Tue Jan  4 19:06:54 2022 using RSA key ID DB4639719867C58F
                            Good signature from "Fedora <fedora-35-primary@fedoraproject.org>"
           LayeredPackages: cri-o kubeadm-1.21.8 kubectl-1.21.8 kubelet-1.21.8 qemu-guest-agent tuned
            EnabledModules: cri-o:1.20

I then ran:

sudo rpm-ostree cleanup -p

Followed by:

sudo rpm-ostree upgrade --bypass-driver --reboot

When the server comes back up, it is still running “35.20211215.3.0 (2022-01-04T18:57:51Z)” and does not upgrade.

I posted the logs from the reboot:
https://paste.centos.org/view/c6c7ec25

Looks like they will delete the logs after 1 day, happy to post them again if needed.

Thanks again for looking into this.
Ray

OK this is interesting:

Jan 21 12:36:28 fcos-01 ostree[2408253]: Not finalizing; found /run/ostree/staged-deployment-locked                                                                                                                      
Jan 21 12:36:28 fcos-01 systemd[1]: ostree-finalize-staged.service: Deactivated successfully.                                                                                                                            
Jan 21 12:36:28 fcos-01 systemd[1]: Stopped OSTree Finalize Staged Deployment.

When Zincati deploys an update, it keeps it “locked”. This means that if the system is restarted (e.g. power outage) we still don’t go into the update. Only when Zincati itself triggers the reboot as per the configured schedule does it unlock first to allow booting into the new deployment. This is the staged-deployment-locked message in Dusty’s printout.

If you want to have full manual control over updates, you can systemctl disable --now zincati first. If Zincati had previously run, it might’ve already laid down the lock, which you could manually delete.

The UX here of course is not ideal. We’re tracking enhancements for “manual overrides” in RFE: Allow manual update checks and reboots · Issue #498 · coreos/zincati · GitHub.

After some more discussion the rpm-ostree cleanup -p should have deleted the /run/ostree/staged-deployment-locked file. Fix in:

Still you should be able to get out of this. @rayray221 try this now:

sudo systemctl stop zincati
sudo rpm-ostree cleanup -p
sudo rm /run/ostree/staged-deployment-locked
sudo rpm-ostree upgrade --reboot

Thanks @jlebon and @dustymabe, I really appreciate the help and clarity on how these technologies work together.

Dusty, the above steps do indeed work in order to manually update!

Thanks so much for helping me understand this and how I can update outside the maintenance windows.
Ray

1 Like