And I found the problem was caused by wrong user:group of the file /var/cache/ddclient/ddclient.cache (which should be ddclient:ddclient instead of clevis):
-rw-------. 1 clevis 960 system_u:object_r:ddclient_var_t:s0 614 Dec 30 14:08 /var/cache/ddclient/ddclient.cache
I also found user:group of the file /etc/ddclient.conf is wrong:
-rw-------. 1 clevis 960 system_u:object_r:ddclient_etc_t:s0 10410 Oct 5 08:53 /etc/ddclient.conf
It seems that the user:group id of ddclient:ddclient has changed.
While the problem can be solved by running chown on the files. I still don’t know why suddenly user:group id changed. Anyone has any clues?
I am using silverblue fc41. By checking the journalctl log, I found the error occurred after I updated the system, so it looks like the scripts in the rpm did this.
By inspecting the contents of /etc/passwd, /etc/group and /usr/lib/passwd, /usr/lib/group, I found that clevis user/group is defned in /etc/passwd and /etc/group while ddclient user/group is defned in the /usr/lib/passwd and /usr/lib/group.
$ grep ddclient /usr/lib/passwd
dclient:x:963:962:Dynamic DNS Client:/var/cache/ddclient:/sbin/nologin
$ grep ddclient /usr/lib/group
ddclient:x:962:
I think system users/groups in silverblue should all go to /usr/lib/passwd and /usr/lib/group. Maybe it is an upstream bug that handled user/group ids incorretly?
After reading the github issue, I have some clues why this happened but still not very sure.
I occasionally run sudo rpm-ostree apply-live --allow-replacement after rpm-ostree upgrade to avoid rebooting my system every time.
I think maybe rpm-ostree does not handle user/groupd id allocation correctly when executing apply-live, which caused clevis scripts in the rpm modified /etc/passwd and /etc/group instead of /usr/lib/passwd and /usr/lib/group. Therefore clevis appeared in /etc/passwd and /etc/group instead of /usr/lib/passwd and /usr/lib/group.
So I manually removed clevis from /etc/passwd and /etc/group and uninstalled some layered packages to enable rpm-ostree to update my system. After rebooting, clevis user/group now is defined in /usr/lib/passwd and /usr/lib/group. However, I am not sure this is the real cause because I can not reproduce this bug by running apply-live.
Ok, this error occurred again.
This time I did not run apply-live, so my geuss was wrong.
I only rpm-ostree upgrade and the error occasionally happens.
I think system users/groups in silverblue should all go to /usr/lib/passwd and /usr/lib/group. Maybe it is an upstream bug that handled user/group ids incorretly?