NTP broken on Silverblue 29?


#1

I’m enjoying the Silverblue 29 beta a lot, but it seems like I’ve been unable to turn on NTP on any systems (reproducible on multiple hosts).

Attempting to turn on “Automatic Date & Time” in the GNOME Settings just results in the toggle switching itself back off.

Similarly, trying to start the systemd-timesyncd service raises the following error:

Oct 12 02:11:09 X240 systemd[11091]: systemd-timesyncd.service: Failed at step STATE_DIRECTORY spawning /usr/lib/systemd/systemd-timesyncd: Permission denied
-- Subject: Process /usr/lib/systemd/systemd-timesyncd could not be executed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- The process /usr/lib/systemd/systemd-timesyncd could not be executed and failed.
-- 
-- The error number returned by this process is 13.
Oct 12 02:11:09 X240 systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=238/STATE_DIRECTORY
Oct 12 02:11:09 X240 systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Oct 12 02:11:09 X240 systemd[1]: Failed to start Network Time Synchronization.

There seems to be some kind of permissions error, but I don’t think it’s an SELinux issue as the audit log shows:

type=SERVICE_START msg=audit(1539306880.140:439): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-timedated comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

which clearly states res=success (if I’m reading it right).


#2

This looks like https://bugzilla.redhat.com/show_bug.cgi?id=1559286

I dug around the GitHub issue linked from there and ended up taking a look at the spec file for systemd. It looks like there are a number of directories/files that should be created by the install of systemd-timesyncd that don’t exist.

https://src.fedoraproject.org/rpms/systemd/blob/f28/f/systemd.spec#_398-405

I tried creating those manually, but it didn’t seem to help anything. :frowning:

# mkdir -p /var/lib/private/systemd/timesync                                                               
# ln -s /var/lib/private/systemd/timesync/ /var/lib/systemd/timesync/                                                                                                                          
# touch /var/lib/private/systemd/timesync/clock  

# systemctl start systemd-timesyncd                              
Job for systemd-timesyncd.service failed because the control process exited with error code.                                                                                                                      
See "systemctl status systemd-timesyncd.service" and "journalctl -xe" for details.                                                                                                                                
[root@mastershake ~]# systemctl status systemd-timesyncd.service             
● systemd-timesyncd.service - Network Time Synchronization                      
   Loaded: loaded (/usr/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: disabled)                                                                                                           
   Active: failed (Result: exit-code) since Fri 2018-10-12 14:19:14 EDT; 7s ago                                                                                                                                   
     Docs: man:systemd-timesyncd.service(8)                                                                                                                                                                       
  Process: 472 ExecStart=/usr/lib/systemd/systemd-timesyncd (code=exited, status=238/STATE_DIRECTORY)
 Main PID: 472 (code=exited, status=238/STATE_DIRECTORY)                             
                                                                                                                                                                                                                  
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Service has no hold-off time, scheduling restart.                                                                                              
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 5.
Oct 12 14:19:14 mastershake systemd[1]: Stopped Network Time Synchronization.                         
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Start request repeated too quickly.
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Oct 12 14:19:14 mastershake systemd[1]: Failed to start Network Time Synchronization.
[root@mastershake ~]# journalctl --since 14:19:14 -u systemd-timesyncd
-- Logs begin at Mon 2018-04-30 16:12:58 EDT, end at Fri 2018-10-12 14:19:14 EDT. --
Oct 12 14:19:14 mastershake systemd[1]: Starting Network Time Synchronization...
Oct 12 14:19:14 mastershake systemd[454]: systemd-timesyncd.service: Failed to set up special execution directory in /var/lib: File exists
Oct 12 14:19:14 mastershake systemd[454]: systemd-timesyncd.service: Failed at step STATE_DIRECTORY spawning /usr/lib/systemd/systemd-timesyncd: File exists
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=238/STATE_DIRECTORY
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Oct 12 14:19:14 mastershake systemd[1]: Failed to start Network Time Synchronization.
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Service has no hold-off time, scheduling restart.
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 1.
Oct 12 14:19:14 mastershake systemd[1]: Stopped Network Time Synchronization.
Oct 12 14:19:14 mastershake systemd[1]: Starting Network Time Synchronization...
Oct 12 14:19:14 mastershake systemd[458]: systemd-timesyncd.service: Failed to set up special execution directory in /var/lib: File exists
Oct 12 14:19:14 mastershake systemd[458]: systemd-timesyncd.service: Failed at step STATE_DIRECTORY spawning /usr/lib/systemd/systemd-timesyncd: File exists
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=238/STATE_DIRECTORY
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Oct 12 14:19:14 mastershake systemd[1]: Failed to start Network Time Synchronization.
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Service has no hold-off time, scheduling restart.
Oct 12 14:19:14 mastershake systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 2.
Oct 12 14:19:14 mastershake systemd[1]: Stopped Network Time Synchronization.
...

I’ve tried removing all the directories under /var/lib/private and various permutations, but it just brings us back to the original problem of Failed to set up special execution directory in /var/lib: File exists

It might be worth a shot to file another BZ about this specific problem on Silverblue or maybe a GitHub issue.


#3

Thanks for attempting to investigate! Good to see the issue is reproducible for more than just myself, even if there’s no simple fix.

It might be worth a shot to file another BZ about this specific problem on Silverblue or maybe a GitHub issue.

Is Red Hat’s Bugzilla the right place for Silverblue issues, or should I be using the Pagure tracker? I’m not familiar with Pagure but it looks easy enough that I could file a bug tomorrow for this issue if I have the time.


#4

Start with the Silverblue Pagure tracker and we can go from there