Changing user UID/GID broke toolbox/podman on Fedora 37 Silverblue

Ask Fedora
, , ,
1

Happy New Year everyone! It’s only the 3rd and I already broke my new install.

After changing the UID/GID I cannot run toolbox or Podman anymore.

➜  ~ id
uid=1985(mihalyr) gid=1985(mihalyr) groups=1985(mihalyr),10(wheel) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
➜  ~ toolbox enter
Error: failed to get the Podman version
➜  ~ podman version
Error: creating tmpdir: mkdir /run/user/1000: permission denied

This is a fresh F37 Silverblue installed with the installer. I created the user with the first setup wizard with UID 1000 as usual. But I have old backups to restore and I use my custom UID/GID for my users to work simpler across systems. So, after finishing the basic setup and confirming it works, I went on to change the UID/GID from 1000 to 1985.

usermod -u 1985 mihalyr
groupmod -g 1985 mihalyr
find /home/mihalyr -user 1000 -exec chown mihalyr {} \;
find /home/mihalyr -group 1000 -exec chgrp mihalyr {} \;

Actually, I ended up running the find for all the mounts, just in case.

After this everything Just Works™. At least I thought, until I tried toolbox.

This is a very similar issue here which is closed without a solution that works for me: podman stop work after user ID change · Issue #15310 · containers/podman · GitHub

They suggest to run podman system reset

➜  ~ podman system reset
Error: creating tmpdir: mkdir /run/user/1000: permission denied
➜  ~ sudo podman system reset
[sudo] password for mihalyr: 
WARNING! This will remove:
        - all containers
        - all pods
        - all images
        - all networks
        - all build cache
        - all machines
        - all volumes
Are you sure you want to continue? [y/N] y
A storage.conf file exists at /usr/share/containers/storage.conf
You should remove this file if you did not modify the configuration.

Then I tried to use the overlay to force this reset, but didn’t help:

➜  ~ sudo rpm-ostree usroverlay
Development mode enabled.  A writable overlayfs is now mounted on /usr.
All changes there will be discarded on reboot.
➜  ~ rm /usr/share/containers/storage.conf
rm: remove write-protected regular file '/usr/share/containers/storage.conf'? y
rm: cannot remove '/usr/share/containers/storage.conf': Permission denied
➜  ~ sudo rm /usr/share/containers/storage.conf
➜  ~ sudo podman system reset
WARNING! This will remove:
        - all containers
        - all pods
        - all images
        - all networks
        - all build cache
        - all machines
        - all volumes
Are you sure you want to continue? [y/N] y
➜  ~ podman version
ERRO[0000] User-selected graph driver "vfs" overwritten by graph driver "overlay" from database - delete libpod local files to resolve.  May prevent use of images created by other tools 
Error: creating tmpdir: mkdir /run/user/1000: permission denied

And after a restart, I’m back where I started again. I’m running out of ideas on how to fix this. Any help is appreciated.

2

Oh, I just had to look a little bit further, the hint was in the last podman output:

➜  ~ podman version
ERRO[0000] User-selected graph driver "vfs" overwritten by graph driver "overlay" from database - delete libpod local files to resolve.  May prevent use of images created by other tools 
Error: creating tmpdir: mkdir /run/user/1000: permission denied

I googled for libpod local files and found ERRO[0000] User-selected graph driver "overlay" overwritten by graph driver "vfs" from database - delete libpod local files to resolve · Issue #5114 · containers/podman · GitHub

sudo rm -rf ~/.local/share/containers/

This was all I needed, I did a systemctl reboot to exit the usroverlay to confirm and everything works now again, I can use toolbox!

Sorry for bothering, and again, a very happy new year to everyone!

4 Likes
3

You are not bothering, you are actively helping and debugging issues.
Happy new yer to.