Errors running podman containers on Fedora IoT 35

Ask Fedora
, ,
1

I am running Fedora IoT on a raspberry pi 4 and I am running some LinuxServer.io containers.

Everything is working fine for fedora 34 versions of fedora iot. But for version 35 I am getting strange permission errors! Such as:

Error relocating ./run: RELRO protection failed: Permission deniedError relocating ./run: RELRO protection failed: Permission denied

Error relocating /usr/lib/libreadline.so.8: RELRO protection failed: Permission deniedError relocating /usr/lib/libreadline.so.8: RELRO protection failed: Permission denied

I have found this github issue with similar errors. This suggests that the problem is SELinux related but I can’t be sure.

Some details about my system:

$ rpm-ostree status
State: idle
Deployments:
  fedora-iot:fedora/stable/aarch64/iot
                   Version: 34.20211019.0 (2021-10-19T15:13:36Z)
                BaseCommit: edf0f814bb325ccf75f4dd10cd98f3687fc4c32085884bbc4c50d8ed954e7836
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
                      Diff: 387 downgraded, 7 removed, 14 added
           LayeredPackages: python3-pyserial cockpit-podman usbutils cockpit cockpit-ostree nano python3-libgpiod
                    Pinned: yes

* fedora-iot:fedora/stable/aarch64/iot
                   Version: 35.20211108.0 (2021-11-08T08:12:21Z)
                BaseCommit: 037d7a07010a0a616723110b2225bb15a32d5b3fb1e2982ed1988684e2e2448c
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: python3-pyserial cockpit-podman usbutils cockpit cockpit-ostree nano python3-libgpiod

The above status is after I run rollback to move to the Fedora-IoT-34 version that works fine.

Any ideas on how I can solve this issue? Do I need to file a bug request?

2

Hello @iolaum ,
I think you should go to this issue and state your problem there. It is a recurring issue that keeps getting closed after fixing, it would seem nearly every Fedora Linux release. Which to me would indicate the symptom and not the root cause is being fixed.

1 Like
3

Thank for the suggestion @jakfrost

I posted about the issue here as you suggested.

4

Hello @iolaum ,
Sorry for that response, the individual could have at least provided a link to you when you filed the issue.
I checked bugzilla like mentioned on github in the comment you rec’d. I did find this which seems similar and is relevant to F35 and IoT and Podman + Rootless containers. It is at https://bugzilla.redhat.com/show_bug.cgi?id=2019324, and would seem to be related to your issue.

2 Likes
5

Thanks for your reply again @jakfrost

Thank you also for looking for a relevant bug, I don’t think I could have connected the two by looking myself.

I posted on that bug report which will hopefully help developers address the issue.

6

Posted that the issue is still happening with the latest aarch64 iot update.

The issue still hasn’t had an update. Is there any way to know if the issue has ended up on someone’s todo list, or if another action is needed before that?

7

Does the container run if you temporarily disable selinux?

Are you getting any avc denials?

https://docs.fedoraproject.org/en-US/quick-docs/troubleshooting_selinux/

8

Yes, when I make selinux permissive containers are running fine (after I restart them, some errors pop up if I don’t).

I am able to see an error in cockpit if I make selinux enforcing and run this command:

$ podman run --rm -ti fedora:35 bash

But I don’t know what to make of it:

Entry at Dec 12, 2021, 4:21:12 PM

kernel

audit: type=1327 audit(1639318872.013:1483): proctitle="bash"
PRIORITY    5
SYSLOG_FACILITY    0
SYSLOG_IDENTIFIER    kernel
_BOOT_ID    6458a01509c248deafd57e463b7151e4
_HOSTNAME    fedora-iot1
_MACHINE_ID    4514d1e3b25f45258eebfb08bbe686ba
_SOURCE_MONOTONIC_TIMESTAMP    951214855
_TRANSPORT    kernel
__CURSOR    s=0e7f393743bd432a85745f06b57e722e;i=2792;b=6458a01509c248deafd57e463b7151e4;m=38b22ba2;t=5d2f3ab0a7dda;x=7add39133104ffd3
__MONOTONIC_TIMESTAMP    951200674
__REALTIME_TIMESTAMP    1639318872030682

Note it’s not actually an error but a notice (I have to enable Notice level and above in cockpit logs to see this message).

Not sure what AVC denials means.

Anyone with a rpi4 running Fedora-iot 35.xxx should be able to reproduce the issue with podman run --rm -ti fedora:35 bash.

9

They’re selinux denials in /var/log/audit/audit.log

Have you already done, restorecon -R ~/.local/share/containers/storage/overlay*

Thanks

10

Thanks for the suggestions. I 'll look into it and come back with more details.

11

Had some time too look into it today. And @grumpey 's solution worked! Thank you very much!

I m not sure what was the problem. From what restorecon appears to do I m guessing that SELinux content for those files was different in F34 compared to F35 and something didn’t move over properly to the newer versions of ‘stuff’.

Here’s a minimal example of my testing:

$ rpm-ostree status
State: idle
Deployments:
* fedora-iot:fedora/stable/aarch64/iot
                   Version: 35.20211212.1 (2021-12-12T21:08:43Z)
                BaseCommit: 6e5da88473ff059d6fb9bc11a39ad2610474178a16b0397ef56f697569f86efd
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: cockpit cockpit-ostree cockpit-podman nano python3-libgpiod python3-pyserial usbutils

  fedora-iot:fedora/stable/aarch64/iot
                   Version: 34.20211019.0 (2021-10-19T15:13:36Z)
                BaseCommit: edf0f814bb325ccf75f4dd10cd98f3687fc4c32085884bbc4c50d8ed954e7836
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: cockpit cockpit-ostree cockpit-podman nano python3-libgpiod python3-pyserial usbutils
                    Pinned: yes

$ podman run --rm -ti fedora:34 bash
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:34...
Getting image source signatures
Copying blob a11f831469cd done  
Copying config 71280d2620 done  
Writing manifest to image destination
Storing signatures
bash: error while loading shared libraries: libtinfo.so.6: cannot change memory protections

$ sudo restorecon -R ~/.local/share/containers/storage/overlay*
$ podman run --rm -ti fedora:34 bash
# /