Openvpn in podman: /dev/net/tun Operation not permitted

Ask Fedora
1

I run an openvpn client inside a podman container (with podman-compose:sudo podman-compose -f compose.yaml up -d).

Everything worked fine until recently. Now running the command above, openvpn fails with an error :

ERROR: Cannot open TUN/TAP dev /dev/net/tun: Operation not permitted (errno=1)

Last time I successfully started the containers was about two weeks ago and I have not modified any configs since.

I think maybe some recent update (e.g. kernel or selinux or podman) broke my setup. I am running podman 5.2.5 and in silverblue 41 with kernel 6.11.6-300.fc41.x86_64. Does anyone have any clues?

Thanks!

2

My compose.yaml (with some unrelated services removed):

version: "3"

services:
  vpn:
    build:
      context: ./openvpn
    volumes:
      - ./servers:/servers:Z
      - ./auth:/auth:Z
    networks:
      network:
        ipv4_address: 192.168.120.79
    dns:
      - 1.1.1.1
      - 8.8.8.8
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv6.conf.all.disable_ipv6=1
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun:/dev/net/tun
    restart: unless-stopped

networks:
  network:
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.120.0/24
          gateway: 192.168.120.1
    driver_opts:
      com.docker.network.bridge.name: myvpn

My workaround is to add privileged: true in compose.yaml file. Then everything works again. But I still have no idea what permission is missing.

3

You should be able to remove privileged if if you use CAP_NET_ADMIN and CAP_NET_RAW, at least it worked for me

4

Thank you. I tried the solution but the problem still existed after I removed the privileged flag.

I tried the following command, but it failed (/dev/net/tun Operation not permitted) if I run /usr/sbin/openvpn inside the container:

sudo podman run --rm -ti --device /dev/net/tun --cap-add NET_ADMIN --cap-add NET_RAW -v $PWD/servers:/servers:Z -v $PWD/auth:/auth:Z --entrypoint /bin/bash localhost/myvpn_vpn

Currently only this command worked for me:

sudo podman run --rm -ti --device /dev/net/tun --privileged -v $PWD/servers:/servers:Z -v $PWD/auth:/auth:Z --entrypoint /bin/bash localhost/myvpn_vpn

the image localhost/myvpn_vpn is built from

FROM alpine:latest

RUN apk add --no-cache \
    bash \
    iptables \
    openresolv \
    openvpn

ADD update-resolv-conf /etc/openvpn/update-resolv-conf
ADD entrypoint.sh /entrypoint.sh

WORKDIR /etc/openvpn
ENTRYPOINT ["/entrypoint.sh"]

:dizzy_face: Everything worked fine until some update broke it, but I don’t know specifically which one.

5

You might have a wrong capability, ie for podman run command try: --cap-add=CAP_NET_ADMIN,CAP_NET_RAW without --privileged

Compose syntax might be different for the same capabilities

6

Still not working. I checked the capabilities of the bash process (PID 7737) of the container using the command sudo getpcaps 7737. It has the following capabilities:

7737: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_setfcap=ep

But running /usr/sbin/openvpn still gave the same error.

7

After recent update, the bug has been fixed in the upstream. But I think I still should write it down here.

After updating the system, the container without privileged: true flag worked again without producing the error. By rolling back to the previous deployment, and the error reappeared. So the culprit should be one of the following packages (output by rpm-ostree db diff):

ostree diff commit from: rollback deployment (36488a7ecd070b9f71cf3c7b9f6be8b1bcc957eea483d73bb1eaa981e38efff2)
ostree diff commit to:   booted deployment (f0f9362b63bfd5642a19a1a1a53aa618e422ab8aa8e98a6fd2df1e461084a483)
Upgraded:
  SDL2 2.30.3-2.fc41 -> 2.30.9-1.fc41
  crun 1.18.1-1.fc41 -> 1.19-1.fc41
  cups 1:2.4.11-7.fc41 -> 1:2.4.11-8.fc41
  cups-client 1:2.4.11-7.fc41 -> 1:2.4.11-8.fc41
  cups-filesystem 1:2.4.11-7.fc41 -> 1:2.4.11-8.fc41
  cups-ipptool 1:2.4.11-7.fc41 -> 1:2.4.11-8.fc41
  cups-libs 1:2.4.11-7.fc41 -> 1:2.4.11-8.fc41
  gst-editing-services 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugin-libav 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-bad-free 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-bad-free-extras 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-bad-free-libs 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-base 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-good 1.24.9-3.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-good-qt 1.24.9-3.fc41 -> 1.24.10-1.fc41
  gstreamer1-plugins-ugly-free 1.24.9-1.fc41 -> 1.24.10-1.fc41
  gstreamer1-vaapi 1.24.9-1.fc41 -> 1.24.10-1.fc41
  libdrm 2.4.123-1.fc41 -> 2.4.124-1.fc41
  llvm-libs 19.1.4-1.fc41 -> 19.1.5-1.fc41
  netavark 2:1.13.0-1.fc41 -> 2:1.13.1-1.fc41
  openldap 2.6.8-5.fc41 -> 2.6.8-6.fc41
  pixman 0.44.0-0.fc41 -> 0.44.2-1.fc41
  python3-argcomplete 3.5.1-1.fc41 -> 3.5.2-1.fc41
  qpdf-libs 11.9.1-2.fc41 -> 11.9.1-3.fc41
  rpm-sequoia 1.7.0-2.fc41 -> 1.7.0-3.fc41
  vim-data 2:9.1.895-1.fc41 -> 2:9.1.906-1.fc41
  vim-minimal 2:9.1.895-1.fc41 -> 2:9.1.906-1.fc41

I think netavark or crun might be the culprit because they are related to the container and their change logs show its update date is approximately when I encountered the problem:

Output of rpm -q --changelog crun:

* Fri Dec 06 2024 Packit <hello@packit.dev> - 1.19-1
- Update to 1.19 upstream release

* Thu Oct 31 2024 Packit <hello@packit.dev> - 1.18.2-1
- Update to 1.18.2 upstream release

* Wed Oct 30 2024 Packit <hello@packit.dev> - 1.18.1-1
- Update to 1.18.1 upstream release

Output of rpm -q --changelog netavark:

* Thu Dec 05 2024 Packit <hello@packit.dev> - 2:1.13.1-1
- Update to 1.13.1 upstream release

* Tue Oct 29 2024 Packit <hello@packit.dev> - 2:1.13.0-1
- Update to 1.13.0 upstream release

* Mon Aug 19 2024 Packit <hello@packit.dev> - 2:1.12.2-1
- Update to 1.12.2 upstream release