Rootless podman wireguard container unable to start: no such file or directory

I’m trying to set up a wireguard container using linuxserver/docker-wireguard and podman-compose. The compose file looks like this:

---
services:
  wireguard:
    image: lscr.io/linuxserver/wireguard:latest
    cap_add:
      - NET_ADMIN
    container_name: wireguard
    environment:
      - PUID=1001
      - PGID=1001
      - TZ=Etc/UTC
    volumes:
      - /var/home/container/qbittorrentvpn/wireguard-config:/config:Z
    ports:
      - 1111:1111
      - 6881:6881
      - 6881:6881/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

When I try to start this with podman-compose up (running as the same user the container will itself run as), I get this error:

Error: unable to start container 7deaf440a78dffd24be3648629fe70e755b99d357c1bb7136cea80d15c05e00a: crun: open `/var/home/container/.local/share/containers/storage/overlay/a48ba50912157972110d5f5e751a9c517755a10e174440b8dde9e05f1c4679c2/merged/etc/resolv.conf`: No such file or directory: OCI runtime attempted to invoke a command that was not found
exit code: 125

I have no idea what’s going on here. Should I use a different container? If so, what’s recommended?

Why do you need a container for wireguard?
All I have needed to consider is the firewall rules.

The error seems to be saying that resolve.conf is being run as a command.

/var/home/container/.local/share/containers/storage/overlay/a48ba50912157972110d5f5e751a9c517755a10e174440b8dde9e05f1c4679c2/merged/etc/resolv.conf does not exist. This suggests that there might be an issue with the volume mapping for the container’s configuration directory /config.

Make sure that the host directory /var/home/container/qbittorrentvpn/wireguard-config contains the necessary configuration files for WireGuard. verify that the directory has the correct permissions so that the container can read from it. Also, NET_ADMIN capability is correctly added to the container.

try running the container directly with Podman commands podman run to see if you encounter the same issue.

1 Like

I want to route all the traffic of another container through a VPN, while letting all the other traffic on my server use the regular connection, and the simplest and most reliable way of doing that that I know of is setting up a vpn container and using it as the network for the other container. Also, im on fedora coreos so I’d have to layer to install vpn stuff on the bare metal.

1 Like

I’m pretty sure this container is supposed to create those files on first boot, but I’ll see what happens when I create them manually

im pretty sure both of those hold true

Added coreos

Hi! That is a strange error message, why would it try to execute /var/home/container/.local/share/containers/storage/overlay/a48ba50912157972110d5f5e751a9c517755a10e174440b8dde9e05f1c4679c2/merged/etc/resolv.conf?

Could you give us some more information? Do you already have config for this container inside /var/home/container/qbittorrentvpn/wireguard-config? Could the error be in the config? The docker-compose looks very minimal, I doubt the error is in there.

The setup sounds really interesting. So the wireguard container is functioning as a client, and is forwarding all outgoing traffic for other containers into the vpn? Or what does it do exactly? I run wireguard on my server, but it runs on the host. Coreos should have it installed by default.

I tried to do something like what you are doing now, but never got it to work. I did never try to use a container inside the same network though, how would that even work? Why do the containers route all traffic over the vpn? A container can’t change firewall rules for as far as I know.

For two or more containers to talk on the same (private) network they need to be a part of the same Pod that has the network defined in it I thought. As per podman-pod — Podman documentation

That was precisely my question, but OP assured it was. I have my doubts.

Sure, but a container in the same pod can’t force network of another container to go over a specific interface right? They could talk to each other but not reroute traffic.

1 Like

Yeah, the VPN is acting as the network for the other container, so that other container’s traffic is all routed over the VPN and it has no network access otherwise, while the other things the server is doing can still be done over the regular internet connection without also having to go through the VPN.

I run wireguard on my server, but it runs on the host. Coreos should have it installed by default.

With wireguard on the host, it would effect the entire server’s traffic.

I tried to do something like what you are doing now, but never got it to work. I did never try to use a container inside the same network though, how would that even work? Why do the containers route all traffic over the vpn? A container can’t change firewall rules for as far as I know.

I actually had this working with docker compose on a previous server using this exact config:

---
services:
  wireguard:
    image: lscr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1004
      - PGID=1003
      - TZ=Etc/UTC
    volumes:
      - /opt/docker/qbittorrentvpn/config:/config
      - /lib/modules:/lib/modules
    ports:
      - 6969:6969
      - 6881:6881
      - 6881:6881/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped
  qbittorrent:
    image: lscr.io/linuxserver/qbittorrent:latest
    container_name: qbittorrent
    network_mode: service:wireguard
    environment:
      - PUID=1004
      - PGID=1003
      - TZ=Etc/UTC
      - WEBUI_PORT=6969
    volumes:
      - /opt/docker/qbittorrentvpn:/config
      - /home/docker/qbittorrent-downloads:/downloads
      - /opt/docker/qbittorrentvpn/certs:/certs
    restart: unless-stopped

I haven’t even gotten this far yet, I am just trying to get the wireguard container to boot properly first

That’s why I have to specify network_mode: service:wireguard in the rerouted container’s compose config.

As I said before, wireguard-config was empty, because the container is supposed to create its own config on first boot according to the README and past experience using this exact docker container previously.

I tried adding the minimal config I know it needs, but that didn’t seem to change anything substantially.

1 Like

Very interesting, didn’t know that existed. See that there is an option to use another containers network stack, and that that is what service: is using underneath. Could be very useful.

My bad, read your replies to quickly.

Depends on how you set it up, but routing outgoing traffic of a specific container would be very difficult yes.

When I run the container it works, so podman-compose might be doing something strange. I successfully connected to a server and was able to use the VPN. I still do not understand how you are routing all outgoing traffic over the VPN interface though, so please teach me. Would be very interested in getting that part working.

This is the command I used to run the client, it should do the same as podman-compose. Can you test using podman instead of the compose file? Want to rule out that compose is screwing things up.

podman run -it --rm --cap-add NET_ADMIN --env PUID=1001 --env PGID=1001 --env TZ=Etc/UTC --sysctl net.ipv4.conf.all.src_valid_mark=1 -v wireguard-conf:/config --name wg lscr.io/linuxserver/wireguard:latest

This was my client, as you see I manually specify the interface because else it doesn’t route the traffic correctly:

 podman run -it --rm --network container:wg fedora curl --interface proton 'https://api.ipify.org'

And then my VPN config that I manually placed inside the config directory:

[Interface]
Address = 10.2.0.2
PrivateKey = ...

[Peer]
PublicKey = ...
Endpoint = ...:51820
AllowedIPs = 0.0.0.0/1
PersistentKeepalive = 25
1 Like

Well that’s good to hear, at least it’s possible, probably. I’ll give this a try when I can! thank you!

I still do not understand how you are routing all outgoing traffic over the VPN interface though, so please teach me. Would be very interested in getting that part working… This was my client, as you see I manually specify the interface because else it doesn’t route the traffic correctly:

That was something that “just worked” using docker compose as far as I can tell, because in docker compose when you specify a network, it excludes all other networks, and the wireguard container only presented one. I double and triple checked to make sure it was going only through my vpn and exec’d in and made sure there weren’t other interfaces. podman might have subtly different behavior maybe

same as what im doing, cool, at least i don’t have to change that

Interesting, that might be it. Podman seems to also include the default network:

$  podman run -it --rm --network container:wg alpine ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN qlen 1000
...
4: proton: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 65440 qdisc noqueue state UNKNOWN qlen 1000
    link/[65534] 
    inet 10.2.0.2/32 scope global proton
       valid_lft forever preferred_lft forever

1 Like

@job79 okay i think something non-wireguard specific is going on, because I just rebooted, and despite having rebooted multiple times between creating this thread and now and it all working fine, suddenly now a totally separate container is complaining about random missing directories and files — resolv.conf, but also .containerenv, /etc/secrets being too large, etc — and everything that used to be reachable from another computer on the sever (on ports 80 and 2222) is now completely unreachable as well even for the containers that still say they’re running fine.

Sampling of nginx container errors:

Error: unable to start container 447bc87e421cf926ec33d241e6f6d6aebc9ab1d0085e8a8663d58c9b5116fe46: crun: open `/var/home/container/.local/share/containers/storage/overlay/801ab164291199045f492b0c46f400d3a284d562767ffb7e3a6a5b023bac75a0/merged/run/.containerenv`: No such file or directory: OCI runtime attempted to invoke a command that was not found
Error: OCI runtime error: unable to start container 447bc87e421cf926ec33d241e6f6d6aebc9ab1d0085e8a8663d58c9b5116fe46: crun: mkdir `/run/secrets`: Value too large for defined data type
Error: unable to start container 447bc87e421cf926ec33d241e6f6d6aebc9ab1d0085e8a8663d58c9b5116fe46: crun: open `/var/home/container/.local/share/containers/storage/overlay/801ab164291199045f492b0c46f400d3a284d562767ffb7e3a6a5b023bac75a0/merged/etc/resolv.conf`: No such file or directory: OCI runtime attempted to invoke a command that was not found