Docker causing errors in firewall startup

garberw · October 25, 2023, 3:42pm

I upgraded from fedora 38 to fedora 39 beta.
after boot I run

journalctl -b -p4

and I get the warnings
firewalld[1580]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables … DOCKER, DOCKER-ISOLATION, DOCKER-ISOLATION-STAGE-1, DOCKER-ISOLATION-STAGE-2
I would like to fix the cause of these warnings.

systemctl status docker

and

systemctl status firewalld

do not show any errors other than the first mentioned ones (they are both “green” not “red” and running okay).
I have read elsewhere that it may be due to the order in which firewalld and docker are started.

vgaetera · October 25, 2023, 4:14pm

You can get the full error message like this:

journalctl --no-pager -b -u docker.sevice -u firewalld.service

garberw · October 25, 2023, 6:11pm

garberw@electron> journalctl --no-pager -b -u docker.service
Oct 25 11:01:02 electron.localdomain systemd[1]: Starting docker.service - Docker Application Container Engine...
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.719929499-07:00" level=info msg="Starting up"
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.720957722-07:00" level=info msg="containerd not running, starting managed containerd"
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.727381235-07:00" level=info msg="started new containerd process" address=/var/run/docker/containerd/containerd.sock module=libcontainerd pid=2514
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.790209216-07:00" level=info msg="starting containerd" revision= version=1.6.23
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.801264240-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804229310-07:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/6.5.6-300.fc39.x86_64\\n\"): skip plugin" type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804252670-07:00" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804461266-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804643336-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804656597-07:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804664779-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804696383-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804858952-07:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.804989388-07:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.805002764-07:00" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.805023898-07:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.805031538-07:00" level=info msg="metadata content store policy set" policy=shared
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807801374-07:00" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807821362-07:00" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807830646-07:00" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807851213-07:00" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807913156-07:00" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.807963039-07:00" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808006858-07:00" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808188336-07:00" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808211188-07:00" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808220799-07:00" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808230355-07:00" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808239448-07:00" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808247926-07:00" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808255920-07:00" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808263743-07:00" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808271894-07:00" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808287348-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808296566-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808304287-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808312364-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808320222-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808328583-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808342410-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808350565-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808358291-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808369851-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808379734-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808386654-07:00" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808397100-07:00" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="no OpenTelemetry endpoint: skip plugin" type=io.containerd.tracing.processor.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808404153-07:00" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808415712-07:00" level=error msg="failed to initialize a tracing processor \"otlp\"" error="no OpenTelemetry endpoint: skip plugin"
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808440656-07:00" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.808466134-07:00" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.828343637-07:00" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.828561415-07:00" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.828662881-07:00" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
Oct 25 11:01:02 electron.localdomain dockerd[2514]: time="2023-10-25T11:01:02.828697319-07:00" level=info msg="containerd successfully booted in 0.038975s"
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.832249204-07:00" level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf"
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.880554863-07:00" level=info msg="[graphdriver] using prior storage driver: btrfs"
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.882031274-07:00" level=info msg="Loading containers: start."
Oct 25 11:01:02 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:02.986205271-07:00" level=info msg="Firewalld: docker zone already exists, returning"
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.255333729-07:00" level=info msg="Firewalld: interface docker0 already part of docker zone, returning"
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.506646608-07:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.653467505-07:00" level=info msg="Firewalld: interface docker0 already part of docker zone, returning"
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.800963964-07:00" level=info msg="Loading containers: done."
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.822349283-07:00" level=info msg="Docker daemon" commit="%{shortcommit_moby}" graphdriver=btrfs version=24.0.5
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.822672296-07:00" level=info msg="Daemon has completed initialization"
Oct 25 11:01:03 electron.localdomain dockerd[2425]: time="2023-10-25T11:01:03.920338551-07:00" level=info msg="API listen on /run/docker.sock"
Oct 25 11:01:03 electron.localdomain systemd[1]: Started docker.service - Docker Application Container Engine.

Don’t think it’s a good idea to post my firewall info on the web.

garberw · November 6, 2023, 10:45pm

root@electron# journalctl -b -u firewalld
Nov 06 14:37:52 electron.localdomain systemd[1]: Starting firewalld.service - firewalld - dynamic firewall daemon...
Nov 06 14:37:52 electron.localdomain systemd[1]: Started firewalld.service - firewalld - dynamic firewall daemon.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER' failed: iptables v1.8.9 (legacy): Couldn't load target `DOCKER':No such file or directory
                                                      
                                                      Try `iptables -h' or 'iptables --help' for more information.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER' failed: iptables v1.8.9 (legacy): Couldn't load target `DOCKER':No such file or directory
                                                      
                                                      Try `iptables -h' or 'iptables --help' for more information.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER' failed: iptables v1.8.9 (legacy): Couldn't load target `DOCKER':No such file or directory
                                                      
                                                      Try `iptables -h' or 'iptables --help' for more information.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D PREROUTING' failed: iptables: Bad rule (does a matching rule exist in that chain?).
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT' failed: iptables: Bad rule (does a matching rule exist in that chain?).
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -F DOCKER' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -X DOCKER' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -D FORWARD -i docker0 -o docker0 -j DROP' failed: iptables: Bad rule (does a matching rule exist in that chain?).
Nov 06 14:37:58 electron.localdomain firewalld[1572]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -D FORWARD -i docker0 -o docker0 -j DROP' failed: iptables: Bad rule (does a matching rule exist in that chain?).
root@electron#

vgaetera · November 7, 2023, 12:43am

sudo sed -i -e '/^"$/i --iptables=false' /etc/sysconfig/docker
sudo systemctl restart docker.service

garberw · November 7, 2023, 1:11am

https://stackoverflow.com/questions/40792765/docker-internet-connectivity-with-iptables-false

The docker network model uses iptables to set up internet connectivity for your containers. I would only set iptables=false if you explicitly do not want your containers that are using bridge or overlay network drivers to have any network connectivity at all.

docker or some other service is restarting the firewall rules for docker redundantly.
So it looks like one of those silly warnings that you should just ignore.

what do I need docker for ? could I disable that service ?

vgaetera · November 7, 2023, 1:20am

Docker can work without iptables by using the zone provided by firewalld:

sudo firewall-cmd --info-zone=docker

You can as well disable/remove Docker if you don’t need it:

sudo systemctl disable docker.service
sudo dnf remove docker

garberw · November 7, 2023, 1:26am

the response about firewalld answers the question. Also…
I was trying to figure out if a lot of rpms use docker on a typical fedora install.
So maybe I should disable the service.
Thank you