Two more thoughts.
You can try to manually stop / start firewalld and see, does it stop successfully or does it hang when you try to stop it.
As we’re suspecting firewalld, you can try to enable debugging for firewalld, and inspect the (more verbose) logs to see if there’s any clues as to why it can’t stop as it should.
Take notice that there’re several debug levels are available.
In stopping the firewalld service, it got hung up for a while. Once it finished, the rmmod process was running for a really long time until I forced a restart. I started firewalld again using the debug level 2 and got the following output:
Although I don’t know what this output means or think it’s what’s causing the issue, I think that rmmod process is what is hanging things up when shutting down the firewalld.service.
2019-07-10 11:12:43 DEBUG2: <class ‘firewall.core.ipXtables.ip4tables’>: /usr/sbin/iptables will be using -w10 option.
2019-07-10 11:12:43 DEBUG2: <class ‘firewall.core.ipXtables.ip4tables’>: /usr/sbin/iptables-restore will be using -w option.
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipXtables.ip6tables’>: /usr/sbin/ip6tables will be using -w10 option.
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipXtables.ip6tables’>: /usr/sbin/ip6tables-restore will be using -w option.
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ebtables.ebtables’>: /usr/sbin/ebtables-restore /run/firewalld/temp.57z34kmn: 0
2019-07-10 11:12:44 DEBUG1: start()
2019-07-10 11:12:44 DEBUG1: Loading firewalld config file ‘/etc/firewalld/firewalld.conf’
2019-07-10 11:12:44 DEBUG1: CleanupOnExit is set to ‘True’
2019-07-10 11:12:44 DEBUG1: IPv6 rpfilter is enabled
2019-07-10 11:12:44 DEBUG1: LogDenied is set to ‘off’
2019-07-10 11:12:44 DEBUG1: AutomaticHelpers is set to ‘system’
2019-07-10 11:12:44 DEBUG1: FirewallBackend is set to ‘iptables’
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipset.ipset’>: /usr/sbin/ipset list
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipset.ipset’>: /usr/sbin/ipset --help
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipXtables.ip4tables’>: /usr/sbin/iptables -w10 -p icmp --help
2019-07-10 11:12:44 DEBUG2: <class ‘firewall.core.ipXtables.ip6tables’>: /usr/sbin/ip6tables -w10 -p ipv6-icmp --help
2019-07-10 11:12:44 DEBUG1: Conntrack helpers supported by the kernel:
2019-07-10 11:12:44 DEBUG1: nf_conntrack_amanda: amanda
2019-07-10 11:12:44 DEBUG1: nf_conntrack_ftp: ftp
2019-07-10 11:12:44 DEBUG1: nf_conntrack_h323: H.245, Q.931, RAS
2019-07-10 11:12:44 DEBUG1: nf_conntrack_irc: irc
2019-07-10 11:12:44 DEBUG1: nf_conntrack_netbios_ns: netbios-ns
2019-07-10 11:12:44 DEBUG1: nf_conntrack_pptp: pptp
2019-07-10 11:12:44 DEBUG1: nf_conntrack_sane: sane
2019-07-10 11:12:44 DEBUG1: nf_conntrack_sip: sip
2019-07-10 11:12:44 DEBUG1: nf_conntrack_snmp: snmp
2019-07-10 11:12:44 DEBUG1: nf_conntrack_tftp: tftp
2019-07-10 11:12:44 DEBUG1: NAT helpers supported by the kernel:
2019-07-10 11:12:44 DEBUG1: nf_nat_amanda: amanda
2019-07-10 11:12:44 DEBUG1: nf_nat_ftp: ftp
2019-07-10 11:12:44 DEBUG1: nf_nat_irc: irc
2019-07-10 11:12:44 DEBUG1: nf_nat_sip: sip
2019-07-10 11:12:44 DEBUG1: nf_nat_tftp: tftp
2019-07-10 11:12:44 DEBUG1: nf_nat_h323: h323
2019-07-10 11:12:44 DEBUG1: nf_nat_pptp: pptp
2019-07-10 11:12:44 DEBUG1: Loading lockdown whitelist
2019-07-10 11:12:44 DEBUG1: Loading icmptype file ‘/usr/lib/firewalld/icmptypes/address-unreachable.xml’
@bojohnson02, thing you’ve posted is the start of the service. We need to see debug output of process trying to stop if we hope to see something there. I.e. note the time you’ve issued the stop command and post the output from that moment till the end. If it’s too long to post here you can use something like fpaste or pastebin.
Am I right in understanding you’ve stopped the firewalld process manually, not tried to shutdown your system? It that’s the case, you now have a much simpler way to troubleshoot the issue and try some things – without the need to shut down and power back up your computer every time.
rmmod itself is for unloading kernel modules. This link says firewalld uses netfilter kernel modules, presumably that’s what get unloaded on service stopping (I don’t know for sure). If you’re right and rmmod is to blame, then you need to look for info on which modules are used and what can prevent their unloading.
@nightromantic I’ve tried debugging in the terminal but can’t seem to get the output for stopping firewalld. Based on the documentation, I placed debug=2 into the firewalld systemd service file at /usr/lib/systemd/system/firewalld.service. I then started and stopped firewalld, it again gets hung up with rmmod taking up a lot of CPU and really slowing down the computer. It even stops Firefox from working. Where do I find the debug log for firewalld? Just the output in journalctl?
It’s nice to start and stop firewalld manually, but because rmmod keeps hanging things up, I need to restart my computer each time I stop firewalld anyway.
Ack I just realized I never added the link on how to disable CleanupOnExit before, try this. It basically means that, if you stop firewalld, the firewall rules won’t be automatically cleaned up. If the only time firewalld is stopped is on shutdown, however, that’s pretty insignificant. I’m not sure if it’ll fix this in particular, but it might be worth a shot. However, this would still be a workaround and not a true “fix”.
To get more debugging information, try this:
journalctl -b-1 -e (like before, it’ll show the logs from the last boot right around the end). Now, look for any errors that may have been logged around when you ran rmmod, particularly from the kernel. (To see only kernel messages, add -k to the journalctl command.)@refi64 Thanks for that link. I was able to make the change to CleanupOnExit and it seems to have fixed the issue. I can start and stop firewalld.service and there’s no hang ups anywhere, and rmmod doesn’t seem to go crazy either. The only issue is, when I restart the computer, firewalld.service doesn’t automatically start up again. Is there a way to make sure it starts back up after a reboot?
systemctl enable firewalld, might have gotten disabled at some point while experimenting.
I think that fixed it, thanks