Problem setting up a router with PPPoE using NetworkManager and firewalld

I really appreciate your help. Sorry about the delay in replying; this site blocked me for 14 hours because I’d exceeded a first day user posting limit.

root@gata[/etc/NetworkManager/system-connections]# nmcli connection show
NAME        UUID                                  TYPE      DEVICE    
enp1s0f1    35507e9e-c8a6-47f6-a7bb-75f790c1c7fb  ethernet  enp1s0f1  
bridge      5fd824d9-2ff3-41af-b28f-832dc1abdca9  bridge    nm-bridge 
teksavvy    3b328d01-8011-4900-a6c6-df2699fc2e31  pppoe     eno1      
br-slave-1  15e46689-e17f-4b0e-8a33-816fc08e03ee  ethernet  enp1s0f0  
br-slave-2  d0ea0f74-e014-46e5-987f-0e8b60f35517  ethernet  enp2s0f0  
br-slave-3  4faa3c37-b9dd-48c6-bc6e-658357c6f945  ethernet  enp2s0f1  
br-slave-2  d8931fde-521f-4336-abe8-9a6e80090d57  ethernet  --        
br-slave-3  cb94ce72-8979-4102-a66d-daa87069c1f1  ethernet  --        
eno1        0fb26940-f17e-45bc-a161-c2be2f6a3ca2  ethernet  --        
enp1s0f0    75c008b3-00ce-452b-9750-92a343ee9fef  ethernet  --        
enp2s0f0    8dcc5603-eb73-4a1e-bd14-3d1c57f3ff07  ethernet  --        
enp2s0f1    3ac73ba1-2cd2-4add-80eb-3f9728108918  ethernet  --        
root@gata[/etc/NetworkManager/system-connections]# nmcli connection del uuid d8931fde-521f-4336-abe8-9a6e80090d57
Connection 'br-slave-2' (d8931fde-521f-4336-abe8-9a6e80090d57) successfully deleted.
root@gata[/etc/NetworkManager/system-connections]# nmcli connection del uuid cb94ce72-8979-4102-a66d-daa87069c1f1
Connection 'br-slave-3' (cb94ce72-8979-4102-a66d-daa87069c1f1) successfully deleted.
root@gata[/etc/NetworkManager/system-connections]# nmcli connection show
NAME        UUID                                  TYPE      DEVICE    
enp1s0f1    35507e9e-c8a6-47f6-a7bb-75f790c1c7fb  ethernet  enp1s0f1  
bridge      5fd824d9-2ff3-41af-b28f-832dc1abdca9  bridge    nm-bridge 
teksavvy    3b328d01-8011-4900-a6c6-df2699fc2e31  pppoe     eno1      
br-slave-1  15e46689-e17f-4b0e-8a33-816fc08e03ee  ethernet  enp1s0f0  
br-slave-2  d0ea0f74-e014-46e5-987f-0e8b60f35517  ethernet  enp2s0f0  
br-slave-3  4faa3c37-b9dd-48c6-bc6e-658357c6f945  ethernet  enp2s0f1  
eno1        0fb26940-f17e-45bc-a161-c2be2f6a3ca2  ethernet  --        
enp1s0f0    75c008b3-00ce-452b-9750-92a343ee9fef  ethernet  --        
enp2s0f0    8dcc5603-eb73-4a1e-bd14-3d1c57f3ff07  ethernet  --        
enp2s0f1    3ac73ba1-2cd2-4add-80eb-3f9728108918  ethernet  --        

Should I delete these?

NAME        UUID                                  TYPE      DEVICE    
enp1s0f0    75c008b3-00ce-452b-9750-92a343ee9fef  ethernet  --        
enp2s0f0    8dcc5603-eb73-4a1e-bd14-3d1c57f3ff07  ethernet  --        
enp2s0f1    3ac73ba1-2cd2-4add-80eb-3f9728108918  ethernet  --
1 Like

Yes, It’s best to delete inactive connections which have no device: DEVICE --
This helps prevent race conditions and unexpected results.


I updated the script above to better match your config, so you can re-apply it.

After that post the diagnostics once again.

I managed to get ping from internet hosts to work, as well as to port 80 hosted on the router. The only thing not working is the port forwarding from the internet to an internal host.

Thanks!

Roger WillCo.

FYI I got sick of all the goofy NIC names, so i renamed them to what their existing/eventual purpose will be:

root@gata[~]# nmcli c s
NAME       UUID                                  TYPE      DEVICE    
teksavvy   c2256a93-e34f-46a3-890e-9af8a0284f5f  pppoe     teklink   
coggw      1051c1e2-1b5b-4b59-8eaf-14efe731de32  ethernet  coggw     
tekbridge  33b7a459-59b3-40d4-9398-4245ae0668c0  bridge    nm-bridge 
coglink    3370a069-335b-4705-bda6-59fd22ae7df0  ethernet  coglink   
mobo       a1150b76-9021-4024-8121-c457d2db4935  ethernet  mobo      
tekgw      85476dbf-6aaa-4a3d-a37a-217031d2f824  ethernet  tekgw     

The box has 5 NICs. The plan is to use it as a router for 2 internet connections:

  1. TekSavvy DSL
  2. Cogeco Cable

Currently it’s acting as a router for TekSavvy until I get another switch to free up 2 of the 3 bridged NICs. The mobo NIC will connect to a NAS for external logging. Once both internets are working, it will have auto failover.

/etc/firewalld/zones/external.xml:  <interface name="ppp0"/>
/etc/firewalld/zones/internal.xml:  <interface name="nm-bridge"/>
/etc/firewalld/zones/internal.xml.old:  <interface name="nm-bridge"/>

etc/NetworkManager/system-connections/teksavvy.nmconnection:zone=external
connection.id:                          teksavvy
connection.zone:                        external
connection.id:                          tekbridge
connection.zone:                        --
connection.id:                          coglink
connection.zone:                        --
connection.id:                          mobo
connection.zone:                        --
connection.id:                          tekgw
connection.zone:                        --
connection.id:                          coggw
connection.zone:                        --
external (active)
  target: default
  icmp-block-inversion: no
  interfaces: ppp0 teklink
  sources: 
  services: http
  ports: 514/tcp 514/udp
  protocols: 
  masquerade: yes
  forward-ports: 
	port=514:proto=udp:toport=:toaddr=10.0.0.5
	port=514:proto=tcp:toport=:toaddr=10.0.0.5
  source-ports: 
  icmp-blocks: echo-reply
  rich rules: 
internal (active)
  target: default
  icmp-block-inversion: no
  interfaces: coglink mobo nm-bridge tekgw
  sources: 
  services: http ssh
  ports: 
  protocols: 
  masquerade: yes
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 
	rule family="ipv4" source address="10.0.0.0/24" masquerade

Nothing returned.

ipv4 filter FORWARD 0 -i lo -j ACCEPT
ipv4 filter FORWARD 0 -i nm-bridge -j ACCEPT
ipv4 filter FORWARD 0 -i eno1 -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT
ipv4 filter FORWARD 0 -i eno1 -p icmp -m conntrack --ctstate NEW -j ACCEPT
ipv4 filter FORWARD 0 -i eno1 -m conntrack --ctstate INVALID -j DROP
ipv4 filter FORWARD 0 -j REJECT --reject-with icmp-admin-prohibited
ipv6 filter FORWARD 0 -i lo -j ACCEPT
ipv6 filter FORWARD 0 -i nm-bridge -j ACCEPT
ipv6 filter FORWARD 0 -i eno1 -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT
ipv6 filter FORWARD 0 -i eno1 -p ipv6-icmp -m conntrack --ctstate NEW -j ACCEPT
ipv6 filter FORWARD 0 -i eno1 -p tcp -m tcp --dport 51413 -m conntrack --ctstate NEW -j ACCEPT
ipv6 filter FORWARD 0 -i eno1 -p udp -m udp --dport 51413 -m conntrack --ctstate NEW -j ACCEPT
ipv6 filter FORWARD 0 -i eno1 -m conntrack --ctstate INVALID -j DROP
ipv6 filter FORWARD 0 -j REJECT --reject-with icmp6-adm-prohibited
ipv4 -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Thanks for your help!

1 Like

Looks fine, I updated the code above.
Run the entire script to make the firewall config match the new interface names.

Avoid connecting the second ISP until we fix port forwarding.
Multi-WAN adds complexity and requires to configure metric and policy-based routing to work properly.