F41 Change Proposal: Libvirt Virtual Network NFTables (self-contained)

:link: Libvirt Virtual Network NFTables

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

The default firewall backend for the default libvirt virtual network (the virbr0 bridge device), will change from ‘iptables’ to ‘nftables’.

:link: Owner

:link: Detailed Description

A default installation of libvirt will create an isolated bridge device (virbr0), to which guest virtual machines can have their NIC connected. Firewall rules are used to provide NAT based connectivity on this bridge device, as well as allow access to the DNS/DHCP services provided by the dnsmasq it launches on the host.

Historically libvirt has always used the iptables tools, which first talked to the iptables kernel modules, but in recent Fedora uses the nftables kernel modules. When Fedora switched Firewalld to use nftables for its own firewall rules, libvirt kept using its iptables userspace backend, which then indirectly created kernel nftables rules. To work correctly, libvirt added the ability to associate its bridge devices with a ‘libvirt’ zone in firewall to ensure guest->host DHCP/DNS/SSH traffic was not blocked by firewall.

With this change, libvirt will now prefer to directly use the nftables userspace tools to create its firewall rules, where they are installed:

  • nftables only installed => libvirt uses its nftables backend
  • iptables only intsalled => libvirt uses its iptables backend
  • nftables and iptables installed => libvirt uses its nftables backend

This default behaviour can be overridden to force a specific backend by editting settings in /etc/libvirt/network.conf.

Note that this change only applies to libvirt’s virtual network functionality. Libvirt’s nwfilter (network filter) functionality is still exclusively using iptables/ebtables. This will be switched to nftables in a future Fedora release, timeframe to be determined.

:link: Feedback

TBD

:link: Benefit to Fedora

  • Libvirt will be using the same recommended modern nftables userspace as firewalld instead of the legacy iptables compatibility tools which secretly talk to nftables behind the scenes.
  • All the libvirt nftables rules will be self-contained in a dedicated ‘libvirt_network’ nftables table, reducing the scope for other applications accidentally changing/removing them.
  • Improved performance scalability since libvirt’s nftables rules are written to use an interface ID match which is a simple lookup, while traditional string based interface name matches in iptables require multiple string comparisons.

:link: Scope

  • Proposal owners: Change the libvirt RPM spec to set the ‘nftables’ backend as the preferred default

  • Other developers: N/A

  • Release engineering: N/A

  • Policies and guidelines: N/A

  • Trademark approval: N/A

  • Alignment with the Fedora Strategy:

:link: Upgrade/compatibility impact

Existing hosts with libvirt deployed will be using iptables for their firewall rules. When the Fedora upgrade brings in the new libvirt, it will detect the switch to nftables and attempt to remove the historically created iptables rules, and create new nftables rules providing equivalent functionality.

A number of iptables chains will remain present after an upgrade

  • Chain LIBVIRT_FWI
  • Chain LIBVIRT_FWO
  • Chain LIBVIRT_FWX
  • Chain LIBVIRT_INP
  • Chain LIBVIRT_OUT

These chains should not contain rules and thus be harmless. They will go away permanently the next time the host is rebooted, or can be deleted manually if desired.

:link: Reverting to iptables for compatibility

If an incompatibility is discovered between libvirt’s nftables backend and some other use case, it is possible to tell libvirt to revert to its iptables backend.

  • Edit /etc/libvirt/network.conf and set ‘firewall_backend = “iptables”’
  • systemctl restart virtnetworkd

Libvirt should then automatically delete its nftables rules and create equivalent iptables rules as present on previous Fedora releases.

Alternatively, if the ‘nftables’ userspace tools are not installed at all, libvirt will attempt to use the ‘iptables’ userspace tools instead, if present.

:link: Known issue: docker

The Docker iptables integration force changes the iptables FORWARD chain policy from ACCEPT to DENY. This makes Docker incompatible with any application that is using nftables to try to allow forwarding:

When libvirt uses its iptables backend, its FORWARD rules will end up in the same table as docker’s, and so override the DENY policy docker had created. When libvirt uses its nftables backend, its FORWARD rules end up in a separate table from dockers’. Since nftables requires a packet to be allowed by all top level tables, docker’s DENY rules will block the libvirt traffic.

The possible workarounds are:

  • Reconfigure libvirt to use iptables when compatibility with docker is required
  • Use podman instead of docker, since podman does not create problematic iptables rules
  • TBD: whether libvirt can do something automagic to workaround docker’s DENY policy

:link: Known issue: non-firewalld firewall mgmt tools

Libvirt requires the ability to mark traffic from the guest to the host, for DHCP and DNS as permitted.

When using iptables kernel modules, libvirt could achieve this by inserting some rules in the INPUT chain to allow DHCP/DNS.

When using nftables kernel modules, there are an arbitrary number of application defined top level tables with unknown names. All top level tables must allow a packet in order for it to pass. libvirt must not touch tables created by other applications, thus it is no longer practical to seemlessly allow DHCP & DNS when nftables userspace is used by any component on the system.

To address this, when Fedora 32 switched firewall to its nftables backend, libvirt gained ability to install a firewalld zone files to allow guest traffic. Libvirt does not have equivalent integration for any other nftables based firewall mgmt tool, but other tools using iptables remained compatible as long as libvirt also kept using iptables.

With libvirt changing to prefer its nftables backend, libvirt is will be incompatible with any other non-firewalld firewall management tools.

The compatibility matrix is as follows

  1. Kernel module: iptables, firewall mgmt userspace: iptables, libvirt backend: iptables => WORKS for all firewall mgmt tools
  2. Kernel module: nftables, firewall mgmt userspace: iptables, libvirt backend: iptables => WORKS for all firewall mgmt tools
  3. Kernel module: nftables, firewall mgmt userspace: iptables, libvirt backend: nftables => WORKS for firewalld, other tools require workaround
  4. Kernel module: nftables, firewall mgmt userspace: nftables, libvirt backend: iptables => WORKS for firewalld, other tools require workaround
  5. Kernel module: nftables, firewall mgmt userspace: nftables, libvirt backend: nftables => WORKS for firewalld, other tools require workaround

The two main workaround options:

  • Change /etc/libvirt/network.conf ‘firewall_backend’ setting to ‘iptables’
  • Manually add rules to the firewall mgmt tool to allow DHCP, DNS & SSH from guests.

The long term answer is to enhance libvirt upstream to natively learn about integration for more firewall mgmt tools than just firewalld. eg UFW: Libvirt 10.4 Default NAT Network not working with UFW (#644) · Issues · libvirt / libvirt · GitLab

:link: Early Testing (Optional)

Do you require ‘QA Blueprint’ support? N

:link: How To Test

  • Perform a default libvirt KVM install ‘dnf groupinstall @Virtualization
  • Create a KVM guest using virt-install, virt-manager, or cockpit, and configure it to use the ‘default’ virtual network
  • Boot the KVM guest
  • Login to the KVM guest graphical console, and attempt to connect to the internet. eg curl https://google.com
  • Configure the KVM guest to enable SSH (if not already allowed & started)
  • Attempt to SSH to the guest from the host

:link: User Experience

  • Libvirt firewall rules will no longer be present when running ‘iptables -L’
  • A new ‘libvirt_network’ table will be present when running ‘nft list ruleset’

:link: Dependencies

None

:link: Contingency Plan

  • Contingency mechanism: Libvirt maintainers to change the libvirt.spec to set ‘prefer_nftables’ variable to ‘0’ for Fedora 41
  • Contingency deadline: Final freeze
  • Blocks release? No

:link: Documentation

General libvirt network documentation at libvirt: Network XML format

:link: Release Notes

The libvirt virtual network has been changed to prefer use of the nftables firewall backend instead of iptables.

Last edited by @amoloney 2024-06-17T11:37:05Z

2 Likes

How do you feel about the proposal as written?

  • Strongly in favor
  • In favor, with reservations
  • Neutral
  • Opposed, but could be convinced
  • Strongly opposed
0 voters

If you are in favor but have reservations, or are opposed but something could change your mind, please explain in a reply.

We want everyone to be heard, but many posts repeating the same thing actually makes that harder. If you have something new to say, please say it. If, instead, you find someone has already covered what you’d like to express, please simply giving that post a :heart: instead of reiterating. You can even do this by email, by replying with the heart emoji or just “+1”. This will make long topics easier to follow.

Please note that this is an advisory “straw poll” meant to gauge sentiment. It isn’t a vote or a scientific survey. See About the Change Proposals category for more about the Change Process and moderation policy.

This change proposal has now been submitted to FESCo with ticket #3237 for voting.

To find out more, please visit our Changes Policy documentation.

3 Likes

This change has been accepted by FESCo for Fedora Linux 41. A full list of approved changes to date can be found on the Change Set Page.

To find out more about how our changes policy works, please visit our docs site.