Yes, it is just NetworkManager, but perhaps somebody could provide a working static IPv6 configuration.
I’m using FCOS on a netcup server. IPv4 is configured by DHCP (but I would accept a static solution as well), but IPv6 does not work out of the box…
Kind regards,
aanno2
The docs have examples for many network configs. Here’s the one for static networking: Host Network Configuration :: Fedora Docs.
The IPv6 variant should be similar (you can use ipv6: NetworkManager Reference Manual as a reference for accepted keys).
Well, I still have not figured it out. At present IPv6 is working, but only on the private network.
I start the FCOS libvirt with:
#!/bin/bash
# make a local vm from coreos image
set -eu
CONFIG_FILE_NAME=my
# absolute path needed
IMAGE="$HOME/.local/share/libvirt/images/fedora-coreos-33.20210201.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos-33"
VCPUS="4"
RAM_MB="8192"
DISK_GB="20"
STREAM="stable"
fcct -p -s -o "${CONFIG_FILE_NAME}.ign" "${CONFIG_FILE_NAME}.fcc" || exit -1
# absolute path needed
IGNITION_CONFIG=`readlink -f "${CONFIG_FILE_NAME}.ign"`
virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" \
--os-variant="fedora-coreos-$STREAM" --import --graphics=none \
--disk="size=${DISK_GB},backing_store=${IMAGE}" \
--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}" \
--network network=default
I’m using the virt default network (that is including ipv6 support):
$ sudo virsh net-dumpxml default
<network connections='1'>
<name>default</name>
<uuid>041ae9e3-1a0d-4299-b0ab-ef382b525f09</uuid>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address='52:54:00:01:65:9a'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254'/>
</dhcp>
</ip>
<ip family='ipv6' address='fd0f:5c5b:f011:d0fb::1' prefix='64'>
<dhcp>
<range start='fd0f:5c5b:f011:d0fb::100' end='fd0f:5c5b:f011:d0fb::1ff'/>
</dhcp>
</ip>
</network>
The relevant part of my.fcc looks like this:
storage:
files:
- path: /etc/NetworkManager/system-connections/enp1s0.nmconnection
mode: 0600
contents:
inline: |
[connection]
id=enp1s0
type=ethernet
interface-name=enp1s0
[ipv4]
dns-search=
method=auto
[ipv6]
dns=fd00::3681:c4ff:fec9:6d8b
addr-gen-mode=eui64
address1=<public-ipv6>
address2=fe80::4049:b055:c122:abcd
method=auto
From inside the CoreOS VM, the network looks like:
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:e6:d5:70 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.64/24 brd 192.168.122.255 scope global dynamic noprefixroute enp1s0
valid_lft 2246sec preferred_lft 2246sec
inet6 <public-ipv6>/64 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fd0f:5c5b:f011:d0fb::1d8/128 scope global dynamic noprefixroute
valid_lft 85047sec preferred_lft 85047sec
inet6 fe80::4049:b055:c122:abcd/64 scope link noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fee6:d570/64 scope link noprefixroute
valid_lft forever preferred_lft forever
$ ip -4 r
default via 192.168.122.1 dev enp1s0 proto dhcp metric 100
192.168.122.0/24 dev enp1s0 proto kernel scope link src 192.168.122.64 metric 100
$ ip -6 r
::1 dev lo proto kernel metric 256 pref medium
<public-ipv6>/64 dev enp1s0 proto kernel metric 100 pref medium
fd0f:5c5b:f011:d0fb::1d8 dev enp1s0 proto kernel metric 100 pref medium
fd0f:5c5b:f011:d0fb::/64 dev enp1s0 proto ra metric 100 pref medium
fe80::/64 dev enp1s0 proto kernel metric 100 pref medium
default via fe80::5054:ff:fe01:659a dev enp1s0 proto ra metric 100 pref medium
These are some ping6 tries:
$ ping6 fe80::5054:ff:fe01:659a%enp1s0
PING fe80::5054:ff:fe01:659a%enp1s0(fe80::5054:ff:fe01:659a%enp1s0) 56 data bytes
64 Bytes von fe80::5054:ff:fe01:659a%enp1s0: icmp_seq=1 ttl=64 Zeit=0.058 ms
$ ping6 fd0f:5c5b:f011:d0fb::1
PING fd0f:5c5b:f011:d0fb::1(fd0f:5c5b:f011:d0fb::1) 56 data bytes
64 Bytes von fd0f:5c5b:f011:d0fb::1: icmp_seq=1 ttl=64 Zeit=0.227 ms
# But
$ ping6 google.com
PING google.com(fra16s50-in-x0e.1e100.net (2a00:1450:4001:810::200e)) 56 data bytes
^C
--- google.com ping statistics ---
3 Pakete übertragen, 0 empfangen, 100% packet loss, time 2082ms
Local IPv6 seems to be in place. But public IPv6 didn’t work (certainly, it is working on the dom0 host!).
Any idea how to solve the problem?
Hmm, I’m not sure to be honest. Maybe easiest to iterate on different configurations is to keep tweaking the NM configuration in the host (using e.g. nmcli or nmtui), and then once it’s working, you can copy the final keyfile back into your FCC.
As for the problem itself, someone with more experience with IPv6 might be able to help. Are you sure the DNS server (fd00::3681:c4ff:fec9:6d8b) is configured properly? (E.g. does host google.com fd00::3681:c4ff:fec9:6d8b work?)
The DNS configuration seems to be fine, as the “google.com” in ping gets resolved to a reasonable IPv6 address.
I think this is an overall network configuration issue, maybe not strictly related to FCOS / NM.
I see that your node is supposed to have multiple IPv6 addresses on the same interface, and that you are using local addresses for routing. I’d recommend that you do a tcpdump somewhere along the network and check the headers of these packets.
Without further configuration steering the source-selection algorithm, it is quite possible that your ping packets are going out with a reserved-local source address. If that’s the case, they are either dropped at some public interconnection border, or responses are unable to get back to you due to missing public routing for the local address.
Additionally, looking at the rest of your environment, the outer infrastructure seems to have DHCPv6 in place but the VM configuration is using two hardcoded addresses instead.
This results in the interface having a link-local address, a (temporary) DHCP address and route, and two (local/reserved) static address. Depending on what you really want to achieve, you should drop the redundant parts in this setup.
After much fiddling, I found a solution. The problem has several parts, I will begin with libvirt.
If you do nothing, the networking of virt-manager will be ‘NAT’. This is easy, but I didn’t get IPv6 working in this configuration. Hence I tried the ‘bridge’ network. In order to use that, you probably have to reconfigure your (dom0) host to use a bridge (br0 in our case). How to get there is out of scope of this text (but you can find good help on the internet). You know you use bridge when:
$ ip a
...
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether fa:6f:40:ad:78:67 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.57/24 brd 192.168.10.255 scope global dynamic noprefixroute br0
valid_lft 220719sec preferred_lft 220719sec
inet6 <dom0-ipv6-addr>/128 scope global dynamic noprefixroute
valid_lft 6240sec preferred_lft 2640sec
inet6 fe80::4049:b055:c122:77f3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
...
Now you must create an libvirt network configuration that uses the bridge. You do this with sudo virsh net-create --file <filename_of_xml_below> (and you could later edit it with sudo virsh net-edit <name>).
$ sudo virsh net-dumpxml br0
<network>
<name>br0</name>
<uuid>07a573e1-6edd-4d77-95e6-bc25bcb551a5</uuid>
<forward mode='bridge'/>
<bridge name='br0'/>
</network>
The sudo is very important. You could create network configuration as normal user, but libvirt will not use them. You could read more details about libvirt networking at Networking - Libvirt Wiki .
storage:
files:
- path: /etc/NetworkManager/system-connections/enp1s0.nmconnection
mode: 0600
contents:
inline: |
[connection]
id=enp1s0
type=ethernet
interface-name=enp1s0
[ipv4]
dns-search=
method=auto
[ipv6]
# see https://developer.gnome.org/NetworkManager/stable/settings-ipv6.html
dns=<i_use_an_private_dns_here>
gateway=<public-ipv6-of-your-router>
addr-gen-mode=eui64
address1=<public-ipv6-of-fcos>
method=manual
$ cat ./install2.sh
#!/bin/bash
# make a local vm from coreos image
set -eu
CONFIG_FILE_NAME=my
# absolute path needed
IMAGE="$HOME/.local/share/libvirt/images/fedora-coreos-33.20210201.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos-33"
VCPUS="4"
RAM_MB="8192"
DISK_GB="20"
STREAM="stable"
fcct -p -s -o "${CONFIG_FILE_NAME}.ign" "${CONFIG_FILE_NAME}.fcc" || exit -1
# absolute path needed
IGNITION_CONFIG=`readlink -f "${CONFIG_FILE_NAME}.ign"`
virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" \
--os-variant="fedora-coreos-$STREAM" --import --graphics=none \
--disk="size=${DISK_GB},backing_store=${IMAGE}" \
--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}" \
--network network=br0
This this setup, FCOS on libvirt is fully connected to the internet with IPv4 and IPv6.