Configure static IPv6 address

Yes, it is just NetworkManager, but perhaps somebody could provide a working static IPv6 configuration.

I’m using FCOS on a netcup server. IPv4 is configured by DHCP (but I would accept a static solution as well), but IPv6 does not work out of the box…

Kind regards,

aanno2

The docs have examples for many network configs. Here’s the one for static networking: Host Network Configuration :: Fedora Docs.

The IPv6 variant should be similar (you can use ipv6: NetworkManager Reference Manual as a reference for accepted keys).

Well, I still have not figured it out. At present IPv6 is working, but only on the private network.

I start the FCOS libvirt with:

#!/bin/bash
# make a local vm from coreos image
set -eu

CONFIG_FILE_NAME=my

# absolute path needed
IMAGE="$HOME/.local/share/libvirt/images/fedora-coreos-33.20210201.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos-33"
VCPUS="4"
RAM_MB="8192"
DISK_GB="20"
STREAM="stable"

fcct -p -s -o "${CONFIG_FILE_NAME}.ign" "${CONFIG_FILE_NAME}.fcc" || exit -1

# absolute path needed
IGNITION_CONFIG=`readlink -f "${CONFIG_FILE_NAME}.ign"`

virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" \
        --os-variant="fedora-coreos-$STREAM" --import --graphics=none \
        --disk="size=${DISK_GB},backing_store=${IMAGE}" \
        --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}" \
        --network network=default

I’m using the virt default network (that is including ipv6 support):

$ sudo virsh net-dumpxml default 
<network connections='1'>
  <name>default</name>
  <uuid>041ae9e3-1a0d-4299-b0ab-ef382b525f09</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:01:65:9a'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
  <ip family='ipv6' address='fd0f:5c5b:f011:d0fb::1' prefix='64'>
    <dhcp>
      <range start='fd0f:5c5b:f011:d0fb::100' end='fd0f:5c5b:f011:d0fb::1ff'/>
    </dhcp>
  </ip>
</network>

The relevant part of my.fcc looks like this:

storage:
  files:
    - path: /etc/NetworkManager/system-connections/enp1s0.nmconnection
      mode: 0600
      contents:
        inline: |
          [connection]
          id=enp1s0
          type=ethernet
          interface-name=enp1s0
          [ipv4]
          dns-search=
          method=auto
          [ipv6]
          dns=fd00::3681:c4ff:fec9:6d8b
          addr-gen-mode=eui64
          address1=<public-ipv6>
          address2=fe80::4049:b055:c122:abcd
          method=auto

From inside the CoreOS VM, the network looks like:

$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:e6:d5:70 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.64/24 brd 192.168.122.255 scope global dynamic noprefixroute enp1s0
       valid_lft 2246sec preferred_lft 2246sec
    inet6 <public-ipv6>/64 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fd0f:5c5b:f011:d0fb::1d8/128 scope global dynamic noprefixroute 
       valid_lft 85047sec preferred_lft 85047sec
    inet6 fe80::4049:b055:c122:abcd/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fee6:d570/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
$ ip -4 r
default via 192.168.122.1 dev enp1s0 proto dhcp metric 100 
192.168.122.0/24 dev enp1s0 proto kernel scope link src 192.168.122.64 metric 100 
$ ip -6 r
::1 dev lo proto kernel metric 256 pref medium
<public-ipv6>/64 dev enp1s0 proto kernel metric 100 pref medium
fd0f:5c5b:f011:d0fb::1d8 dev enp1s0 proto kernel metric 100 pref medium
fd0f:5c5b:f011:d0fb::/64 dev enp1s0 proto ra metric 100 pref medium
fe80::/64 dev enp1s0 proto kernel metric 100 pref medium
default via fe80::5054:ff:fe01:659a dev enp1s0 proto ra metric 100 pref medium

These are some ping6 tries:

$ ping6 fe80::5054:ff:fe01:659a%enp1s0
PING fe80::5054:ff:fe01:659a%enp1s0(fe80::5054:ff:fe01:659a%enp1s0) 56 data bytes
64 Bytes von fe80::5054:ff:fe01:659a%enp1s0: icmp_seq=1 ttl=64 Zeit=0.058 ms

$ ping6 fd0f:5c5b:f011:d0fb::1
PING fd0f:5c5b:f011:d0fb::1(fd0f:5c5b:f011:d0fb::1) 56 data bytes
64 Bytes von fd0f:5c5b:f011:d0fb::1: icmp_seq=1 ttl=64 Zeit=0.227 ms

# But
$ ping6 google.com
PING google.com(fra16s50-in-x0e.1e100.net (2a00:1450:4001:810::200e)) 56 data bytes
^C
--- google.com ping statistics ---
3 Pakete übertragen, 0 empfangen, 100% packet loss, time 2082ms

Local IPv6 seems to be in place. But public IPv6 didn’t work (certainly, it is working on the dom0 host!).

Any idea how to solve the problem?

Hmm, I’m not sure to be honest. Maybe easiest to iterate on different configurations is to keep tweaking the NM configuration in the host (using e.g. nmcli or nmtui), and then once it’s working, you can copy the final keyfile back into your FCC.

As for the problem itself, someone with more experience with IPv6 might be able to help. Are you sure the DNS server (fd00::3681:c4ff:fec9:6d8b) is configured properly? (E.g. does host google.com fd00::3681:c4ff:fec9:6d8b work?)

The DNS configuration seems to be fine, as the “google.com” in ping gets resolved to a reasonable IPv6 address.

I think this is an overall network configuration issue, maybe not strictly related to FCOS / NM.

I see that your node is supposed to have multiple IPv6 addresses on the same interface, and that you are using local addresses for routing. I’d recommend that you do a tcpdump somewhere along the network and check the headers of these packets.
Without further configuration steering the source-selection algorithm, it is quite possible that your ping packets are going out with a reserved-local source address. If that’s the case, they are either dropped at some public interconnection border, or responses are unable to get back to you due to missing public routing for the local address.

Additionally, looking at the rest of your environment, the outer infrastructure seems to have DHCPv6 in place but the VM configuration is using two hardcoded addresses instead.

This results in the interface having a link-local address, a (temporary) DHCP address and route, and two (local/reserved) static address. Depending on what you really want to achieve, you should drop the redundant parts in this setup.

After much fiddling, I found a solution. The problem has several parts, I will begin with libvirt.

Use a bridge on your (dom0) host

If you do nothing, the networking of virt-manager will be ‘NAT’. This is easy, but I didn’t get IPv6 working in this configuration. Hence I tried the ‘bridge’ network. In order to use that, you probably have to reconfigure your (dom0) host to use a bridge (br0 in our case). How to get there is out of scope of this text (but you can find good help on the internet). You know you use bridge when:

$ ip a
...
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:6f:40:ad:78:67 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.57/24 brd 192.168.10.255 scope global dynamic noprefixroute br0
       valid_lft 220719sec preferred_lft 220719sec
    inet6 <dom0-ipv6-addr>/128 scope global dynamic noprefixroute 
       valid_lft 6240sec preferred_lft 2640sec
    inet6 fe80::4049:b055:c122:77f3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
...

Create a libvirt network configuration

Now you must create an libvirt network configuration that uses the bridge. You do this with sudo virsh net-create --file <filename_of_xml_below> (and you could later edit it with sudo virsh net-edit <name>).

$ sudo virsh net-dumpxml br0
<network>
  <name>br0</name>
  <uuid>07a573e1-6edd-4d77-95e6-bc25bcb551a5</uuid>
  <forward mode='bridge'/>
  <bridge name='br0'/>
</network>

The sudo is very important. You could create network configuration as normal user, but libvirt will not use them. You could read more details about libvirt networking at Networking - Libvirt Wiki .

Configure IPv6 in FCOS

storage:
  files:
    - path: /etc/NetworkManager/system-connections/enp1s0.nmconnection
      mode: 0600
      contents:
        inline: |
          [connection]
          id=enp1s0
          type=ethernet
          interface-name=enp1s0
          [ipv4]
          dns-search=
          method=auto
          [ipv6]
          # see https://developer.gnome.org/NetworkManager/stable/settings-ipv6.html
          dns=<i_use_an_private_dns_here>
          gateway=<public-ipv6-of-your-router>
          addr-gen-mode=eui64
          address1=<public-ipv6-of-fcos>
          method=manual

Start FCOS

$ cat ./install2.sh 
#!/bin/bash
# make a local vm from coreos image

set -eu

CONFIG_FILE_NAME=my

# absolute path needed
IMAGE="$HOME/.local/share/libvirt/images/fedora-coreos-33.20210201.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos-33"
VCPUS="4"
RAM_MB="8192"
DISK_GB="20"
STREAM="stable"

fcct -p -s -o "${CONFIG_FILE_NAME}.ign" "${CONFIG_FILE_NAME}.fcc" || exit -1

# absolute path needed
IGNITION_CONFIG=`readlink -f "${CONFIG_FILE_NAME}.ign"`

virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" \
        --os-variant="fedora-coreos-$STREAM" --import --graphics=none \
        --disk="size=${DISK_GB},backing_store=${IMAGE}" \
        --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}" \
        --network network=br0

This this setup, FCOS on libvirt is fully connected to the internet with IPv4 and IPv6.