Afterburn (on vSphere) not functioning?

Server, Cloud, and IoT CoreOS
#23

I just noticed something on my VMs: While the guestinfo properties appear in the vSphere web console, the “guestinfo.afterburn.initrd.network-kargs” does not, even though it is set on the VM (via govc) and works as expected.

Looking at your screenshot, the attribute appears in the web console. How are you setting the property? Through the web console?

BTW, I have a bash script to help consistently deploy RH/Fedora CoreOS nodes with Afterburn and ignition modifications.

https://github.com/JaimeMagiera/oct/blob/master/deploy-coreos-node.sh

You can run it as follows…

deploy-coreos-node.sh --ova fedora-coreos-32.20201104.3.0-vmware.x86_64 --ignition “/path/to/my.ign” --name “my-vm” --cpu 2 --memory 4000 --disk 100 --folder “/myfolder” --library “My Library” --ipcfg “ip=10.103.2.95::10.103.0.1:255.255.255.0:my-vm:ens192}:off” --boot

Note that the script assumes you’ve installed govc into your path, loaded your FCOS images into a Content Library on your vSphere cluster and that you have set your vSphere username, password, and host as environment variables in your shell. I’ve found that deploying with consistency helps with debugging. In this case, it would help us clarify if the issue is setting the attribute via the web console vs. other method.

Edit: Confirmed that adding the property, with the same exact value, through the web console does not work.

Screen Shot 2020-12-30 at 3.19.01 PM
Screen Shot 2020-12-30 at 3.19.01 PM1716×162 11.4 KB

Boot Log from Deployment with Web Console Input

#24

Of course I had also set the value through a script … I’m working on automation too, if you remember…

#25

Ok, NOW we are getting closer.

The way how that Afterburn is documented with the guestinfo.afterburn.initrd.network-kargs setting to someone who was exposed to the ignition configuration before is misleading.

The ignition settings (e.g. guestinfo.ignition.config.data and guestinfo.ignition.config.data.enconding) are usually specified via the $vm.ExtensionData.Config.VAppConfig (at least that’s how the FCOS OVA is pre-configured) while (apparently) the Afterburn value must be specified through $vm.ExtensionData.Config.ExtraConfig … but I assume that the ignition keys/values can also be specified through $vm.ExtensionData.Config.ExtraConfig which is what govc vm.change -e does.

BTW, the $vm.ExtensionData.Config.ExtraConfig values are shown in the vSphere Web UI in “Edit settings” -> “VM Options” -> “Advanced” -> “Configuration Parameters”.

I will try this out … tomorrow (it’s already 10 pm here).

#26

So that was it. The Afterburn config string needs to be an advanced config parameter to the VM and not a vApp parameter (like the ignition strings by default are).
Now it is working.

@jaimelm, your statement “While the guestinfo properties appear in the vSphere web console, the “guestinfo.afterburn.initrd.network-kargs” does not” finally brought me to the solution. Thanks for mentioning this!

I now noticed that configuring statip IP via Afterburn for FCOS is a permanent thing and not only for the initial ignition boot. So in theory I could stop creating NetworkManager config file and /etc/hostname and /etc/hosts via Ignition. I’ll think over this.

But I also found one limitation in static IP configuration via Afterburn: it simply passes through kernel arguments. So it supports what the kernel offers in terms of arguments. I’m missing one feature there: while there is “nameserver=” available there is no karg to configure the DNS search suffixes. I need to think over this and test if it could really be a limitation for my use case of OKD/OCP clusters or if it in practice not really is one.

#27

What I wrote in my previous message was what I tested with FCOS33.
I tested it with FCOS32 also and it works there, too.

I noticed a difference:

  • when configuring FCOS32 with static IP via Afterburn there is no IPv6 address on the interface afterwards
  • when configuring FCOS33 the same way, it then has a interface-local IPv6 address on the interface in addition to the specified IPv4 address

When upgrading such a FCOS32 VM to FCOS33 aftwards leaves it without IPv6 address, BTW.

#28

Fantastic. Glad you got it going. I’ll update my revision to the docs to include a blurb about the actual location for the parameters. That will be helpful for people who are not using govc.