VMWare Fusion Deployment Issue w/OVA format

[mp3689 (Marc Pusey)]

When trying to deploy a FCOS cluster locally on my Mac using VMWareFusion (11.5.1) I am encountering an issue with the OVA package format.

The error returned is the same whether I try to Import the OVA package via the UI or whether I try simply to introspect the package using VMWare’s command line utility ovftool that comes bundled with VMWare Fusion: “Did not find an .ovf file at the beginning of the OVA package”.

Most easily reproduced with:
ovftool fedora-coreos-31.20200118.3.0-vmware.x86_64.ova

As a workaround I have confirmed that it is possible to export from vSphere into an OVF format, and then use this to create clusters locally by deploying it using the ovftool into VMWareFusion.

It would still be better to address the OVA format issue at its core.

Notes:

• I am able to deploy the OVA file to VMWare VSphere (v6.5.0.30000) without issues.
• ContainerLinux CoreOS OVA tools (even the most recent ones - do not suffer this issue).

[an101 (Michael Dyer)]

Same problem with VMware Workstation 15 Pro (build 15.5.1 build-15018445)

[dans (Dan)]

I am having this same issue with VMWare Fusion Professional Version 11.5.1 (15018442) using 31.20200127.3.0.ova for Fedora CoreOS.

There is an ovf file in the ova file, but ovftool does not like it. If I extract the ova file and run ovftool in ovf mode, it complains about four things:

• Line 54: Unsupported virtual hardware device ‘VirtualSCSI’.
  • This can be ignored
• Line 89: Unsupported element ‘Property’
  • This must be addressed (see below)
• No manifest file found.
  • This can be ignored, but RH really needs to provide a manifest
• Wrong file size specified in OVF descriptor for ‘disk.vmdk’ (specified: 8589934592, actual 713167360).
  • This is fixed by changing ovf:size on the vmdk to 713167360

So what is the deal with with the Property error? The ovf has two properties which are marked as required:

• guestinfo.ignition.config.data
• guestinfo.ignition.config.data.encoding

In order for ovftool to build, these need to be specified using the --prop switch. However, ovftool messes this up and does not propagate those properties to the resultant vmx file. So if you really needed those to be set on the vmx file, you have to edit the resultant vmx file yourself to add them.

Once I do all this, I can get a valid vmx file that I can use with my packer build.

Compare this to the 1688.5.3.ova for the pre-fedora CoreOS and that ova works just fine. So whatever automated process Fedora is using to generate this ova is not working properly.

[dustymabe]

I think this is probably something we need to look in to. Can one of you file an issue in our issue tracker for this? GitHub - coreos/fedora-coreos-tracker: Issue tracker for Fedora CoreOS

[mp3689 (Marc Pusey)]

@dans

I am having this same issue with VMWare Fusion Professional Version 11.5.1 (15018442) using 31.20200127.3.0.ova for Fedora CoreOS.

There is an ovf file in the ova file, but ovftool does not like it. If I extract the ova file and run ovftool in ovf mode, it complains about four things:

• Line 54: Unsupported virtual hardware device ‘VirtualSCSI’.
  • This can be ignored
• Line 89: Unsupported element ‘Property’
  • This must be addressed (see below)
• No manifest file found.
  • This can be ignored, but RH really needs to provide a manifest
• Wrong file size specified in OVF descriptor for ‘disk.vmdk’ (specified: 8589934592, actual 713167360).
  • This is fixed by changing ovf:size on the vmdk to 713167360

The guest info properties are very similar to those that existed on CL and are the means to pass in the ignition file data. So I would expect these to be in the OVF file right? I have found that these are successfully passed into the VM as it is created providing you use the --allowExtraConfig switch on the ovftool. Is there something else wrong with these properties?

I’ll happily do the grunt work of creating the issue in git once I confirm this. I think the net net would be that the one thing wrong with the current ova that MUST be fixed is simply the size?

[dans (Dan)]

The guest info properties are very similar to those that existed on CL and are the means to pass in the ignition file data.

Yes, but the difference is the OVF for CL marked properties as optional, FCOS does not. Marking them as required and not supplying them when invoking ovftool causes ovftool to fail with the misleading Unsupported element ‘Property' error.

I think the net net would be that the one thing wrong with the current ova that MUST be fixed is simply the size?

The size is a must fix because the only solution is to hand-edit the OVF. I would say if you don’t mark the ignition props as optional you should document how people who are not using ignition can pass dummy values into those props to satisfy ovftool.

[mp3689 (Marc Pusey)]

@dans, thanks!

https://github.com/coreos/fedora-coreos-tracker/issues/391

[eightup (octavius octavian)]

I get the same problem when trying to import with VMware vSphere on a standalone ESXi server (not using vCenter). One cannot use the ovftool to extract the .ovf file from the .ova due to the fact that the .ovf file itself that exists inside of the .ova is named desc.ovf for some reason:

So one is left using a tar file extractor to manually extract the .ovf file from the downloaded .ova file. But when attempting to import the .ovf file into vSphere using the ‘Deploy OVF Template’ option, it also errors out citing the ‘Property’ element. It’s possible to edit the ‘Property’ element out of the file manually, but as I understand it is quite a necessary element if one wishes to ever log in to the system once it is deployed.

Update:
One thing that I found out is that the files were packaged incorrectly. There are two files in the latest .ova download; desc.ovf and disk.vmdk. That naming convention apparently causes an issue because I extracted those two files, then used 7zip to create a new .ova (using tar format) which I named desc.ova. I was able to use ovftool on that new .ova file, and it completed the extraction without errors. However, I still run into the exact same issue with line 89 when I try to deploy that new .ovf file.

I could manually remove those elements, but then how do I authenticate? I do not want to use pxe in any way.

Futher update:
Apparently the problem with the Property tag not being supported is due to uploading the OVF directly onto the ESXi server, rather than using a vCenter server to broker it. However, according to VirtualGhetto, there is a workaround: –X:injectOvfEnv. The new ovftool includes an argument that supports the Property tag when uploading the OVF file directly to the ESXi server. It appears that you simply need to include this switch in the ovftool command:

"C:\Program Files (x86)\VMware\VMware OVF Tool\ovftool.exe" ^
    --X:injectOvfEnv  --powerOn  ^
    --prop:ovf:key=guestinfo.ignition.config.data="$PutTheBase64StringofTheIgnitionFileHere." ^
    --prop:ovf:key=guestinfo.ignition.config.data.encoding=base64 ^
C:\Users\username\FedoraCoreOS-VM\fedora-coreos-31.20200127.3.1-vmware.x86_64.ova ^
C:\Users\username\FedoraCoreOS-VM\fedora-coreos-31.20200127.3.1-vmware.x86_64.ovf 

To replace the Property stanza from the original desc.ovf:

I have followed the example set here:: no luck.
I have also tried to use the –allowExtraConfig switch and had no luck.

Shouldn’t the whole .ova just be repackaged so that it works properly? Is it even ready for release?

[bgilbert (Benjamin Gilbert)]

This PR should fix most of this.

@eightup, could you try a quick test?

1. Untar the OVA.
2. Edit desc.ovf, adding an attribute ovf:required="false" to each of the <Property> elements.
3. Create a new OVA from desc.ovf and disk.vmdk, e.g. with tar cf test.ova --format=ustar desc.ovf disk.vmdk.
4. Try deploying that OVA directly to an ESXi node (not through vCenter).

If that fixes the “Unsupported element” error, we could make that change to our OVF. I’m not sure whether we should, though. While it’s technically possible to launch Fedora CoreOS on VMware without providing an Ignition config, it’s not very useful, since there’d be no way to log into the machine afterward. (Without further hacks, anyway, such as modifying the .vmx file or booting with init=/bin/sh.)

[eightup (octavius octavian)]

@bgilbert

it’s not very useful, since there’d be no way to log into the machine afterward

1. The Property tags appear to be 100% useless anyway, as the config actually requires a custom base64 string in the Property tags to have any value. Please correct me if i misunderstand this.
2. It would be useful if the ovftool can actually be leveraged to supply that custom string
3. Adding the “required=false” string to the Property tag didn’t work anyhow, as I continue to receive the error:
   

[dans (Dan)]

Did you try using the namespace for the required attribute?

<ProductSection ovf:required="false">

[mp3689 (Marc Pusey)]

@eightup the base64 string property is meant to be the encoded ignition file produced by the fcct transpiler. Ideally this injects all aspects of a machine’s configuration in one shot with no subsequent need to install anything locally; and as such creating disposable servers that suit a strict purpose. As such we find it a very useful property when creating docker swarms.

Certainly it is possible to use the ovftool on MacOS to deploy to VMWareFusion locally using this property. For us it looks something like this:

ovftool \
	--powerOffTarget \
	--overwrite \
	--name=${MACHINE} \
	--acceptAllEulas \
	--memorySize:'*'=${MEM_SIZE} \
	--numberOfCpus:'*'=${VCPUS} \
	--allowExtraConfig \
    --extraConfig:guestinfo.ignition.config.data.encoding="base64" \
	--extraConfig:guestinfo.ignition.config.data="${EXTRA_CONFIG}" \
	${FCOS_OVA} \
	~/Virtual\ Machines.localized

Incidentally we have also successfully used powershell from Mac using VMWare’s PowerCLI tooling. However, this has required a slightly different property key. It is possible you are coming across a property key issue like this when deploying to vSphere directly using ovftool.

Our pre-processed powershell (with some bash expansions) looks something like this. The bit I want to highlight is the suffix _1 in the guestinfo properties that we found we needed to use.

...
$ovfPath = "${FCOS_OVA}"
$ovfConfig = Get-OvfConfiguration -Ovf $ovfPath
...
$ovfConfig.Common.guestinfo.ignition.config.data.encoding.value = "base64"
# The _1 suffix only gained through detailed introspection and only
# applicable on VSphere.
$ovfConfig.Common.guestinfo.ignition.config.data_1.value= ${EXTRA_CONFIG}

$vmhost = (Get-Cluster | Get-VMHost | Get-Random)

$resource = Get-ResourcePool -Name "${VI_SERVER_RESOURCE_POOL}"

Write-host "Import-VApp \
			-Source \$ovfPath \
			-OvfConfiguration \$ovfConfig \
			-VMHost \$vmhost \
			-Location \$resource \
			-Datastore ${VI_STORAGE} \
			-DiskStorageFormat ${VI_STORAGE_TYPE} \
			-Name '${MACHINE}' "

Hopefully this may help.

[lucab (Luca Bruno)]

FYI I’ve been gathering some notes in order to later improve docs on how to provision nodes on VMware at user-guides: add platform docs for vmware · Issue #50 · coreos/fedora-coreos-docs · GitHub.

[eightup (octavius octavian)]

@dans

I modified the exisiting Property tags to contain that value:

[eightup (octavius octavian)]

@ mp3689

Using the ovftool sounds like the way to go. I haven’t yet tried VMware’s Power CLI, but I shall attempt that.
Regardless, that still renders the ovf’s Property tags useless. When the base64 string is getting injected into the file, then the Property tags are getting overwritten at that time; So there is no use to having empty Property tags pre-populated in the ovf file.

[dans (Dan)]

I modified the exisiting Property tags to contain that value

As you can see from my original response, I put the optional tag in the ProductSection and it worked for me. I was able to build using the OVF without having to supply the ignition properties.

[eightup (octavius octavian)]

@ dans

@dans

I put the optional tag in the ProductSection and it worked for me.

So you had to modify the ovf file for it to work for you: same as me. It would be nice not to have to modify the ovf file at all; that is my only argument.

[dans (Dan)]

So you had to modify the ovf file for it to work for you: same as me. It would be nice not to have to modify the ovf file at all; that is my only argument.

With respect to the properties, I actually supplied dummy values to satisfy the requirements. However, given the disk size is wrong, I had to edit the OVF anyways to get the correct size.

[bgilbert (Benjamin Gilbert)]

@eightup

So you had to modify the ovf file for it to work for you: same as me. It would be nice not to have to modify the ovf file at all; that is my only argument.

That’s the goal. @eightup, could you confirm that this change works for you?

[eightup (octavius octavian)]

@ bgilbert
I can only confirm that the Property tags had to be removed altogether. If the Property tags exist at all in the ovf file, then it errors out.