F43 Change Proposal: Golang Packages Vendored By Default (system-wide)

Project Discussion Change Proposals
,
1

Golang Packages Vendored By Default

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

Use vendored dependecies as the default and preferred option when building Golang applications, instead of relying on pre-packaged dependencies. This includes adopting Go Vendor Tools, a new set of tooling to handle license scanning, generating a cumulative SPDX expression for all dependencies, and creating reproducible vendor archives.

:link: Owner

:link: Detailed Description

:link: Plan

:link: Current state

Golang packages can be packaged either by having all dependencies available as packages (current default) or by using vendored dependencies. Non-vendored builds require defining GO111MODULE=off and having all dependencies available as RPM packages.

The current Packaging Guidelines state: At the moment golang projects packaged in Fedora SHOULD be unbundled by default. It means projects are built from dependencies packaged in Fedora. As part of this proposal, this would be changed to favor vendoring.

Golang packages can be any of these types:

  • Binary only package: provides only the compiled binary and no sources/libraries. The package is usually named after the original name of the source. For example: doctl, gh or opentofu. Some of these packages have already moved to vendored dependencies (or used them from the beginning), while many still use the dependencies packaged in Fedora.
  • Source only package: provides only the sources that will be imported as dependencies when building other packages, including for just tests. The naming of this packages is created by normalizing the “goipath” with golang- prefix and the -devel suffix: for example: golang-github-spf13-cobrad-devel or golang-github-prometheus-common-devel.
  • Binary and source packages: a package that provides a binary that is useful for Fedora users, but also provides the “source only package” that is needed to build other packages, for example: golang-github-cpuguy83-md2man or golang-github-prometheus.

As of 2025-05-08, the Go SIG tracks 1898 packages. Of these 484 include a binary, meaning that approximately 1400 source-only packages could potentially be eliminated.

As posted a few months ago, a simple package like doctl (a CLI app to manage DigitalOcean resources) had 122 dependencies listed in the go.mod file, but requires 752 packages to be installed to build it, where 629 of them are golang-*-devel source-only packages. This is because each package not using non-vendored builds has to have the full dependency chain available in the buildroot, even if a project only depends on a single import path from a larger Go module.

:link: Current method’s challenges

  • Many packages, at various points in time, were out of date, failed to build, or failed to install due to difficulties in maintaining such a large number of dependencies in a language ecosystem not suited to the non-vendored packaging model.
  • Importing a new binary Golang package for Fedora or updating any of the existing ones may require adding new source-only packages that are not shared with any other package. Packages like rclone or acme require many new packages as new clouds are supported on each release. In the case of rclone the Fedora package disables some backends/clouds (storj & proton drive for example) due to the complexity of adding new packages.
  • If upstream does not adhere to Semantic Versioning (or has no tagged versions at all), two binary packages may require different versions of the same source-only package, that have non versioned goipath. Example: prometheus-common 0.48.0 introducing a backward incompatible change.
  • Fast moving package stacks, like the whole Kubernetes or OTEL, require compat packages to avoid breakages in packages that import them but are not updated that often.
  • The entire Docker and Containerd stacks were moved to vendored packages after multiple groups of maintainers could not contend with its fast-moving, circular dependency tree.
  • Some Cloud SDK packages like some Azure’s or Google’s modules are packaged in bootstrap mode (no dependency check or tests) to avoid extra dependencies.
  • Sharing work with EPEL for packages with large dependency chains make it nearly impossible.
  • Many source-only packages are owned by single owners and this caused issues in the past after non-responsive processes.
  • A minor update in a package can break the whole dependency chain of many packages. Examples: releng-11471, releng-11498, releng-11565, releng-11684 or releng-11966

:link: Downsides of the proposal

  • If a vulnerability exists on a dependency, with the new approach all the affected packages would need to be updated and rebuilt. With non-vendored builds, the fix would be applied to the dependency package (usually a source-only package) and a simple rebuild of affected packages is enough. With vendored builds, each package would need to be updated or patched. go-vendor-tools makes it easy to update vendored dependencies and has been used to address CVE-2024-6257 CVE-2024-6104 CVE-2024-24789 in OpenTofu for example.
  • If a maintainer wants to import a non-vendored package, it will require extra effort because many source-only dependency packages will be retired once binary packages adopt vendoring.
  • Packagers need to be more aware of the license terms for each vendored dependency. Go Vendor Tools does have license scanning functionality to mitigate this burden.

See also Fedora Go Unbundling is Broken - Maxwell G (@gotmax23) from last year that discusses the issues that prompted this Change in greater depth.

:link: Feedback

This topic has been discussed many times during Go SIG meetings and in the Matrix Channel. The initial ticket that is the root of the proposed change was opened more than 2.5 years ago.

As current recommendation is causing several issues, multiple discussions happened in different places like:

Additionally, the Go Vendor Tools beta has already been adopted by 30+ packages, and we have received positive feedback from maintainers.

:link: Benefit to Fedora

  • Simplify packaging Golang applications and potentially increase the number of packaged applications.
  • Reduce maintainer burnout by simplifying the packaging and updating process.
  • Reduce the amount of packages in Fedora by removing source only Golang packages.
  • Eliminate the version drift of dependencies between upstream projects and what Fedora can provide.

:link: Scope

  • Proposal owners:

    • Change the Golang packaging guidelines to reflect the new default method for packaging Golang applications.
    • Migrate packages to use the new method.
    • Retire leaf packages that are no longer required.

  • Other developers:

    • Adopt the new default method of packaging for their packages. While existing packages that build successfully can continue using the non-vendored method, it will not be the encouraged approach.

  • Release engineering:

    • Nothing will be required from Release engineering.

  • Policies and guidelines:

  • Trademark approval: N/A (not needed for this Change)

  • Alignment with the Fedora Strategy:

    • Better collaborative with EPEL.
    • Potentially more active Go SIG.
    • Ease latest version adoption.

:link: Upgrade/compatibility impact

Users consume Golang application packages that only include the binaries, so this new way of packaging won’t affect users or upgrades.

For packagers that manage Golang packages, if they adopt the new method of vendoring they would reduce the churn of having to maintain multiple packages for dependencies.

:link: Early Testing (Optional)

Several packages exist that are using go-vendor-tools to vendor the dependencies like: opentofu, gopass, or helm.

Do you require ‘QA Blueprint’ support? No.

:link: How To Test

  1. Create the package with go2rpm -p vendor --name $application_name $goipath. For example: go2rpm -p vendor --name opentofu github.com/opentofu/opentofu.
  2. Manually adjust licenses if required as described in Manually detecting licenses.
  3. Modify go-vendor-tools.toml if required as described in Go Vendor Tools Configuration.
  4. Build the package.

:link: User Experience

  • Packages will be easier to update, so users will get up-to-date software more quickly, and Fedora will be closer to upstream. Maintainers will primarily focus on the consumable binary-only package, reducing the need to manage all new dependencies or update them while ensuring nothing breaks.
  • Packages will use the same dependency versions as upstream, providing all features present in upstream binary distributions.
  • It will be easier to package Golang applications for Fedora.

:link: Dependencies

  • go-vendor-tools package would be critical for the Go SIG.

:link: Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) Go-sig to keep the lights on maintaining source-only packages requried for current binaries.
  • Contingency deadline: Before F43 branches.
  • Blocks release? No

:link: Documentation

:link: Release Notes

Last edited by @amoloney 2025-06-02T20:38:30Z

Last edited by @amoloney 2025-06-02T20:38:30Z

2 Likes
2

How do you feel about the proposal as written?

  • Strongly in favor
  • In favor, with reservations
  • Neutral
  • Opposed, but could be convinced
  • Strongly opposed
0 voters

If you are in favor but have reservations, or are opposed but something could change your mind, please explain in a reply.

We want everyone to be heard, but many posts repeating the same thing actually makes that harder. If you have something new to say, please say it. If, instead, you find someone has already covered what you’d like to express, please simply give that post a :heart: instead of reiterating. You can even do this by email, by replying with the heart emoji or just “+1”. This will make long topics easier to follow.

Please note that this is an advisory “straw poll” meant to gauge sentiment. It isn’t a vote or a scientific survey. See About the Change Proposals category for more about the Change Process and moderation policy.