Continuity for Linux distributions

The Water Cooler
1

Hi, folks,

I wrote a new article:

I know people who imagine distribution development as the process of piling up the code in the git repository for 6 months and then building it all in one go at the end of those 6 months, so that it can finally be shipped. This is very far from reality. And it is impossible to explain things like CentOS Stream without addressing this confusion.

While it was inspired by some recent CentOS-related debates, it is a more generic overview of how distributions are developed, and therefore may fit this audience.

I’d love to hear your feedback, questions, corrections and so on.

The main idea I was trying to address is that Linux distribution is not just its sources.

3 Likes
2

I haven’t had time to fully digest this, but it’s interesting!

One comment, on

In my opinion it is a big fail for the entire industry to think about a Linux distribution as a fixed set of RPMs and SRPMs on a DVD(or the iso image). I understand where it comes from, but it is a fail anyway.

I think this helps articulate one of the ways rpm-ostree is important.[1] This provides a different and continuous[2] “set”, which theoretically can be tracked back through the build system to expose the whole web — complex, but describable.

Two things will make this even better:

  1. Branching to CentOS Stream and RHEL in a way that preserves Fedora history
  2. Using source git, which extends traceability into each packages’ upstream source history.

Reproducable builds might fit into this somewhere, too — there’s a session at Flock!

  1. Which relates to Objective Review: Immutable variants are the majority of Fedora Linux in use ↩︎

  2. well, at least, step-wise ↩︎

3

I think this helps articulate one of the ways rpm-ostree is important.[1] This provides a different and continuous [2] “set”, which theoretically can be tracked back through the build system to expose the whole web — complex, but describable.

I am not sure images solve it for me.

The direction where I am going is that development of a complex integrated project is not described through propose - test - merge - build - deploy workflow.

It is rather

propose-test-merge-build                          // sources
                     |
                     propose-test-merge-build     // packages
                                          |
                                          propose-test-merge-build  // images (layers)
                                                               |
                                                                    // environments..

And when doing CI we need to respect all layers and have Git-like functionality for each of them, not just the first one.

  • well, at least, step-wise

You should also check my other article on quantum nature of continuous integration :slight_smile:

4

Oh, I’m not thinking about the images. It’s the ostree commits.

… and ostree brings that git-like functionality to the third step, right?

5

… and ostree brings that git-like functionality to the third step, right?

I am not sure on this part. Because i think that the ostree git-like concept spans in a “different dimension”?

I don’t quite understand it, so I am may be saying something completely wrong.

But when I say buildroot needs to be git-like, it is still a flat pool of packages. The tree-like git-like history of the buildroot is based on time (green-blue on the image). But at every point in time, there is still just one associated state of the buildroot.

When we talk about ostree images, their tree structure is not about time. It is also about the layers in the images themselves (base image, image with post install applied, image with package installed,…) And that git-like structure of images exists even at a single point in time (red on the image).

git-trees

So ostree images are git, but that’s a different git? Or just a “larger” git?

6

Well, maybe I should step back.

When you say we need to have “git-like functionality” at all of the source / package / image levels… why? What does that provide that is useful in the distro-building context, and what is it necessary for?

What can we do differently and better with this?

7

Wait, are you saying my beautiful abstractions should have a purpose?!

OK, I have two:

  1. we can align the language

Koji tag is a branch, sidetag is a feature branch, package in the gate is a merge request. Promotion of a package to the release tag is a merge.

This whole Fedora Gating thing is not different from the usual CI workflow.

  1. we can align tooling

Again, we had to implement and maintain a lot of project-specific custom tooling which tracks the history of the buildroot (koji event-id), implements branching, implements tagging, implement inheritance, implement promotion workflows, implements CI, provide dashboards and notifications and so on.

While these are solved problems. Once we stop using the custom language we invented to talk about buildroot and implement it as handling text files in the git repo, we get access to the entire ecosystem of tools which know how to handle it. We literally can run Zuul CI on the buildroot git repo and it would be a complete solution for Fedora Gating.

1 Like