Dist-git decoupling investigation

Project Discussion
1

Hi everyone,

We would like to inform you that Advanced Reconnaissance Crew (ARC) from the CPE team is currently conducting research into decoupling dist-git from its pagure-related dependencies.

This investigation has come about as part of discussions that arose from Issue #26: DistGit to GitLab Move - initiatives-proposal - Pagure.io.

The investigation has the following objectives:

  1. Enumerate and write down a description of all the integrations we rely on in our current git forge (Pagure)
  2. Create a list of integrations to continue after moving to different git forge
  3. Write a recommendation plan on how we can loosely couple these integrations generically with another git forge (make it git forge agnostic as possible)
  4. Add the list and descriptions of integrations to the right place in our documentation

This will then inform future design requirements and the next set of objectives that looks like this:

  • Map API calls that need to remain
  • Removing the Pagure from dist-git
  • Base git as backend
    ** What functionality needs to move to dist-git when git is used as base backend

You can find a list of existing work items here: Issues - fedora-infra/arc - Pagure.io
You can also follow along with the findings on our HackMD: ARC Dist-Git decoupling & ecosystem mapping - HackMD

On behalf of ARC,
Michal

4 Likes
2

The investigation is now finished and the final document is available.

We are now looking for feedback for the document. Feel free to reply in this thread.

1 Like
3

I don’t mean to be rude, but:

The replacement would be a system chiefly consisting of three elements - Messages Handler, Dist Git Service and Compatibility Metaservice.

So, looks like a decision to drop Pagure/pagure-dist-git has already been made; in order to do so, a new system which will work as compatibility layer between other services and the new git forge must be designed from scratch and deployed; a new git forge has to be chosen, set up and the data migrated.

Isn’t simpler to continue with Pagure? Who will be responsible for maintaining and running the new compatibility layer tool after it’s deployed? What’s the gain here on dropping Pagure?

4

Very interesting task you’ve got here. Thanks. )

I feel like What is Dist-git? is a missing chapter.

I like how the interaction between Dist-git and each other tool is catalogued and represented by a separate page.

I miss same pages for summarizing Dist-git User eXperience stories. I am not aware of Dist-git workflow to see how it fits different forges. Explaining the workflow in a simple and friendly manner would enable people skilled in UX/UI to sketch needed workflows, and then evaluate forges with that design in mind.

The Web Interface chapter in particular gives me a feeling that something is not right:

The workflow is swamped into too many things with strange names. I feel like it is all more complicated than Homebrew, Alpine/Arch, but with my current level of understanding I can’t compare.

And yes, more diagrams is awesome.

5

We just investigated the possibility, the decision is not ours to make. That discussion is going on here. This solution will work even with Pagure, but give us more freedom in case we ever decide to use something else.

6

You are right that one is missing and it should be on index page. I created an issue for this.

We tried to do that in Pagure Dist Git Interactions by Fedora Packagers section. There isn’t anything like this existing in Fedora, so we didn’t have much to start with.

We just looked at the services the dist-git is directly interacting with, it would take us much longer to describe whole packager workflow and this was not in the scope of investigation. This is why the links are there for people to look at what those are.

7

That’s a very good start. I’d move that page into top level section called Workflows, so that it is more visible. And renamed it to Fedora Packager’s Workflow.

The current description is missing the workflow when the Packager edits specs online. It is the same as PR based workflow, but the PR is created through forking and editing in Pagure interface. As I don’t know what dist-git does, I can’t say if that makes sense here.

We just looked at the services the dist-git is directly interacting with, it would take us much longer to describe whole packager workflow and this was not in the scope of investigation.

It is still worth to add a Basic Workflow section that describes intended operations from a typical day of dist-git user. Especially if it is not described anywhere.

8

Moving it to separate section is not a bad idea.

I don’t even know if this is recommended way of doing that, all of this should be done through fedpkg as a packager of the project you don’t need to create a fork of the repository. In our proposed solution this kind of operations will be handled by the git forge directly.

I would like to see that in packager documentation as this is not part of this investigation, but is definitely useful for Fedora packagers.

9

dist-git is not a product, exactly; it’s a term for the overall concept that Fedora package definitions (spec files, source lists, and patches) are stored in git repositories, and the build system (that’s Koji) retrieves those things from git when performing a build. The name was logically derived from the predecessor system, referred to as dist-cvs, which stored all those things in CVS. It also refers to the concrete artifacts of the concept - the git repositories themselves.

“Pagure Dist Git” is the specific Pagure instance that acts as a forge-style interface to the dist-git system, which predates it (before we had a Pagure instance, we had a cgit instance on dist-git for read-only web viewing).

10

yeah, except confusingly, we have also a project/thing called
‘pagure-dist-git’ that is the compatibility layer run on top of pagure
on src.fedoraproject.org. :slight_smile: ( ie, Overview - pagure-dist-git - Pagure.io )