The council has released an announcement yesterday in the Fedora Magazine on the git forge evaluation. At this time, we are leaning towards Forgejo! Before we make the final decision though, we would like to have a two week feedback period before we bring this to a formal vote for council. The article walks you through how we arrived at this stage of the investigation, and has a lot of great links to further documentation on the git forge investigation. I encourage you all to have a read, and if you would like to share some feedback you have, please drop a comment on this thread. Please be mindful of your tone and language used. This is a big topic that has been many months, if not years, in the making and a lot of hard work has gone into reaching this point. I am really looking forward to seeing how the Fedora community will work together next year to bring about this change and thank you all for your patience, understanding and contributions to this topic over the last few, well, years!
As you correctly say the article spends a lot of time explaining how we arrived at the current investigation, which to be frank is not really very interesting or useful at this point, but very little on how the proposed decision was arrived at.
The closest it comes is the link to the investigation results which details the evaluation of each user story against each proposed solution but as far as I can see it makes no attempt to compare those results or suggest any sort of preference.
The article then simply says:
Once the investigation completed, Fedora Council was slated to make the final decision on which forge the project would choose to migrate to.
But there seems to be nothing more about how that evaluation of the results proceeded or arrived at the proposed solution.
What makes it particularly curious is that even a cursory reading of the user story evaluations would suggest that gitlab could fulfill far more of them than forgejo at the current time so presumably there must have been some significant considerations that weight against it and in favour of forgejo?
I wasnāt following this discussion and the proposal to use Forgejo caught me by surprise, but I suspect one of the considerations is that GitLab is quite complicated to host. Itās doable, but can consume considerable sysadmin time. I hope Forgejo is easier.
Both of these forges are suitable, so Iām just glad we have two good options.
Oh I absolutely understand that there are other considerations besides the user stories, and that is certainly one and I can think of others. Itās just surprising that I canāt see any record of other factors that were considered or of how the final comparison happened.
The council members have had a lot of conversation amongst ourselves, both privately as a group, in smaller groups and in one to one calls. Most of the members had their own bias towards one forge over another and when we actually spent some time talking about it, we realised a lot of us had our minds made up already for Forgejo. We just needed to see proof that the user stories would work.
If you want to hear from me about how I reached this preference, I have been meeting with the lead from the ARC team every week doing my due diligence on the options. From what I have been told by the team, who I trust their technical judgement implicitly, Forgejo was the better option (in my opinion) because of:
Its open source nature is stronger
Its written in go, vs ruby, which GitLab uses, and go seems to be more appealing/popular/insert word of choice here for developers to learn so developing whatever features we need might be easier to come by
I also heard from people, when doing my own investigating, that yes in fact GitLab self-hosted is or can be quite complex, and I would really like to make life a little easier for the folks who will run and maintain our git forge, not harder.
Individuals reached the decision that they would like to go with Forgejo through their own way, and if they want to explain as well how they reached it, they are welcome to.
In our last council meeting, we decided we will run with Forgejo. We had all reached the conclusion that thats the forge we wanted and think fits best. If you think this is the wrong decision, please let us know why so we dont make that mistake.
Iām certainly not saying I think the decision is wrong, rather that I didnāt feel that I had the necessary information to evaluate what I think about the decision.
I can absolutely understand that forgejoās single binary structure makes it vastly easier to deploy and I know gitlab has a reputation for being very complex to deploy so that is indeed a clear advantage.
Likewise I can understand that the fully open source nature of forgejo is better than the open code model of gitlab.
On the flip side it appears that forgejoās more limited feature set will either mean dropping current features, having to add features to forgejo ourselves, or bolting features on the side using separate services which obviously starts to bring back more deployment complexity.
How do you resolve that tension? I donāt know, which is why I was interested in how the things were weighed.
I think mostly I was quite frustrated with what seemed to be very poor communication of the decision and how it was reached - the magazine article spends a lot of time on history and the previous attempt to move to gitlab which isnāt very important to current events and then just says there was an investigation but doesnāt really explain the results of the investigation other than that it somehow led to a decision.
I can see in the council minutes that it was suggested that it include āa basic diff of feature sets between Pagure and Forgejoā which would certainly have been a good idea in helping to understand the consequences of the decision even if not the consequences of the alternative.
Thatās fair, and I can completely see where you are coming from. Iām hyper aware of this work so I am talking from a place of assumption - I am assuming people know as much or more than I do about this, which is not always the case. Thereās a really interesting graph that was presented to the council when we met with the investigation team a few weeks ago that really helped me reach my conclusion that forgejo would be the better choice, as it was showing some of the work flows the project has and a very high level estimation of complexity to develop the features needed for Forgejo and GitLab. I will try to find it to post it here as part of this discussion too, but hopefully someone else from the council or the investigation team can post it and talk to what it shows.
The results of the investigation I would say are still not complete. We are missing a summary of what went into it and a summary of results. The write up thatās linked in the article does show results, but in a very detail-heavy way whereas a snappier summary is needed to reach a wider audience. Iāve requested a summary that would be good to have when we finally announce the decision.
For full transparency, when the decision is made, which will be post this final feedback call to make sure we are not proceeding with the wrong choice, but when the decision is (finally) made by a formal council vote, there will be a lot more detail given in the announcement post about this whole endeavour. This announcement should be considered as a āHey this is what were thinking, is this ok? If not, why not - please tell us before its too late!ā. This is also something we will make sure happens in response to hearing some folks concerns that there wasnāt enough information in this initial announcement. This is the Beta, that you for flagging some bugs that we need to fix before Final
FWIW, Iāve offered to help bridge gaps using ghostflow in an earlier thread. While it doesnāt support forgejo today, I have contacted the author of the forgejo-api crate to redesign its API a bit to follow the patterns I describe in this blog post and the author was receptive (the main issue right now is that everything in the crate is async while ghostflow is not yet async through-and-through). If it is something that Fedora is interested in, I can look at finding time for it (of course, Red Hat can also contact my employer for something moreā¦concrete if that is something that is palatable). Contributions are, of course, also welcome.
And I think you have just highlighted the main reason why Forgejo currently wins the argument.
The evaluation of user stories was an important step to verify that we donāt have a particularly strong showstopper in either of the variants. We were looking for blockers - things which would make us say - No, this is absolutely impossible to do.
The result of the evaluation was sort of unsurprising - we came up to conclusion that well, both of the solutions will require work, here or there, but there were no complete blockers.
But then the other line of argument starts to take precedence: if each solution requires work, then it is only a matter of time once we hit the wall with the GitLab CE. We know for a fact (because CentOS workflow uses GitLab EE) that there will be GitLab features we want to use, which are considered Enterprise-only. And we know that GitLab will not allow us to develop and merge these features upstream.
So with GitLab eventually we will be blocked in whatever work we are doing.
With Forgejo - the current situation is the same, work needs to be done. But there is no known āglass ceilingā for it. It is a matter of applying the resources and asking community for help.
Thus we think that together, through community collaboration, we will be more successful. And we hope that Fedora needs will overlap with the needs of other users of the Forgejo system, not limited by some artificial paywall restrictions.
Note that ghostflow-director has implementations of some features of GitLab EE through its core ghostflow crate (e.g., the āstageā is GitLabās āmerge trainsā). Of course, the UI is nowhere near as āintegratedā, but Iāve filed issues to get āclose enoughā features (which are reasonable in and of themselves anyways) for things like the āstageā (of note would be the ability to tie some custom pipeline with an MR so that one ābuild the stageā pipeline can report back to all MRs that it is a part of). One thing that isnāt implemented is something like ācode ownersā and multi-user review, but thereās a feature called āmerge policiesā which support doing checks of that kind of thing but we havenāt needed it yet to implement it ourselves so far. The main thing implemented today is āmake sure CI passesā logic.
I want to thank everyone involved in the requirements gathering and analysis process. Never in Fedora have I seen a project-wide decision be made with this level of research and discussion. It has been a lot of information, but I really like how we have done this in the open and really making sure everyoneās voice is heard.
When we started down the path of finding a replacement for Pagure, I personally felt the only viable option that existed was GitLab CE. I will be honest and say that I was not particularly excited about that being the only option. GitLabās open core approach was disconcerting to me. Would we find ourselves stuck at some point unable to extend functionality because what we were trying to do already existed in the commercial product? Would GitLab entertain us extending the community edition far? Would that even work for us?
On the other hand, I had never heard of Forgejo until we were in these discussions. The more I learned about it, the more I felt it fit Fedoraās needs better than GitLab. The one thing that sticks in my mind is that it is younger than other forges and just doesnāt have the proven track record.
That said, I am excited for us to move to Forgejo and extend it for our needs. I hope we, as Fedora, can extend it to the point where even Red Hat could consider it for use over GitLab for RHEL and CentOS Stream (on their own hosted instances, of course).
Thank you again to everyone involved in the research and thank you to the community for providing all of the input to help make this decision. Now itās time for the hard work as well as thinking about our next infrastructure challenges (like Bugzilla ).
Please find the rundown of the conversations that we have had over here. We have kept our communications across various platforms like Fedora Discussions, Mailing Lists, Flock To Fedora, Fedora Linux Release Parties and other places. You should be able to connect to us with those places - each place for a specific purpose.
Thanks againā¦ I have already read those, it was figuring out the path from those results to the decision that was confusing/frustrating me.
One other I would add I think is that the original announcement soliciting use cases could probably have been better phrased as I completely missed it at the time and the complete lack of replies on the devel list suggests I may not have been the only one. It talks a lot about a āgit forge investigationā without once mentioning replacing pagure and/or dist-git and it repeatedly uses the āARCā abbreviation which I had never encountered before without expanding it or explaining it in any way.
The version here is a little clearer mostly because discourse expanded the links to the text instances with big panels but in the devel list version they were just text links in footnotes so it was easy to miss that there were test instances to look at.
Iāve been using Fedora since 2010 and I still consider myself a beginner. I love Fedora, but I fear the take over of open-source by āfor profitā companies. Just taking MonoDevelop as a prime example, the moment Fedora somehow competes, encroaches on, or interferes in any way with GitLab; GitLab will take steps to eliminate, slow down, infringe, or interfere with Fedora. It might not happen openly or even deliberately, but its not out of the realm of likelyhood.
GitLab competes with MicroSoft, one of the most ruthless companies to exist. Fighting against such a monster will leave collateral damage to anything that touches the fight. Go with GitLab if you want to sink or swim to the end with them (indirectly fight MicroSoft). Otherwise, it might be better to roll your own forge. Fedora seems to have the resources to do it. Keep everything as open source dependent as possible. My heart broke with the passing of MonoDevelop.
