Yes! 
Thatās where Iāve started, but beyond thatā¦ A remix has a connected to Fedora, but I would like Fedora to be connected to the initiative, because I believe that everything that Fedora builds up, builds up Fedora.
Sure, but sigs that also make a remix are definitely part of fedora.
For example folks in the mobility sig try and reduce the delta between
what they need to work on moble hardware and whats in fedora, but being
a remix allows them to use a non fedora kernel which they must do until
thereās upstream support for their hardware.
I would like Fedora to provide the platforms and frameworks that machine learning (/AI) developers need, because developers will recommend the systems that they use.
Sure, understood.
I would like Fedora to build channels that direct developers to a space where they can collaborate, and channels that direct changes back upstream into Fedora, whenever possible. Upstreaming successful changes into Fedora should be an explicit goal, as it is an explicit goal for Fedora to upstream changes we make to components.
A sig doing a remix does this though.
I would like Fedora to take part in community building, helping developers find interested users, and helping users find interesting projects, because I believe that the communities we promote will promote the project in return.
Sure.
A few more questions:
This LTS kernelā¦ is that really needed? Or is there a way to maintain
a patchset against the fedora kernel for this? Could that be less work (
I am sure it would be less disruptive for testing/quality/etc).
Iād love to hear some feedback on this from Fedora kernel maintainersā¦
I worry that doing this would open the door to a bunch of custom kernels
for different cases where the support is not upstreamā¦
2 Likes
+1, I think what the mobility group has been doing with Pocketblue is really cool and should be a good starting point for that effort.
1 Like
Please add me as an interested party.
This is a good topic to bring up at the ai/ml meetings.
I would like to get into the details of the timelines as I feel we already have AI in Fedora now. For example there is no reason why ollama could be used for some agentic ai tools now.
It would be good if hardware acceleration for ai could also be used generally for Fedora. I am wondering as the pytorch maintainer, how that would that happen and who could also help with its other backends. Other AI frameworks have deep dependencies that need to be packaged. It would be good if we could settle on some frameworks and backends to work towards.
I know people have offered to help with supporting pytorch, ollama, and llama-cpp from not only adjacent SIGs, but also even alternative architectures. I think itād be a good idea if you reached out to them and worked with them to add them as maintainers first. These stacks are hardware centric, so working with those teams to share in the support work is the first step before doing anything else.
I have. That is why we have vulkan support in ollama and llama-cpp and why I think we could do something agentic now. With vulkan support we have broad coverage.
Sorry for the late reply, this thread was just brought to my attention.
As the person who deals with every bug reported to Fedoraās kernel bugzilla, you would be quite surprised. There is not much of an increase in hardware support regressions in rebases vs stable updates. The current stable update cycles tend to run in the hundreds of patches backported. Also, as vendors have started moving functionality out of drivers and into firmware, linux-firmware updates seem to introduce a large number of regressions as well. Of course they get reported as kernel bugs most often, because users donāt use a new firmware until they rebuild the initramfs and the bug is exposed in the new kernel, but doesnāt appear in the previous kernel because the initramfs for it still contains the old firmware.
The purpose of test weeks isnāt so much what you think. Rarely do we see some massive regression in hardware support. Most of that is caught in rawhide testing. What we are actually looking for in test weeks is breaks in userspace on older releases because the kernel has dropped or changed something which works fine on rawhide or even the current release, but might not work so well on an older supported Fedora release.
1 Like
This is absolutely not true. By official policy, we will not hold up a rebase because the nvidia module will not build. In practice, this has not been a problem in a decade or so. Not because of anything we have done, but because Nvidia has changed their policies on upstream support. They support upstream at .0. The policy we have isnāt really written down anywhere other than an email thread 15 years ago, and I would likely hold things up these days, at least for the open source driver. Of course most of the build issues used to revolve around GPL only exports, which donāt impact the open source driver at all.
The only real modules which occasionally take a little bit to catch up are zfs (upstream seems to be dying off here, or at least slowing down), and VirtualBox.
I know that one of the biggest reasons people would like to have kernel modules built and packaged in Fedora has to do with secure boot signing. The issues here are not really technical, but it has been discussed at various points internally and there is just not something that would be allowed.
I did speak to Dave Airlie about the possibility of bringing the open source driver into our kernel as a stop gap until such time as Nova is ready, and I was told that it is not really possible even. It would take considerable work to make it even buildable as an in tree module.
This leaves us with Nova. Development is moving quickly there, and I expect it will actually be useful before we could ever get through the legal hurdles of signing an out of tree module. I expect it may even be useful by the F45 release, certainly before F46.
2 Likes
Neal also disputed that statement, so Iāll rephrase it.
When I discussed the idea of building the OpenRM driver in the Fedora kernel with you, one of the reasons that you cited for not doing that is that it might not build after the kernel is rebased, and that might delay rebasing to a new kernel. I think thatās a perfectly reasonable objection. Itās also a really good argument for having the option of one ālongtermā kernel per Fedora release.
Having said that, a ālongtermā kernel package is a means to an end in the context of this discussion, it is not itself the goal. The goal is the ability to support hardware that requires an out-of-tree module, one way or another.
I did it and it didnāt seem like that much work. Itās probably a little sloppy, and I am probably not done, per se, in the sense that I will probably work on it further. But I find the āconsiderable workā argument unconvincing.
It is not so trivial that I am asking you to do it, but itās not so much work that I couldnāt do it myself.
I honestly had not looked at it myself, but was expressing to you what Dave Airlie expressed to me when I asked him about the feasibility of it. Specifically:
ācanāt be done in the kernel itself, has to be a kmod, well 4 kmods per nvidia version per unique kernel nvr releaseā and āScripting it is also difficult. Itās not just every fedora kernel release, you also have to monitor 5 streams of nvidia driver releases.ā
Perhaps that is just the full Nvidia solution, and not simply the GPU portion of it. And as I expressed earlier in the thread, we do have a āwe wonāt hold up a rebase for nvidiaā policy, but in practice that has not been a problem in a very long time. When it was a problem, it was always because Nvidia depended on something that was changed to EXPORT_SYMBOL_GPL which should not be a problem for the OpenRM driver as it is dual licensed. Nvidia themselves have a policy that their drivers are supported from the time Linus does an actual release, so that is long before test week. Bringing the driver in tree though could break rawhide badly, as there is no guarantee things work before Linus does a release.
Itās not hard to bodge another driver into the tree. Itās just annoying and not really worth it. It means that the kernel is implicitly tainted and thatās going to upset the upstream project a fair bit.
OpenZFS has their own CI that tracks Fedora and CentOS Stream to keep compatibility. As for VirtualBox, they are actively working to make their system work with KVM.
NVIDIAās OpenRM driver is the only one weāve been discussing, and Nova is essentially the replacement for it. I donāt think anyone has really discussed the other drivers in a Fedora context that come from Mellanox. Iād rather push them to resume upstreaming things instead.
ā¦ sorry, arriving a bit late to the party.
Iāve now read or at least skimmed most of the replies above, and Iām not sure that I would like Fedora to make this an āobjectiveā. Not primarily because of technical reasons, but because of philosophical differences.
Fedora is already being perceived as ātainted by AIā due to at least one other Council decision (the AI contribution policy) and by its close association with Red Hat, which is driving users and contributors away to distributions which are perceived as not drinking the AI Kool-Aid (- whether deserved or not). I fear that making anything āAIā in its title an official Objective would further alienate users and contributors like this, and further damage project reputation.
I am not necessarily opposed to some of the technical work that you outline - making akmods / kmods work better (or making them obsolete) would be great, an LTS kernel might be interesting, better hardware support is always good, etc. - however, while I agree with some of the steps, I certainly wouldnāt agree with the goal here.
On the other hand, reading your proposal, I was also wondering: Why is this not a CentOS SIG instead? It sounds as if that might be a much better fit (stable kernel, SIG support for building kernel modules, etc.). Some of the technical puzzle pieces could get developed in Fedora first, but shipping the āspinā as CentOS SIG deliverables instead would sidestep many problems (and provide some philosophical distance between one more āAIā effort and the Fedora Project).
3 Likes
The Open Source Definition - Open Source Initiative #6: āNo Discrimination Against Fields of Endeavorā
I think that one of the clearly evidenced foundations of the Free Software / Open Source ideology is that we are free to work on the things we are interested in working on, and we are not free to stop other people from working on the things that they are interested in working on.
Fedoraās policies on permitted content are consistent with the Open Source definition; there are no policies that ban software based on field of endeavor.
Generally, I think the idea that this decision should be made on the basis of the field of endeavor is one that should be considered very carefully, because it calls into question whether Fedora is an open source community project.
I want to be very specific about that statement. The council is expected to make decisions about the project. Deciding either way about this proposal does not call into question whether Fedora is an open source community project. Making that decision based on the field of endeavor does, in my opinion. Fedora doesnāt ship pre-built binaries or not-open-source software. We are dedicated to open source software, and our policies allow us to make assertions about provenance and security. I think thatās good for our reputation, even when itās inconvenient. Fedora doesnāt ship patent-encumbered software even when otherwise Free implementations exist. Not always good for our reputation, but at least defensible. If we start discriminating on the basis of the field of endeavor, I think that would actually be really bad for our reputation.
Is the title the only thing standing in the way of moving forward? Would you approve of this work if it were a āMachine Learning Developer Desktop Initiativeā?
Because ML is an emerging technology, not a mature one. Software in this field benefits from a rapid release cycle, not a very slow one. Delivering features to developers every 3 years is not aligned with the needs of an emerging technology field.
I believe that the purpose of a distribution is to distribute software. LTS distributions are actually not very good at doing that. The vast majority of software releases in upstream projects are never shipped by any given LTS release.
Perhaps itās a āGPU-Accelerated Computational Desktop Initiativeā? The deliverable that is envisaged could also be useful for workloads that arenāt AI or ML, for example physics and engineering simulations.
3 Likes
Yes, I expect that some of them will be!
GPU acceleration is really a small part of the initiative. But understand, people have been running nvidia drivers with Fedora for a very long time. And while secure boot made it slightly more difficult, it really isnāt that hard to do. I run their open source driver on one of my systems now just to ensure that everything works. More importantly, the GPU side of it will be handled by Nova in a fairly reasonable time frame. Development there is moving quickly on multiple fronts with people working on drm for compute and display, as well as virtualization for vgpu and vfio support.
I do not understand this concern. Any project needs to be able to make decisions of the āwe could do this, but will not / do not want do itā kind - if only to define and shape (and also limit) the scope of the project. āWe disagree with this because of ethical / philosophical considerationsā MUST be a valid reason not to do something.
2 Likes
The principle you are citing is part of the definition of an Open Source Definition by the OSI organization. That doesnāt mean a project must adopt whatever technology to be considered Free Software or Open Source. There are multiple examples of technologies that Fedora project decided not to adopt for multiple reasons.
The argument risks expanding a negative freedom into a positive obligation, which isnāt implied by the OSD.
And to be precise regarding the OSD, you are always free to Fork any Fedora project and integrate whatever you want within it or use Fedora projects to any endeavour even if the maintainer disagrees. So I do not think the argument is compelling here as accepting the premise āyou shouldnāt restrict fields of endeavourā the conclusion of āFedora should accept or endorse technologiesā doesnāt logically follow[1].
Agree 100%.
Even if the field of endeavor is part of the consideration, the decision here is about whether Fedora should align itself with and promote that field as a project objective, not about restricting others from pursuing it. I can see two different topics being mixed together as a single one.
-
All the technical work you already mentioned on the proposal. Like creating a new kernel-module RPM, include user-friendly tools common to various AI back-ends, provide a contribution guide, etc.
-
Make this a Fedora objective, that is Fedora project as a whole will focus on the success and promotion of this.
I donāt see significant opposition to 1).
AFAICS, the concerns are on 2). Fedora has decided multiple times to avoid making other stuff their objective for multiple reasons. Sometimes due to the nature of the technology, sometimes for legal reasons and sometimes because people didnāt think it was feasible to do it.
This is not a question of whether people are free to work in a given field of endeavor. The question is whether Fedora, as a project, should associate its identity and priorities with that field.
3 Likes