Which LLM-gen-AI model is more aligned with Fedora?

Project Discussion
, , ,
1

Hi folks :waving_hand: I’m opening a discussion to gather thoughts on open source LLM-gen-AI versus proprietary frontier models, specifically looking at how we balance raw performance against software freedom.

Currently, LLM usage across Fedora contributor workflows is fragmented. Some teams use cutting-edge proprietary models, others rely on open-weight options, and some optimize for cost-efficient models. As LLM-gen-AI shows up more and more in Fedora, there is a question to wonder: How much model capability and effectiveness are we willing to trade off in exchange for full transparency, reproducibility, and open licensing?

To ground this debate, consider two models positioned at different points on the openness spectrum:

  • Olmo (Allen Institute): Built by a 501(c)(3) nonprofit, Olmo offers fully open datasets, transparent licensing, and public modeling/training tools through OLMo-core. It strongly aligns with the Four Freedoms and Fedora’s core Freedom foundation, though like most fully open models, it may lag behind proprietary frontier LLMs in raw effectiveness.

  • Nemotron (NVIDIA): Backed by massive enterprise R&D, Nemotron offers impressive performance and speed. While it does largely use open source datasets, NVIDIA does rely on proprietary training tooling and some proprietary third-party datasets. It is more open and public than other open-weight models, putting it closer into the open source field than many other models out there.

While running these models in a homelab isn’t realistic for most, upstream Kubernetes projects like KubeFlow are bringing self-hosted inference closer to reach. If Fedora were to eventually provide centralized inference infrastructure for project workflows, we would need to know what standard and level of accountability to hold these LLM-gen-AI models to.

Questions for Discussion:

  1. How do you view the trade-off between peak performance/effectiveness and strict adherence to open source & Free Software principles?
  2. If Fedora provides standardized inference infrastructure in the future, which model (or architectural philosophy) best fits our project values?
  3. Are there other truly open LLMs with transparent training datasets and licenses we should evaluate? (If suggesting one, please include details on its training data licensing!)
2

Thanks for kicking this off. I think open weight is good enough for Fedora. “Fully open” (data + code + weights) should be the goal, not the bar we gate on.

On the Four Freedoms, open-weight models already clear almost everything the fully-open ideal does:

Freedom Fully Open Open Weights Closed
0 — run for any purpose :white_check_mark: :white_check_mark: :cross_mark: EULA / ToS govern use
1 — study and change :white_check_mark: data, code, recipe published :warning: can finetune/quantize; can’t audit how it was built :cross_mark: no access to weights
2 — redistribute copies :white_check_mark: :white_check_mark: including quantized GGUFs :cross_mark:
3 — distribute modified versions :white_check_mark: :white_check_mark: :cross_mark:

The only difference separating a fully-open model from an open-weight one is Freedom 1.

Unfortunately as you said fully-open model like Olmo sits far behind today’s open weights. I think this is why fully open cannot be the bar today.

AFAIK, Olmo or other fully open source models are not deployed by any Inference providers (Hugging Face currently lists Olmo as “not deployed by any Inference Provider”).
To me that’s a big gap, If I can’t reach the same model through my usual provider, prompts and skills don’t transfer, and we’ve built a Fedora-specific lock-in instead of avoiding one.

6 Likes
3

if we’re going to use LLMs at all, we should at least have the competency to use the ones that aren’t tied to proprietary code and services.
full open models, 100%.
(and, just to make this clearer: i would heavily discourage using any models where the authors have not published their data sources, for reasons relating to copyright and patent law.)

1 Like
4

I would like to mention here that the performance problem is not a speed problem, it’s more a risk of misrepresentation of meaning when summarizing a text for example, or introducing bugs when writing code. I don’t mind a model running more slowly, but I do mind the errors.

In my opinon finetuning is like having a configuration file, I would have put a red cross in this box. Also, since it’s impossible to modify, I would have put a red cross in the Freedom 3 box for open weights.

My understanding is that open weight models are like freeware. You can run it but you can’t see how it’s made or modify it. I’m sure this analogy has its limits, but I think it’s still pretty relevant for this decision.
Models that publish their training data would be the equivalent of open-source software in this regard (provided that modified version can be redistributed). However, the infrastructure and power required to actually rebuild a model is such that it seems unrealistic to use our own locally-recompiled model, as we do for software. But it’s possible and allowed.

That said, we do allow closed source software in the distribution for the special case of firmware, so there’s a precedent.
I agree that there’s a case to be made for using freeware / open-weight-ware (?) right now because it is currently superior in features and quality to open-source software / open-training-data modelware (??), especially if it can move us away from the big proprietary models.

My use of LLMs is pretty limited (just summarizing text) so I’d prefer sticking to fully open models as far as I’m concerned.

EDIT: one more thing, Olmo is available on several providers: OpenRouter, Cirrascale and Parasail.

1 Like
5

To be honest, I’m having some trouble figuring out what exactly we’re trying to discuss here but I’ll try to contribute.

I think we need to have a way to measure these things instead of just relying on subjective feedback from individuals if we’re doing comparisons. Are you talking about relying on existing benchmarks? Are you talking about setting up something that can create benchmarks on a Fedora-specific setup?

Are there discussions about Fedora hosting some AI services? It sounds like that’s what you’re talking about but it also sounds like there are no current plans to do so. Am I missing something?

Considering how quickly things have been changing in the LLM space, is this conversation putting the cart before the horse?

I think that there are plenty of organizations who are focusing purely on performance and I sincerely doubt that we’d be able to compete on model iteration speed or absolute performance, especially if we’re talking about user-owned or Fedora-owned hardware.

If Fedora does end up having some sort of inference infrastructure, I would prefer to see be as open as possible from the hardware drivers and up.

I think there is also a question about the contents of those models. There are open questions on how ethical or legal the process of gathering training data has been for some models that don’t allow auditing of their training data or training process. Even if the legal risk is determined to be minimal, do we, as a project, want to be seen as supporting workflows that have gained negative attention?

6

A quick note, I shortened the title of this topic to focus on the main question being asked and to avoid getting indexed too heavily on trademarked names.

7

I think I’m going to be the dissonance voice here. :grinning_face_with_smiling_eyes:

I’m not sure I agree with the premise that Fedora needs an alignment with LLMs at all.

There are certainly legitimate questions around how contributors use LLMs: privacy, handling of project data, attribution, licensing of generated material, reproducibility, and perhaps whether particular workflows are appropriate for AI assistance. Those are practical contributor-policy questions.

But that’s somewhat different from deciding which kind of LLM aligns with Fedora’s values.

Why does Fedora itself need to choose between proprietary frontier models, open-weight models, or fully Free/Open models? If a contributor uses an external service to help write a script, summarize documentation, or debug something, I’m not sure the model’s licensing philosophy needs to become a Fedora project concern.

I’d see a stronger case for Fedora caring about the infrastructure when Fedora itself is providing it. If the project operates centralized inference infrastructure, then questions about what data is sent to it, who can access it, how the service is operated, and whether contributors can reproduce or audit the service become directly relevant.

Otherwise, I think we risk solving a problem that hasn’t really been established yet.

So before debating which model best fits Fedora, I’d first ask: what does Fedora actually need from an LLM that it cannot simply leave as a contributor tooling choice?

6 Likes
8

+1: unless we’re planning to host inference providers, I’m not sure how this matters. Different contributors will have access to different models, and we’ll use what works for us.

I understand the AI-contribution guidelines too, but they’re not enforceable, right? (Not even on stricter platforms like codeberg—unless they’ve figured out a way of detecting AI generated work). We’re relying on people self-reporting.

So, what is the use case here that requires this decision to be made (or is this a general data gathering exercise at this point)? Are we looking at extending the AI contributing guidelines to say “You can only use models X, Y, Z to contribute to Fedora”?

This is worth a look, maintained by the opencode folks:

And HF, of course:

To answer the questions:

  1. How do you view the trade-off between peak performance/effectiveness and strict adherence to open source & Free Software principles?

There doesn’t need to be a trade-off. There’s no technical reason why FOSS models can’t be as performant as proprietary ones. It’s a question of resources. The Allen Institute, for example, even though well funded, does not have the resources of commercial for-profit model generators that are seeing outsized investment.

If we accept “open models will not be as performant as proprietary ones” (performant as in correctness, task completion time, “reasoning”, NOT speed—which depends more on how much hardware you can cobble together for your inference provider), the trade-off is more about “which model is ‘good enough’”, because from my usage—one does not need to use the newest most expensive frontier model for most things. It’s why providers like OpenCode/go only provide low-cost models, because a lot of them are sufficiently good at coding and related agentic tasks (with the harnesses around them, of course). See the OpenCode/go cataglogue here for example:

  1. If Fedora provides standardized inference infrastructure in the future, which model (or architectural philosophy) best fits our project values?

Ideally we’d use only completely open models, but given that we have been pragmatic recently to try and balance FOSS with usability—ensuring user can “get stuff done” (making it easier to install Nvidia drivers, for example), I wonder if people will be happy enough with less open models if we can implement Zero Data Retention (ZDR) policies. But that’s sort of shifting the question to “would you use less open models if theyre hosted by us securely and your data isnt used for any further training”.

Given that I’m already using less open models, I’d probably be OK with that—but would prefer fully open models if “they’re sufficiently good enough (‘sufficiently’ is task specific)”. It depends a lot on what these will be used for—automation under the hood, or for helping contributors via coding agents etc., for example, for accelerating package maintenance tasks?

  1. Are there other truly open LLMs with transparent training datasets and licenses we should evaluate? (If suggesting one, please include details on its training data licensing!)

I’m not sure—apart from the Allen Institute one, I’m not currently aware of any. Another associated question here: “evaluate” how, with what metrics?

1 Like
9

I think that is exactly what we are talking about…hosting vector processing capability meant to engage with the technology in a way that demonstrates that the local language models (the other LLM) have real value for the collaborative work contributors do in this project.

I keep looking at what gregkh is doing with his gregkh_clanker_t1000 local hardware upstream in the linux kernel development process. What he’s doing gives me hope. I actually picked up my own personal framework desktop that I think mimics his local hardware setup so I can follow his pattern and find ways to use the technology. I think gregkh’s experience with his clanker_t1000 stands as evidence that there is value in local hardware running local models and we don’t have to rely on the frontier inference as a service business model. We can have control and get value and take a more ethical approach to the technology. But it will take work.

I think we can go further than what each of us can do individually by getting our own AMD halo developer desktop or AMD mainboard framework desktop systems. Having communal vector processing capability means we can choose to develop collaborative inference workloads that makes sense for the project and offer them for all contributors. An as a service apporoach, that isn’t proprietary but something that democratizes the technology.. because our project wide service should be something individuals could run if they had an inference-capable workstation/home lab. Like I said, what gregkh is doing with his local hardware gives me hope. Its going to take some trial and error because we have to figure out what workloads make sense for us to expose as a projectwide service and we need to figure out what model we want to rely on. People with local hardware are going to be leaders in the conversation because we’ll be able to be out in front by testing and reporting back our experiences using the models on our home lab hardware. This is likely going to be a journey.

1 Like
10

Open Weights do not always mean “could run for any purpose”.

For example, Kimi K3 license can’t be used for specific commercial purpose, and have some badgeware conditions that are deemed non free by analogy in the past.

Llama3, one of the initial Open Weights model (before it was called like this) has restrictive conditions.

And Nemotron license (OpenMDW 1.1) is currently being at the center of some legal controversy. I wouldn’t use it personally mostly because I think the termination clause is morally shady, legally dubious in France (moral rights cannot be waived) and in the EU (right to justice being a fundamental right, I am not sure it can be easily waived in a indirect manner like what NVidia is trying to push).

3 Likes
11

are there reasonable initial models to consider using OpenMDW licensing or one of the Creative Commons variants the project allows for content?

12

sorry.. let me restrict that last question to just Creative Common licensed models.

Having read up on the lwn summary and comment thread I understand the concerns over the other license more.

13

After looking at @ankursinha list, I found a few under Apache 2 license, like Qwen3, Ministral, gpt-oss among others.

Given the db is on github, it shouldn’t be too hard to automate a search (get the toml files, download repo from huggingface, check license file, rinse and repeat).

2 Likes
14

Thanks,

I think part of the journey on this is having an understanding of near term versus long term goals. I do not expect that we will be able to stick with a single model on this journey.
What I think is likely is we will identify some model community that we believe is the best long term partner that helps us reach our goals of bringing the technology into an ethical framework that best matches our ideals. The Olmo community might be that long term partner. But I think we also may need to gain experience using models probationally, for non critical uses, that have licensing and training concerns that we can live with.

Like for example, I completely understand the termination language concerns in the OpenMDW license. If we were to start with a model under that license, the risk is we’d lose the rights to use that model if individuals were to use their copyright rights on related software. I agree with Fontana’s analysis (as I usually do)

However, I think for probational, optional services, that risk may be okay… if we are explicit that we plan to evaluate and switch models before the service is considered reliable using an OpenMDW model in the short term while we experiment to identify workloads we think have value is probably reasonable. The risk is we shutdown the service or switch models, but its likely as we experiment we are going to be doing those actions anyways.

What we need to make sure we do in the short term is make sure anything we do allows us to reasonably switch models. We may even want to explicitly say anything we spin up in the short term MUST A/B test models because part of the goals is model evaluation.

1 Like
15

Hey everyone! :waving_hand:

Following up on our discussion here, I brought forward this topic to today’s (October 14, 2026) Fedora RHEL CentOS Leads (FRCL) meeting to diving deeply into this exact thread and the core question: Which LLM/gen-AI model is more aligned with Fedora?

As a cross-ecosystem sync bridging the Fedora Council, FESCo, and enterprise Linux engineering leadership, FRCL meets to align strategic architecture, engineering direction, and governance across our shared platforms. Below is an executive summary of our discussion, synthesizing the feedback from this Discourse thread with the broader leadership perspectives explored during the meeting.

Note: Google Gemini was used to summarize the meeting transcript and summary of the public meeting. I edited the AI-generated output before posting. If you notice mistakes, please provide a correction as a reply to this topic.

:memo: Executive Summary: Aligning AI Models, Infrastructure, and Fedora Foundations

1. User Choice vs. Project Defaults (“It’s Your OS”)

A central consensus echoed the points raised here on Fedora Discussion by Flrn Rnd (@flrnd) and Ankur Sinha (@ankursinha): we must clearly differentiate between individual user freedom and community-provided infrastructure.

  • User Freedom: Consistent with our slogan “It’s your OS,” the user’s definition of “yours” is multi-faceted. Users retain total autonomy to execute whatever models they choose on their own workstations—whether that is a fully open model, an open-weight model, or an external commercial frontier API—without distribution-level policing.

  • Project Infrastructure & Defaults: When Fedora provides centralized infrastructure (e.g., automated PR/package review bots, build infrastructure, or contributor tooling) or ships default options, we must adhere to high standards of open source alignment, auditable training data, and license compliance. Much like how Fedora curates opinionated defaults for Editions and Spins while leaving packaging repositories wide open, community infrastructure requires an intentional default.

2. Revisiting the AI-Assisted Contribution Policy: From Procedure to Philosophy

Meeting chair Brendan Conoboy (@blc) noted that Fedora’s initial AI-Assisted Contributions Policy could be ready for a milestone review. While that policy served as an exemplary model of iterative community input and put Fedora ahead of the curve in establishing clear operational guidelines, it is fundamentally procedural: it dictates how contributors must disclose AI usage.

What the community is grappling with now is the missing philosophical framework—defining where, why, and under what ethical criteria Fedora should adopt AI capabilities. Rather than hard-coding transient model names into administrative policy, folks generally agreed that Fedora needs an agile, values-based framework (as Steve Milner [@ashcrow] framed it, “agile, but not fragile”) to guide model selection dynamically over time.

3. Decoupling Tooling: Model Neutrality and Agent Skills

David Duncan (@davdunc) and Jef Spaleta (@jspaleta) challenged the conventional “open core” comparison by framing model weights as data matrices—comparable to database tables or graphic assets (JPEG/PNG)—rather than executable code. Under this model, the paramount open source requirement is ensuring that the runtime engines, scaffolding, and harness applications interpreting the weights are 100% Free and Open Source Software (FOSS).

Rather than binding Fedora to a single model in an ecosystem that shifts bi-weekly, engineering focus must prioritize tool freedom:

  • Standardized Agent Skills: Continuing work spearheaded in the AI/ML SIG by Clement Verna (@cverna), Justin Wheeler (@jflory7), Tomas Tomecek (@ttomecek), Laura Barcziová (@lbarczio), and Carol Chen (@cybette) contributor tooling should adhere to model-neutral, common agent skill specifications.

  • Plug-and-Play Architectures: Contributor-facing applications (such as Aurélien Bompard’s [@abompard] tooling for This Week in Fedora) should remain completely FOSS and expose standardized OpenAI-compatible API endpoints. This allows individual maintainers or infra instances to swap underlying backend models seamlessly without rewriting contributor tooling.

4. Environmental Footprint vs. Open Source Absolutism

We surfaced the critical tension between software openness and ecological/operational cost:

  • Where do we draw the line between “free as in freedom” and “free as in ‘I’m not paying the hyperscaler bill’”?

  • Fully open models (like Allen Institute’s OLMo) provide complete data transparency, but currently exhibit significant usability, capability, and efficiency trade-offs. Conversely, highly optimized open-weight models (such as Nemotron, or DeepSeek and Qwen—the latter adopted by Debian for its developer inference portal) demand substantially lower energy and compute power for equivalent reasoning tasks.

  • If an uncompromisingly open model requires an order of magnitude more electricity and compute overhead to achieve usable results, Fedora should weigh environmental stewardship against licensing purism.

5. Local Inference vs. Global Contributor Equity

While local inference gives users complete sovereignty over their data and allows inference to be offset by renewable local power (such as home solar), Diego Herrera (@dherrera) raised a critical warning: relying solely on local inference risks becoming an inadvertent form of community gatekeeping.

Many global contributors—particularly across Latin America, Africa, and Southeast Asia (such as newcomers joining via Outreachy)—contribute using legacy hardware with only 4 GB of RAM. Expecting contributors to run 8B+ parameter models locally shuts them out of AI-augmented workflows. To bridge this digital divide, Fedora must complement local enablement with shared, opt-in community inference services backed by transparent project governance.

6. Open Upstream Stacks and Hardware Vendor Engagement

Tim Flink (@tflink) highlighted that debates around AI openness frequently neglect the compute runtime and firmware layer. AMD’s ROCm compute tools and firmware are completely open source, with AMD engineers actively packaging the entire stack directly into Fedora Rawhide and EPEL. Furthermore, AMD has provided upstream GPUs for Fedora’s newly-acquired project servers.

  • Upstream Packaging Friction: Gordon Messmer (@gmessmer) pointed out that Rawhide’s rolling-release build constraints make building complex AI stacks extraordinarily difficult, historically pushing developers into third-party container workflows. Work is underway on policies and build routes to build software outside Rawhide’s immediate constraints.

  • Vendor Process Modeling: Fedora should formalize a standard onboarding and enablement model for third-party hardware vendors who invest in upstreaming their runtimes into Fedora, using AMD’s transparent upstream engagement as a blueprint.

1 Like