this is rough list of packages which IMO stand out from current guidelines
The guidelines explicitly say that Packages can depend on virtual file Provides. Package specify virtual Provides with a file name. This Provides becomes part of the main package metadata and other packages can depend on it without the extra filelist metadata:
I’m not sure why rt requires the font path specifically rather than google-droid-sans-fonts – I don’t think there’s another provider for that. (But if there is, seems like a virtual provides could handle the issue.)
libsolv already supports this capability. Insofar as predicting whether the presence of filelists would help resolve failures is pretty easy: if a failed dependency starts with a / (that is, it’s a file path), attempt to download filelists, reload the solver cache with the extra data, and then try again. If it fails that time, then it’s obviously invalid.
Hello Neal, I want to assure you that I am not overlooking your comment. In fact, I have already begun the analysis of potential implementation in DNF. However, it’s turning out to be trickier than it might seem from the DNF perspective.
Hi Tom, thanks for your feedback. I will reword it to better reflect the SHOULD NOT and MUST NOT definitions. To be honest, I was not aware of these formal definitions before and had perceived it just from reading the text as basically forbidden things. Unfortunately, this now cast a negative light on the situation. Although, I believe the proposal remains viable, as this is a Fedora related proposal and as Zbigniew stated, on the Fedora side there is currently just a couple of packages not complying with the discussed rule and they’re not system-critical. Also, on other distributions, the state could remain as it was till now by simply configuring the behavior to continue downloading filelists.
Speaking about the dynamically loading filelists, I’d love to have such a feature in dnf. But as of now, it appears to require a substantial amount of non-trivial work, involving the refactoring of numerous existing blocks in the dnf/libdnf codebase. I don’t think handling this for F40 is realistic. Moreover, given that it impacts essential parts of the code, I view it as a considerable risk. I think it’s better to plan such a feature for DNF5, where workflows are better streamlined.
I think the best way forward here would be to add the following two items to proposal’s scope:
The packages that currently need the filelist are updated to stop that (by merging the existing pull requests)
The Packaging guidelines are updated to say MUST NOT
The analysis that has been done in this thread shows that filelists are basically unused, and the virtual file provide workaround is there for the rare cases where filelists have some use.
One thing I am thinking about are external repositories. What should be done if some of them use file dependencies, either internally or by referring to files provided by Fedora packages? Just let the user figure pass the flag to enable them in such case?
Keeping in mind that the idea is to propose replacing DNF4 with DNF5 in Fedora Linux 41, should we then not even bother with this change and just have dynamically loading filelists be part of the switch to DNF v5?
Yes, this is true but only 9 packages from 7 sources are effected by the change. The original list for Fedora 38 was about 70 packages, therefore there is a huge progress.
The new behavior will be configurable and when a user will experience an issue then we can provide the reasonable suggestion how to resolve it.
Personally, I am supporting to use MUST NOT instead of SHOULD NOT in guidelines in solution.
Dynamic downloading and loading filelists sounds like a good idea, but it will often end up with worst performance or not nice behavior then the current implementation (always download and load filelists), because often the filelist will be gone from all mirrors. Then the solution is - fail or re-download all repository and reset everything in DNF (start from scratch).
The feature was requested time ago and it significantly improves user experience and reduce cost of using Fedora (download, CPU, RAM, storage). The feature was not deliverable in past, because to many packages depended on FILELISTS, but this is not valid anymore because only few packages are affected by the change (2023-11-22 only 7 packages, Fedora 40). The simple workaround for remaining packages is to add Provides: <file> to original file provider, but there are better ways.
The lazy loading functionality may still pose challenges in DNF5, and I cannot confirm right now its presence in Fedora 41 due to the ongoing workload.
On the other hand, the implementation for the current proposal is essentially ready, making the switch quick and straightforward. Also, even if DNF5 is switched as the default in Fedora 41, this would be still beneficial for earlier Fedora versions and other distributions.
This would be a pretty nice solution if dnf always downloads file list and then they can be loaded lazily. But it means that dnf have to downloads all filelists (no change in amount od downloaded metadata).
In case that dnf will not have downloaded filelists then there is a chance that file described in repomd is not available anywhere, because metadata were updated. This might happened with Fedora updates repository. The repository is set to refresh metadata not before 6h metadata_expire=6h. When this happens DNF has only bad options - mark all repositories with not downloadable filelists as expired and fail or start loading and updating all repositories from scratch. @ngompa May I ask you which scenario have you in mind when you mentioned lazy loading of FILELISTS?