Informed Mirror Management on Fedora

Hello,

I’ve been using F40 since ~ a week before Beta release and it’s now the third time I’ve been facing mirror issues on Fedora.

Issues I’ve had:

  • I’ve had Everything installations failed because of:
    • package versions mismatches (x version of package not available on y mirror)
    • inst.sdboot installations failed because a bugfix version of a package has not landed in the out-of-sync mirror that happened to be selected by the Everything install.
  • I’ve had dnf reinstal <package> fail because “installed version is newer than available”

I’m migrating to Fedora from Arch-based distributions, where Arch infrastructure provides me with both the tools and, especially, the data to make informed decisions on selecting the right mirrors.

Quoted from the link provided above for context:

  • Completion %: The number of mirror checks that have successfully connected and disconnected from the given URL. If this is below 100%, the mirror may be unreliable.
  • μ Delay: The calculated average mirroring delay; e.g. the mean value of last check − last sync for each check of this mirror URL. Due to the timing of mirror checks, any value under one hour should be viewed as ideal.
  • μ Duration: The average (mean) time it took to connect and retrieve the lastsync file from the given URL. Note that this connection time is from the location of the Arch server; your geography may product different results.
  • σ Duration: The standard deviation of the connect and retrieval time. A high standard deviation can indicate an unstable or overloaded mirror.
  • Mirror Score: A very rough calculation for ranking mirrors. It is currently calculated as (hours delay + average duration + standard deviation) / completion percentage. Lower is better.

Is there a place where I can find similar data in order to be able to make an informed decision on mirror selection on Fedora?

If not, are there some default “master” mirrors I can select as guaranteed reliably synced (I really don’t care that much about speed at this point, I just really need a functioning system).

It is rare that I see a mirror be stale, but it does happen.
That usually sorts itself out once the mirror catches up, or a retry picks different mirror.

Are you seeing a persistent failure? Can you share the output of dnf for this failure?

My KDE spin F40’s are all updating without issue.

I did reinstall all my f40’s to avoid the possibility of having a bad xz ever being on my test systems.
So I have started either at the f40 beta or systemctl system-upgraded from f30 to the f40 beta.

For my everything installs:

  • One was the case of sdubby 1.0-8 which solves an issue with inst.sdboot. That was consistently stuck on 1.0-7 for the last 3-4 days (landed on F40 stable 4 days ago). Today I finally get the correct version with the default mirror metalink configuration.
  • The other case was while experimenting with kickstarts, and trying to include updates-testing during install (The exact command I used for this was simply adding repo --name="updates-testing" after the url command that pointed to the main repo). It threw an error of some lib package not being found on any of the mirrors, and terminated the installation. I haven’t tried again, so I’m not sure if it is consistent. This might be on me altogether since I’m in the process of learning and experimenting with how to use kickstart.

For the package reinstall, I was having some issues with chromium, so I was investigating and trying to troubleshoot:

  • dnf info chromium showed version 123.0.6312.105 bold yellow
  • packages.fedoraproject shows the same version is in F40 testing
  • dnf history chromium indicates I got the package 05 Apr
  • sudo dnf reinstall chromium failed as stated above
  • I manually downgraded with sudo dnf downgrade chromium. Now (on chromium-123.0.6312.86-1.fc40.x86_64) running sudo dnf up --refresh shows no available updates.

I think it would be interesting for you to raise these issues with fedora test.
It would be worth asking if the testing are doing is expected to work before the release is made.
I’m guessing that needs to be on the fedora testing mailing.

I tried enabling testing repos with dnf update --refresh --enablerepo=*test* and that did offer to update packages for me.

Re chromium I’m being offer version 123.0.6312.86-1.fc40 and with test repos enabled 123.0.6312.105-1.fc40.

I’m not that familiar with mailing lists in general, and more so with Fedora - I’m still slowly getting familiar with the project’s (seemingly many) workflow channels.

That said, enabling the updates-testing repo does indeed offer the chomium version in question… which makes me think…

I completely missed a Final Freeze happening some time between 2024-04-05 and today, didn’t I?
That would explain at least the chromium package and possibly my issue with the everything experiment install. Only the sdubby thing would still need to be blamed on an out-of-date mirror.

Yes. During a freeze only packages that fix blockers or have freeze
exceptions go to stable/base repository. All other updates stay in
updates-testing for now, and right before release the ‘updates’ repo is
opened up.

sdubby was indeed pushed stable 4 days ago for 2271674 – kernel-install detecting btrfs/ext4/etc partitions mounted at /boot as XBOOTLDR

Yeah, I was aware of that information from studying the Fedora Repositories Docs.
The issue was my brain had registered the “end date” (2024-04-16) as the Final Freeze date…

Was my assumption correct then, that the freeze happened after 2024-04-05, which toggled enabled=0 in my [updates-testing] and caused my confusion with the chromium package?


Also to return to my original question, because I would still be interested regardless of the source of the discussed issues, are there any data whatsoever on general availability, success rates, sync delays, etc for the provided mirrors, or any “master” mirrors?

Or is that something simply not available by Fedora’s infrastructure?

Yeah, I was aware of that information from studying the Fedora Repositories Docs.
The issue was my brain had registered the “end date” (2024-04-16) as the Final Freeze date…

Was my assumption correct then, that the freeze happened after 2024-04-05, which toggled enabled=0 in my [updates-testing] and caused my confusion with the chromium package?

The Final freeze started on:

2024-04-02

https://fedorapeople.org/groups/schedule/f-40/f-40-key-tasks.html

But, the fedora-repos update that disabled updates-testing went stable
only a few days ago. It was a automatic blocker bug to push that into
stable.

It was in updates-testing for a while tho, so if you updated with
updates-testing, the update disabled updates-tesing for you. :wink:
(if that makes sense, boy that reads weird. :wink:

Also to return to my original question, because I would still be interested regardless of the source of the discussed issues, are there any data whatsoever on general availability, success rates, sync delays, etc for the provided mirrors, or any “master” mirrors?

Or is that something simply not available by Fedora’s infrastructure?

There is:

https://admin.fedoraproject.org/mirrormanager/propagation

(which looks out of date it seems… will fix that, but we are in freeze
for the release, so might not be until after that).

It does, it makes perfect sense.
Thanks for the clarification :slight_smile:

That’s a useful overview for sure.
It would be extremely helpful if that information could potentially be included in a per-mirror basis on a page like:

https://admin.fedoraproject.org/mirrormanager/mirrors/Fedora/$releasever/$basearch

Do feel free to suggest it at Issues · fedora-infra/mirrormanager2 · GitHub