Can dnf try EVERY mirror before failing?

[b3y]

Is there a way to tell dnf to try every mirror, and perhaps every IP address of every mirror before it gives up?

I’ve noticed several times that mirrors.fedoraproject.org times out. I tracked it down to them using multiple A records and AAAA records in dns and sometimes one of their servers is down. I’d assume that dnf would retry multiple times before failing but I don’t think it does. Or I’m just really unlucky…

either way, i’d like dnf to try harder before erroring out. How do i configure that?

[computersavvy (Jeff V)]

I am not sure exactly how it would be set up, but if you use man dnf.conf it gives you a lot of info on the available options that can be placed in /etc/dnf/dnf.conf and what they do. I would suspect there is where you may need to configure it if what you ask is possible.

Just as an aside. I have noted that sometimes if my system has IPv6 enabled I see timeouts on dns lookups but if I have only IPv4 enabled I do not see the timeouts. You may want to consider that and try disabling IPv6 before you jump down that rabbit hole with the mirrors. IPv6 can be disabled with the settings → network control panel and if it makes no difference it can be re-enabled the same way.

BTW, this seems to be a continuation of the same topic here.
https://discussion.fedoraproject.org/t/ipv6-problems-with-fedoraproject-mirrors/72850
It is strongly encouraged that you not post duplicate threads on the same topic.

[b3y]

@computersavvy

BTW, this seems to be a continuation of

No.

It is not.

They are two different issues. One is that a fedora project mirror seems to be down right now.

The other is that dnf is handling that very poorly.

Both need to be solved, and the same solution will not fix both problems, hence seperate threads.

@computersavvy

I am not sure exactly how it would be set up

you don’t have to reply then. I’ll reply with the instructions myself when I figure it out, to create searchable content on this site that will help others and probably myself too when I forget it and need to figure it out again.

@computersavvy

if my system has IPv6 enabled I see timeouts on dns lookups

okay. That’s a root cause that needs fixing.

Its 2022. Disabling ipv6 is not how you solve a problem. neither is disabling selinux. or the firewall. nor chmod 777 -R /. Put down the shotgun.

My problem is not rooted in dns timeouts. As stated:

@b3y

sometimes one of their servers is down

That means two of the mirror.fedoraproject.org servers are actually not responding to tcp connections on port 80 right now. That is not a dns problem. I’ll list them on the other thread because that’s not what this one is about.

This thread is about configuring DNF to be more resilient to a mirror not responding.

[oprizal (Syaifur Rizal)]

I noticed sometime when I upgrade the system it will changes the mirror automatically. I can see it move from my home country Indonesia to Japan or China server and it’s with default settings.

Update:

When I open /etc/yum.repos.d/fedora-updates.repo file on [updates] part there a line of:

metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch

I change the link to check the available mirror to my machine and open it from browser:

# Open on browser
https://mirrors.fedoraproject.org/metalink?repo=updates-released-f35&arch=x86_64

then I get the metalink file that contain list of mirror available for my system:

...
<resources maxconnections="1">
    <url protocol="rsync" type="rsync" location="ID" preference="100">rsync://fedora.mirror.angkasa.id/fedora-enchilada/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="ID" preference="100">http://fedora.mirror.angkasa.id/pub/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="JP" preference="99">http://ftp.riken.jp/Linux/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="JP" preference="99">rsync://ftp.riken.jp/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="https" type="https" location="JP" preference="99">https://ftp.riken.jp/Linux/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="JP" preference="98">http://ftp.jaist.ac.jp/pub/Linux/Fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="JP" preference="98">rsync://ftp.jaist.ac.jp/pub/Linux/Fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="TH" preference="97">http://mirror2.totbb.net/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="TH" preference="97">rsync://mirror2.totbb.net/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="https" type="https" location="CN" preference="96">https://mirrors.tuna.tsinghua.edu.cn/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="CN" preference="96">http://mirrors.tuna.tsinghua.edu.cn/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="CN" preference="96">rsync://mirrors.tuna.tsinghua.edu.cn/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="https" type="https" location="JP" preference="95">https://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="JP" preference="95">http://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="JP" preference="95">rsync://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="JP" preference="94">http://ftp.iij.ad.jp/pub/linux/Fedora/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="JP" preference="94">rsync://ftp.iij.ad.jp/pub/linux/Fedora/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="https" type="https" location="SG" preference="93">https://download.nus.edu.sg/mirror/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="SG" preference="93">http://download.nus.edu.sg/mirror/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="SG" preference="93">rsync://download.nus.edu.sg/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="CN" preference="92">rsync://mirrors.bfsu.edu.cn/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="http" type="http" location="CN" preference="92">http://mirrors.bfsu.edu.cn/fedora/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="rsync" type="rsync" location="CN" preference="91">rsync://mirror.sjtu.edu.cn/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
    <url protocol="https" type="https" location="CN" preference="91">https://mirror.sjtu.edu.cn/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml</url>
   </resources>
...

May be you want to try it too then check for each servers (they are all mostly volunteering) why they not responding.

Update:

I tried to ping SERVER_ADDRESS -n4 for IP4 then ping SERVER_ADDRESS -n6 for IP6, and found it only works with IP4.

I check further from free web services to check IPv6:

I guess my country living in the past since it’s not able to use IPv6.

You could also check with online available service to ping the server if it support IPv6 or not.

For example bellow are ping to Google:

And bellow ping to nearest server to me and look like they have not support for IPv6:

But it’s ok. The angkasa.id owner look like a good gentleman since he hosted lot of Linux Distros mirror update.

Another update:

Since from other post look like you have server that able to connect to IPv6, may be if you have extra spaces and bandwidths, you could also volunteering to host Fedora Linux update with IPv6. That would be awesome.

[ilikelinux (Ilikelinux (l.s.))]

@oprizal

Another update:

Since from other post look like you have server that able to connect to IPv6, may be if you have extra spaces and bandwidths, you could also volunteering to host Fedora Linux update with IPv6. That would be awesome.

[b3y]

@oprizal

metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch

mirrors.fedoraproject.org

That is the problem. That domain resolves to multiple v4 and v6 ips. Two of them are down. your web browser is smart enough to keep trying until it finds one that is up. DNF is not.

[ilikelinux (Ilikelinux (l.s.))]

@b3y

That is the problem. That domain resolves to multiple v4 and v6 ips. Two of them are down. your web browser is smart enough to keep trying until it finds one that is up. DNF is not.

@IPv6 problems with FedoraProject mirrors

https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64

How about the epel-7 repo, could not be that the culprit for the time-out?
The epel repo is not on every mirror …

[computersavvy (Jeff V)]

@b3y
You also need to remember that for each mirror it tries, dnf must wait for the timeout before it will switch to the next mirror. If it starts through a list where more than one fail to respond in a row that adds dramatically to the time-out delays.

Suggestion:
If you are convinced that this is actually a dnf issue then please take the necessary action and report it as a bug so you can get a definitive answer from the developers.

bugzilla.redhat.com and log in with the same credentials you use here.

Continuing to thrash this out in a discussion forum seems to not move toward an answer that you will be satisfied with.

[b3y]

@computersavvy

for each mirror it tries

for each mirror in the list it gets from the metalink, right?

[computersavvy (Jeff V)]

yes.
As I understand it dnf gets the list from the metalink, then picks one to start and from that point it walks the list if the first one fails. For each one it tries there is a time out delay before it decides it needs to switch to another.

The delay seems to be predicated on slow data transfer as well as non-reply as the case occurs.

[b3y]

@computersavvy

gets the list from the metalink

And what’s it do if the metalink is down? That is what was happening intermittantly.

The metalink is awfully important right? so it’s hosted on multiple servers across the world. The metalink is hosted on mirrors.fedoraproject.org and that domain has a dozen or more IPs that host it.

The hostname mirrors.fedoraproject.org in DNS has multiple A and AAAA records. And when dnf fetches the metalink via libcurl or whatever, the local resolver picks one at random for it to follow. If that one is the one that’s down, dnf throws an error.

[ilikelinux (Ilikelinux (l.s.))]

@b3y

the local resolver picks one at random for it to follow.

You can sett up in /etc/dnf/dnf.conf to speed up the process.

[main]
fastestmirror=True
max_parallel_downloads=10

This settings helped me to speed up my process of upgrade.