DNS Resolution Takes 5+ seconds Fedora 34

Hello!

I’m coming from OpenSUSE (very briefly) then Arch before that for several years. Neither of which had this problem. To distill it down it takes 5 seconds for applications to resolve DNS addresses. It causes timeouts and problems and generally stunts my browsing. CDNs are not fun with DNS problems. It’s hard to build the Internet when every page you reload is slow.

Here’s a fun kicker: If I turn on ProtonVPN the DNS resolves very fast (less than 100ms in some cases).

As much as I want to be able to run a VPN 24/7, I still want the option to connect without one.

System:

 OS: Fedora 
 Kernel: x86_64 Linux 5.13.8-200.fc34.x86_64
 Uptime: 1h 25m
 Packages: 1925
 Shell: zsh 5.8
 Resolution: 3520x1080
 DE: GNOME 40.0
 WM: Mutter
 WM Theme: 
 GTK Theme: Adwaita-dark [GTK2/3]
 Icon Theme: Papirus-Dark
 Font: Cantarell 11
 Disk: 235G / 960G (25%)
 CPU: Intel Core i7-7700HQ @ 8x 3.8GHz [39.0°C]
 GPU: Mesa Intel(R) HD Graphics 630 (KBL GT2)
 RAM: 4183MiB / 15716MiB

Output of /etc/resolv.conf:

nameserver 127.0.0.53
options edns0 trust-ad
search du.shawcable.net 

(Search du.shawcable.net likely comes automatically from my ISP’s provided wireless gateway/router)

When VPN is off:

resolvectl status outputs the following. Note the DNS servers. They come from my router configuration. I cannot change the domain the router is providing. I also cannot remove the 64.xx* dns options.

Global
       Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (wlp2s0)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 64.59.161.69
       DNS Servers: 1.1.1.1 1.0.0.1 64.59.160.15 64.59.161.69
        DNS Domain: du.shawcable.net

Link 4 (virbr0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

If I try a resolvectl query fedoraproject.org it takes 5+ seconds:

fedoraproject.org: 140.211.169.196             -- link: wlp2s0
                   // Lots of IPv4/IPv6 address. Dig the dead beef, guys - via wlp2s0

-- Information acquired via protocol DNS in 5.2557s.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network

If I dig I resolve through my local stub.

; <<>> DiG 9.16.19-RH <<>> fedoraproject.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58509
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;fedoraproject.org.		IN	A

;; ANSWER SECTION:
fedoraproject.org.	//lots of IPv4 only addresses...

;; Query time: 451 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Wed Aug 11 11:49:16 PDT 2021
;; MSG SIZE  rcvd: 206

If I turn my ProtonVPN ON…

Things work perfectly well

resolvectl status

Global
       Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (wlp2s0)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 4 (virbr0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 11 (ipv6leakintrf0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 12 (proton0)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 10.24.0.1
       DNS Servers: 10.24.0.1
        DNS Domain: ~.

resolvectl query fedoraproject.org - 105.1ms

fedoraproject.org: 2620:52:3:1:dead:beef:cafe:fed7 -- link: proton0
                   // Lots more IPv4/6 addresses via link: proton0

-- Information acquired via protocol DNS in 105.1ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network

dig does the same stub resolution

; <<>> DiG 9.16.19-RH <<>> fedoraproject.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46995
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;fedoraproject.org.		IN	A

;; ANSWER SECTION:
  // Only IPv4 responses

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Wed Aug 11 11:57:08 PDT 2021
;; MSG SIZE  rcvd: 206

Apologies for the wall of text.

I wanted to provide as much information as I could. I want to make good use of Fedora without disabling or bypassing its new features. I feel like because the DNS will resolve fast through dig/Proton, I must be some sort of configuration step away from getting to work with my ISP/Cloudflare.

In hindsight I wonder if this is an IPv6 resolution issue?

Any help would be appreciated.

2 Likes

You can try to set some public DNS server in your network configuration instead of using the ones provided by your ISP, and see if something changes.

3 Likes

https://discussion.fedoraproject.org/t/dns-resolution-broken/67067/2?u=vgaetera

As I know many things are abstracted in Gnome’s GUI, I’m wondering if this is much different from toggling “Automatic” DNS in the WiFi settings, then explicitly typing in 1.1.1.1, 1.0.0.1? I had tried this to no avail.

Check the output after applying the proposed workarounds:

resolvectl --no-pager status
resolvectl --no-pager query discussion.fedoraproject.org

To answer my own question, yes. This is the CLI way of going through Gnome WiFi settings as it effectively did the same thing. Good to know that nmcli == Gnome WiFi settings.

After this step

sudo nmcli connection show
sudo nmcli connection modify id CON_NAME \
    ipv4.ignore-auto-dns yes \
    ipv6.ignore-auto-dns yes \
    ipv4.dns 1.1.1.1,1.0.0.1
sudo nmcli connection up id CON_NAME

I intermittently resolve from ‘cache network’ and ‘network’. When it’s cached, resolvectl resolves fast. When it’s not, it’s still 5+ seconds. It resolves via network most often.

Then I followed your directions in creating a custom rule for systemd-resolve and started getting results like this:

discussion.fedoraproject.org: 2001:470:1:791::15      -- link: wlp2s0
                       72.52.80.15             -- link: wlp2s0
                       (askfedora.hosted-by-discourse.com)

-- Information acquired via protocol DNS in 841us.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: yes
-- Data from: cache

When it’s not pulled from cache, it’s done in under 1s. Nice!!

Edit: Thanks for your help :clap: :clap: :clap:

1 Like

The default timeout to failover from nss-resolve to nss-dns in Fedora should be 5 seconds.
So, this could be some poorly implemented DNS hijacking by the ISP/router, since replacing local resolvers with public ones doesn’t change the result by itself, but enabling DoT appears to help.
By the way, nmcli can manage all networking for NetworkManager.

3 Likes

This would not be surprising to me at all. My ISP is not as bad as others, but does keep a firm grip on their network. Fedora and yourself may have helped me see just how firm that grip is :slightly_smiling_face:

Yeah coming from Arch I should’ve known that. I most often didn’t use a GUI for configuration and have only just started getting into Gnome. Thanks again!

2 Likes

Interesting issue btw.
I can ping 64.59.161.69 and 64.59.160.15, also port 53/tcp/udp seems to be open, but they do not respond to dns queries for me.

dig +dnssec fedoraproject.org @64.59.160.15

Same result like below:

dig +dnssec fedoraproject.org @64.59.161.69

Result:
; <<>> DiG 9.16.19-RH <<>> +dnssec fedoraproject.org @64.59.161.69
;; global options: +cmd
;; connection timed out; no servers could be reached

Your other addresses work

1.1.1.1,1.0.0.1
dig +dnssec fedoraproject.org @1.0.0.1

; <<>> DiG 9.16.19-RH <<>> +dnssec fedoraproject.org @1.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52393
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 12, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;fedoraproject.org.		IN	A

;; ANSWER SECTION:
<snip>...
fedoraproject.org.	47	IN	A	38.145.60.20
fedoraproject.org.	47	IN	A	8.43.85.67
fedoraproject.org.	47	IN	A	38.145.60.21
fedoraproject.org.	47	IN	A	140.211.169.206
fedoraproject.org.	47	IN	A	152.19.134.142
fedoraproject.org.	47	IN	A	8.43.85.73
fedoraproject.org.	47	IN	A	152.19.134.198

;; Query time: 29 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)
<snip>...
1 Like

ISP resolvers often filter traffic only answering to their clients to prevent DNS amplification attacks.

Yes, seems like,
Their filtering might be the reason behind the problems of the OP as well?
If he turns on VPN, it also may use different dns servers.

This was mentioned in the OP and you can confirm it with the posted diagnostics.
Using the VPN solves the issue as it changes DNS routing and replaces resolvers.

Ah yes, over-read it somehow

Thanks
<:]

I did some more reading on this via my ISP’s own message boards and it appears that their newer equipment is even more restrictive. DNS is only configurable on the client machine. The reason for this is largely because of their IP-based television delivery service.

The ISP’s cable plant needs to be able to talk back to customer equipment. That equipment needs to interconnect with itself on the LAN. If the settings are messed with on the client home router, it can disrupt/break television service.

My wireless gateway is not for this purpose but it still interacts with the external DNS that is for IPTV. I also know from my time working for them that my ISP doctors their DNS for the “best possible route” according to them, in order to prevent customers from feeling like their service is slow. Ironic!

There are a lot of confused customers who feel like my ISP is breaking their public DNS/VPNs because they don’t understand how to configure them in this scenario. The ISP refuses to support their use-case :rofl:

2 Likes