I’m building Docker images using the Fedora base image occasionally and am facing massiv issues with it being slow. Also to be honest creating the cache is slow in general with all my devices. It also seems to request all resources sequentially instead of in parallel.
There is more than enough bandwidth available, but the latency is varying (1000/50 koaxial cable connection). Location wise I live close to one of the largest internet exchanges in the world.
Building without already build cache
Executed in 161,70 secs fish external
usr time 86,13 millis 652,00 micros 85,48 millis
sys time 72,28 millis 163,00 micros 72,11 millis
Building with an already build cache (I introduced a layer for the cache which is already cached):
Executed in 39,12 secs fish external
usr time 132,37 millis 0,00 micros 132,37 millis
sys time 83,41 millis 989,00 micros 82,42 millis
Is there anything which can be done? Is parallelization already in work?
Although, I think Debian could perform better as well, here and impression:
time docker run --rm -it debian bash -c "time apt-get update && apt-get install -y bash"
Get:1 http://deb.debian.org/debian buster InRelease [121 kB]
Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [291 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [10.9 kB]
Fetched 8447 kB in 3s (2484 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
bash is already the newest version (5.0-4).
bash set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
Executed in 4,71 secs fish external
usr time 14,76 millis 605,00 micros 14,15 millis
sys time 21,16 millis 411,00 micros 20,75 millis
time docker run --rm -it fedora bash -c "time dnf install -y bash"
Fedora 33 openh264 (From Cisco) - x86_64 156 B/s | 2.5 kB 00:16
Fedora Modular 33 - x86_64 206 kB/s | 3.3 MB 00:16
Fedora Modular 33 - x86_64 - Updates 201 kB/s | 3.1 MB 00:15
Fedora 33 - x86_64 - Updates 2.4 MB/s | 27 MB 00:11
Fedora 33 - x86_64 3.3 MB/s | 72 MB 00:21
Package bash-5.0.17-2.fc33.x86_64 is already installed.
Nothing to do.
Executed in 102,65 secs fish external
usr time 38,34 millis 0,00 millis 38,34 millis
sys time 58,31 millis 1,06 millis 57,24 millis
And running some speedtest:
Speedtest by Ookla
ISP: Vodafone Germany Cable
Latency: 7.35 ms (3.54 ms jitter)
Download: 931.28 Mbps (data used: 1.1 GB)
Upload: 38.51 Mbps (data used: 35.3 MB)
Packet Loss: 0.0%
(yes, Vodafone is a bad operator, but it is also slow when being run within data centers)
Please show us your
There are some tunings you can do.
docker run --rm -it fedora cat /etc/dnf/dnf.conf
But to what I’ve seen tuning it doesn’t help. It only effects the download itself, but not the metadata fetching.
Mine is like this:
You also have to take in consideration that you might playing with to different cache’s. On the workstation it self at least. I don’t know how it looks like with docker … do you always use sudo to use dnf?
I had the problem that when i just wanted to get an info of a package that i used dnf without sudo. This creates a cache for my own user … if you do with sudo it creates a other cache for the root privileged user. I always was creating cache while waiting to execute the command it self.
Yep, so to ensure that you only have one cache and update it only once for each transaction you should always run dnf either as root or with sudo. That avoids the extra cache actions and time requirements for the individual user.
I just did a dnf list as my regular user and it took about 2.5 minutes to build the cache. That was with the fastest mirror option in dnf.conf. As root it took ~1 second.
Thank you, both. But it doesn’t bring this forward. The Fedora Docker image has no cache (and shouldn’t have one, as it would blow up the image). The issue is that the creation is taking too long and one issue with that is that dnf does not seem to download all data in parallel but one by one. And especially the cisco repo has very high latency.