Cannot pull fedora-toolbox:32 on 32.20200701.0

Project Discussion
1

Cannot initialize a toolbox on a fresh install of Silverblue 32.20200701.0. When I try to pull the base image, I get the error:

Error: error pulling image "registry.fedoraproject.org/f32/fedora-toolbox:32":
unable to pull registry.fedoraproject.org/f32/fedora-toolbox:32:
unable to pull image: Error initializing source 
docker://registry.fedoraproject.org/f32/fedora-toolbox:32: Error reading manifest 32 in
registry.fedoraproject.org/f32/fedora-toolbox: manifest unknown: manifest unknown

Podman version: 1.9.3

edit: format error message for clarity.

2

Hmm, does it consistently fail? This might just be flakiness from the Fedora registry. Our CI has been hitting that as well recently.

3

Sorry, I didn’t have all the domain names resolved last night for some reason.
For domain only : fedoraproject.org
Everything’s working today.

4

I can confirm - was able to pull the toolbox image in the evening EDT.

5

I went on a spree of reinstalling Silverblue this past weekend and can confirm I hit this issue almost every time.

What was interesting in my case is that it always failed while I was hardwired to the network but when I used wifi it worked. It might have just been coincidence but I almost came here and posted about it until I got distracted.

6

So this is now failing again, but for a different reason:

$ podman pull registry.fedoraproject.org/f32/fedora-toolbox:32
Trying to pull registry.fedoraproject.org/f32/fedora-toolbox:32...
Getting image source signatures
Copying blob 03c837e31708 done  
Copying blob 037a20ccce44 [-------------------------------------] 0.0b / 67.6MiB
  Digest did not match, expected sha256:037a20ccce44f4a50af27f659c9572bba253abf1494f7e3f843d13c5c3910d63, got sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Error: error pulling image "registry.fedoraproject.org/f32/fedora-toolbox:32": unable to pull registry.fedoraproject.org/f32/fedora-toolbox:32: unable to pull image: Error reading blob sha256:037a20ccce44f4a50af27f659c9572bba253abf1494f7e3f843d13c5c3910d63: Digest did not match, expected sha256:037a20ccce44f4a50af27f659c9572bba253abf1494f7e3f843d13c5c3910d63, got sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Did someone (or thing) push a bad image?

7

This has been happening for a while. It seems to be a server’s issue.
Sometimes fails, sometimes doesn’t :frowning:
I just pulled that image and it did fine. No error with the Digest or anything. But I’ve been testing toolbox images for a week and I hit this kind of errors a lot.

8

Fedora has been moving its infrastructure and has had some issues. Infrastructure/2020-post-datacenter-move-known-issues - Fedora Project Wiki

The problems that have been affecting Fedora-built flatpaks and toolbox containers off and on (although pretty bad over the past several days) should have been resolved as of sometime yesterday, IIRC.

Current status @ https://status.fedoraproject.org/ — as of writing, everything should be fine again, except for ABRT and Koji.

Hopefully the new datacenter prevents this from happening again.

2 Likes
9

Great news! I hope they used Cisco switches in their data center :wink:

10

We continued to see some issues with pulls. @kevin just invalidated some caches and the problem seems to have been resolved.