Experiences with package layering

I’m currently starting some exploration of what infrastructure, testing, software, and user experience changes we need to make rpm-ostree package layering something that can be used widely. And a problem I’m having is that I don’t have a lot of data about failures. It may be because people aren’t having problems, but it may be also that early Silverblue adopters are good at fixing things themselves!

I’d appreciate it if people could post to this thread when they hit a problem with package-layering - even if they figure out that it is not Fedora’s fault.

1 Like

The only problems I’ve ever had with package layering would be lack of support for /opt and ex livefs leaving broken deployment symlinks around (latter is easily worked around though, and it is an experimental command).

The biggest issue so far for me with respect to rpm-ostree was a result of trying to remove a package downgrade of podman and buildah (to get them to use the version installed with the ostree), then I followed those commands with an rpm-ostree update command without a reboot in between. This resulted in a first time boot after install for the new upgraded rpm-ostree. In an effort to recreate it, I found I couldn’t actually get to the state I needed to since there was no state info to get back there. I know ostree will allow for depths of commits beyond the two or pinned state you can use with rpm-ostree, but it would be nice to see such a mechanism in rpm-ostree. Another beef I have is having to use dnf to find out what applications I can layer, instead of just using rpm-ostree. Though the dnf info thing is small it is the most annoying for some reason.

Could you elaborate on this a bit? Almost any package you can install with dnf you should be able to install with rpm-ostree, with the exceptions being those that want to install into /opt or the like.

We’ve been seeing a lot of users hitting the Some base packages would be replaced error like this:

I think what is most frustrating about this error for users is that there isn’t enough information to determine why the packages are being replaced, which ones are being replaced, and how to workaround the issue.

We’ve been seeing a lot of users hitting the Some base packages would be replaced error

Yes, this is definitely an unacceptable failure mode. We can improve the error messages and handling of this, but I’m also interested in the root causes of these problems - in the case you linked to the user was stuck on an old tree because of some other issue and couldn’t update to the latest tree, in some other cases, we’ve had releng issues causing the repositories and trees to be out of sync.

What I was referring to was the info function of dnf that tells you about the package, even if it exists. That type of search is not currently available with rpm-ostree as far as I know. So I must use dnf to query what I want to install with rpm-ostree, which is an inconvenience and minor, but annoying.

Ah, that’s a good pain point to know about. And intersects with the whole “containerize your workflow” topic.

I’m sure you have your own workaround for this already, but I’d do something like this:

$ alias dnf="podman run --rm -it registry.fedoraproject.org/fedora:29 dnf $@"
$ dnf search wget
Fedora Modular 29 - x86_64                                                                                                                                                         131 kB/s | 1.5 MB     00:11    
Fedora Modular 29 - x86_64 - Updates                                                                                                                                               194 kB/s | 2.0 MB     00:10    
Fedora 29 - x86_64 - Updates                                                                                                                                                       553 kB/s |  24 MB     00:44    
Fedora 29 - x86_64                                                                                                                                                                 589 kB/s |  62 MB     01:48    
Last metadata expiration check: 0:00:06 ago on Wed Mar  6 16:30:42 2019.
=============== Name Exactly Matched: wget ================
wget.x86_64 : A utility for retrieving files using the HTTP or FTP protocols
============== Name & Summary Matched: wget ===============
gwget.x86_64 : Graphical download manager that uses wget
perl-HTTP-Tinyish-Wget.noarch : HTTP::Tinyish wget backend
texlive-zwgetfdate-doc.noarch : Documentation for zwgetfdate
=================== Name Matched: wget ====================
texlive-zwgetfdate.noarch : Get package or file date

The rpm-ostree team has been pretty good about linking back to the root issue:

There should be a number of examples littered through the comments and linked issues there.

That’s definitely a good source of information! A lot of the issues linked to there are specific to the Fedora Atomic Host / Fedora CoreOS release cycle - where the base tree updates less often than Fedora updates are released - I found less issues with Silverblue than I expected. I wondered how much that was because people have just figured out how to work around problems and didn’t bother filing a new issue. Hence this thread :slight_smile:

I didn’t use an alias for dnf, it is already installed by default I thought since I didn’t layer it.
I did get the same results with search command of dnf, and it is basically my work around. But I’m interested in your use of dnf as an alias and running it as a podman container. What is the reason for that?

Would that possibly mean I pulled it in with layering qemu and virt-manager, and Boxes?

It probably got pulled in by something else you have layered. It is installed by default in toolbox, so you can run ‘dnf search’ there.

Yes, I used to use fedora-toolbox for just that at times. I think it may have gotten brought in then with the fedmod and fedora-module flatpakking tools perhaps.

I was just hedging my bets in case folks did not have dnf on the host. I have it on my host ultimately through the dependency chain on libguestfs, but that’s just me. :smile:

One tiny way to improve on this is -v /var/cache/dnf:/var/cache/dnf. That way you’re not constantly redownloading repo metadata each time.

1 Like

Yeah, for Silverblue, one can almost always work around it by doing rpm-ostree cleanup -m && rpm-ostree upgrade first before trying to install new packages.


Just for data collecting purposes (ping @otaylor) the /opt support issue is tracked here


1 Like