F42 Change Proposal: Unify /usr/bin and /usr/sbin (System-Wide)

Project Discussion Change Proposals
, ,
1

Unify /usr/bin and /usr/sbin

Wiki
Announced

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

:link: Summary

The /usr/sbin directory becomes a symlink to bin, which means paths like /usr/bin/foo and /usr/sbin/foo point to the same place. /bin and /sbin are already symlinks to /usr/bin and /usr/sbin, so effectively /bin/foo and /sbin/foo also point to the same place. /usr/sbin will be removed from the default $PATH.

:link: Owner

:link: Detailed Description

The split between /bin and /sbin is not useful, and also unused. The original split was to have “important” binaries statically linked in /sbin which could then be used for emergency and rescue operations. Obviously, we don’t do static linking anymore. Later, the split was repurposed to isolate “important” binaries that would only be used by the administrator. While this seems attractive in theory, in practice it’s very hard to categorize programs like this, and normal users routinely invoke programs from /sbin. Most programs that require root privileges for certain operations are also used when operating without privileges. And even when privileges are required, often those are acquired dynamically, e.g. using polkit. Since many years, the default $PATH set for users includes both directories. With the advent of systemd this has become more systematic: systemd sets $PATH with both directories for all users and services. So in general, all users and programs would find both sets of binaries.

One additional use of the /bin—/sbin split is consolehelper. In this approach, the user-facing program (/bin/foo) is a symlink to /bin/consolehelper, which is a suid binary that elevates privileges and calls the “real” foo (/sbin/foo or /usr/libexec/foo). Most uses of consolehelper have been moved to polkit (Features/UsermodeMigration - Fedora Project Wiki), but some users remain (https://bugzilla.redhat.com/show_bug.cgi?id=502765). Use of /sbin for the privileged program is incompatible with the proposed merge; those packages will need to be adjusted to move the binary that requires privileges under /usr/lib or /usr/libexec (see Scope below).

Since generally all user sessions and services have both directories in $PATH, this split actually isn’t used for anything. Its main effect is confusion when people need to use the absolute path and guess the directory wrong. Other distributions put some binaries in the other directory, so the absolute path is often not portable. Also, it is very easy for a user to end up with /sbin before /bin in $PATH, and for an administrator to end up with /bin before /sbin in $PATH, causing confusion. If this feature is dropped, the system became a little bit simpler, which is useful especially for new users, who are not aware of the history of the split.

Many years ago we merged /bin and /usr/bin (Features/UsrMove - Fedora Project Wiki). In some ways that split was similar: it had historical justification that went away more than a decade prior, it was impossible to cleanly categorize programs into the the categories so effectively both parts were needed for boot, and even though it was making the system more complicated for little gain, the split was being carried forward because it was easier to do so than to remove it (TheCaseForTheUsrMerge). This split is much less visible, but it’s also making the system more complicated for no gain, and removing it is the natural follow-up.

:link: Feedback

:link: Benefit to Fedora

  • Packagers don’t have to think whether to install programs in %_bindir or %_sbindir.
  • Users don’t have to think whether programs are in %_bindir or %_sbindir.
  • Fedora becomes more compatible with other distributions (for example, we have /sbin/ip while Debian has /bin/ip, and we have /bin/chmem and /bin/isosize, but Debian has /sbin/chmem and /sbin/isosize, and we also have /sbin/{addpart,delpart,lnstat,nstat,partx,ping,rdma,resizepart,ss,udevadm,update-alternatives}, while Debian has those in under /bin, etc.)
  • Fedora becomes more compatible with Arch, which did the merge a few years ago.
  • execvp and related functions iterate over fewer directories. This probably doesn’t matter for speed, but is a nice simplification when looking at logs or strace output.

:link: Scope

  • Proposal owners:

    • Adjust %_sbindir in /usr/lib/rpm/macros (part of rpm package). Packages will be updated automatically during the mass rebuild.
    • Add a %filetrigger to filesystem package to create symlinks to ../bin/foo for every foo that is uninstalled from /usr/sbin.
    • Add a %posttrans trigger to filesystem package to check that /usr/sbin only contains symlinks and do ln -fs bin /usr/sbin. (Those scriptlets make it easier to have a smooth transition. At all times, the old paths will still work. After the transition is complete we can drop the scriptlets and provide the /usr/sbin symlink in the filesystem package.)
    • Adjust systemd package to build with -Dsplit-bin=no.

  • Other developers: programs which user consolehelper and install the same name under both directories will need to be adjusted to use a different directory. Some of those packages may be retired instead. See list below.

  • Packages using usermode with binaries in both directories:

    • anaconda-live
    • beesu
    • chkrootkit
    • hddtemp
    • mate-system-log
    • setuptool
    • subscription-manager
    • system-switch-java
    • xawtv

  • Release engineering: #Releng issue number

  • Policies and guidelines: Fedora Packaging Guidelines :: Fedora Docs will need to be adjusted (and most likely retitled ;))

  • Trademark approval: N/A (not needed for this Change)

  • Alignment with Community Initiatives: nope

:link: Upgrade/compatibility impact

The change should be mostly invisible for users. While the transition is ongoing, both sets of paths should work and users should have both directories in $PATH. Once the transition is finished, both sets of paths should work, but users will only have /usr/bin in $PATH.

:link: How To Test

:link: User Experience

:link: Dependencies

:link: Contingency Plan

  • If the move of a binary of any specific package causes problems, we can create a compat symlink like /usr/sbin/foo → ../bin/foo using a %postin scriptlet in that package.
  • If the change is causing problems in general and needs to be reverted, we’d need to undo the changes to macro definitions in rpm and rebuild some or all packages.
  • Contingency deadline: in principle can be done at any time, but would require a rebuild of some or all affected packages.
  • Blocks release? no. It is OK if the change is done partially.

:link: Documentation

TBD.

:link: Release Notes

Last edited by @zbyszek 2025-03-23T16:52:39Z

Last edited by @zbyszek 2025-03-23T16:52:39Z

6 Likes
2

100% in favor!

3

This should probably be set in redhat-rpm-config instead of modifying the core rpm package.

4

If a package installs stuff to $DESTDIR/usr/sbin (e.g. via some hardcoded path in Make/CMake/etc.) and then uses %{_sbindir} in %files, this will explode. I suspect there will be several such packages out there.

5

Thanks, that’s a good point. Repoquery says that there are 35 source packages which have files in /usr/sbin/: chkconfig cyrus-sasl dmidecode dosfstools e2fsprogs esmtp exim fping glibc glusterfs iproute iptables kmod libcap libreswan lvm2 msmtp nbdkit nfs-utils ntpsec opensmtpd policycoreutils postfix psmisc rpcbind sendmail shadow-utils ssmtp systemd tcpdump ubdsrv util-linux xfsprogs. This is small enough that we can certainly fix any if there is a problem. I’ll add a note to Scope.

2 Likes
6

I understand the proposal and it seems that given the past changes, it makes sense to go forward with this merge of directories. Otherwise, it would be like a half-way change, an hybrid, something from the past and the new.

What I don’t understand is the wording or how it’s proposed. It should be more straightforward and sincere.

The “Detailed Description” is well structured and goes to the point, and it should have been enough to make or take a decision regarding this proposal.

Things that I don’t share or that I think makes no sense:

  • Section: “Benefit to Fedora”

    • Packagers don’t have to think whether to install programs in %_bindir or %_sbindir.
      Isn’t it this exactly the main role of a packager?
      Don’t get me wrong. I don’t mean packager’s life must be something not easy to carry. What I’m trying to say is that this kind of decision is what makes a packager a packager. This is the person who should know how to handle these situations.
      Thus, I consider this argument not really sustainable.

    • Users don’t have to think whether programs are in %_bindir or %_sbindir.
      This is a contradiction.
      At the “Upgrade/compatibility impact” it is stated that: “The change should be mostly invisible for users.”
      In fact, I believe most Users just use GUI interfaces these days and the ones that do use CLI interfaces know what they’re doing or how to make things work their way too.
      Thus, I consider this as a “padding-argument.” One no providing really anything to the table.

    • Fedora becomes more compatible with other distributions
      I thought LSB was responsible for handling these issues of compatibility.

    • Fedora becomes more compatible with Arch, which did the merge a few years ago.
      This is just like the previous point by now mentioning Arch.
      How is the interoperability between Fedora and Arch to justify this argument?

    • execvp and related functions iterate over fewer directories. This probably doesn’t matter for speed, but is a nice simplification when looking at logs or strace output.
      This can be an argument. The question is: How many iterations would save? 1? 10? 100? Show me some stats, examples, etc.

Conclusion:

I think the change is valid because of previous changes made in this same direction.
That is, this would lead to a more concise and coherent filesystem.

What I think we should work upon is in how we argument about these changes:

  • There’s no “Downside” in this article, not even a header for it!
  • The “Benefit to Fedora” section is full of biased arguments in order to just justify the change. From the 5 arguments presented, only 1 might be relevant. 2 are just the same, one contradicts information presented in this proposal and the other is at least biased.

In short:

Go ahead!

7

I was an Arch user at the time of their /usr-merge, so I lived through it.
IIRC they made their process by steps where binaries would be installed in the new path and the old and deprecated path would have a symlink, in the future the old path would be simply dropped in favor of a symlink.

This seems very similar to what is being proposed here.

Although, on Arch they were rolling release, so they could move to the next step whenever they wanted and it was a “you have to read the news to not break your system” type of scenario.
On Fedora it would have to be a prposal that enter in effect in a new release. Though I ask, can this be done in a single release?

8

Packager’s work should be as hard as required, but as simple as possible. It’s not a big chore, but it just makes things a tiny bit easier, so I think it is a valid argument, even if fairly weak.

It is true that for most users this doesn’t matter, but some users need to specify absolute paths. For example, when writing sudo policies (sudo rules require an absolute path). This is more of an issue when trying to write something that is portable between distributions. This argument applies to people who are admins. They are also important users of Fedora.

LSB is mostly dead. We don’t claim LSB compatiblity (Issue #3089: retiring redhat-lsb in Fedora - fesco - Pagure.io) and neither do other distros. And anyway LSB doesn’t actually specify the location of all binaries. And clearly, in practice distros do make differing choices.

Well, people do cross-distro development. I know it’s not that common, but it certainly occurs. Removing pointless differences between distros helps such users.

Usually $PATH has 4–10 entries (4 is the base of /usr/{bin,sbin} and /usr/local/{bin,sbin}, then ccache adds one, qt adds one, environment-modules adds one, and then there is user’s ~/bin and such). We cut out two from that list. So that’s up to 50%, but usually less.

Cool!

9

As an example, I do development in Fedora, CentOS, openSUSE, and Mageia (and in a previous life, I did some Debian/Ubuntu work too). Over the years, I’ve done work across all these distributions to eliminate differences that do nothing but make things annoying.

One example of this is the adoption of /usr/libexec across distributions to rationalize the location of non-path executables in a way that would allow a user to mark /usr/lib{,32,64} as noexec (which is valuable for creating secure appliances, though you can’t yet for now with systemd storing binaries in /usr/lib). I proposed it in openSUSE and did the work to transition the distribution, and encouraged Debian folks to do the same at around the same time.

The result is that Fedora; CentOS; openSUSE; Mageia; Debian; and Ubuntu all use /usr/libexec now for things, and the non-path executable path usage is increasingly consistent. This makes automation work tremendously easier too, because then you can rely on the same configuration in more places.

1 Like
10

You should symlink/merge those as well, otherwise it looks inconsistent.

1 Like
12

Good point. I’ll clarify this in the Change proposal.

13

Symlinks are not deprecated.

The symlinks are there to provide backwards compatibility. It’s easy to move a binary, but we don’t to unnecesarilly break something that explicitly uses the old path. The symlink doesn’t really matter except for that.

I don’t think any of those topics are related :wink:

15

This change proposal has now been submitted to FESCo with ticket #3135 for voting.

To find out more, please visit our Changes Policy documentation.

1 Like
16

Maybe I’m alone (and lonely) in this, but I remove sbin from my path in my profile (but leave it in the configured path for sudo. This removes a bunch of stuff from tab-completion conflicts.

Many years ago, I suggested that we should (instead of putting sbin in end-user paths) do the work to actually sort out

  1. commands that require root, and
  2. binaries which are daemons not really meant to be run from any path directly by a user, root or not

But there wasn’t much enthusiasm for it, and it seemed like kind of lot of churn for not very much tangible gain. So, like then, I may just have to be “old man stares sadly at clouds”. I must admit that the simplification has its own benefits as well.

17

This seems reasonable going forward, but I don’t think anything under /usr/local should be touched by an upgrade, if there are any non-RPM-owned files there.

18

I’ll try to explain why this is I think any such attempt must fail. Buckle up :wink:

In the old times there was “root” and lusers, the privilege separation was completely binary. But some time in the 2.2 era, Linux implemented capabilities. This means that you can be root, completely unprivileged, or in one of the 1099511627774 intermediate states.

In addition, we have MAC, in particular SELinux, which means that you can be “privileged”, but actually unable to do something.

In the other direction, we have extended file attributes, which allow the admin to grant write access to various things to unprivileged or less-privileged users. Systemd uses this to give access to the system journal to users in administrative groups and to certain device nodes to users who are logged in locally.

Finally, processes can use dbus and other IPC to execute operations via a privileged daemon, which in turn can use polkit to allow or disallow operations based on arbitrarily complex policies.

Altogether, answering the question whether “this process can do that thing” is … hard. The only reliable way is to just try and handle failure appropriately.

The interesting corollary is that all programs that are supposed to “run as root” must always treat every individual operation as something that can fail (because even if the process formally has privileges, MAC or something else may prevent the operation) and must report such failure in sufficient detail. It is not OK to say “we failed” or “not enough privileges, must be root”, because that is not enough to figure out what exactly failed. In fact, the process may be root and still fail.

(In systemd, we had a bunch of programs intended to be ran as root which would flat out refuse to start with a different UID. We removed all those checks.)

Then there are cases where it’s actually fine if the process doesn’t have enough privileges. For example, the user executes a privileged command to see what it will try to do. Another case is where the program implements a “dry run” mode where it only logs what it would do. If we were to be strict about this separation, we would have to move programs from /usr/sbin to /usr/bin when they implement a --dry-run option.

We just cannot say “this program requires root”.

That’s a separate issue. We have a bunch of binaries which are intentionally outside of $PATH. /usr/libexec has hundreds of programs, and there are even more in other locations under /usr.

1 Like
19

As described in the proposal, we would check that the directory-to-be-removed only contains symlinks to the other directory and only then replace it by a symlink to the other directory. It doesn’t matter if those files are owned by rpm or not.

20

Unfortunately, LSB seems to no longer be active. There has been little activity on the mailing list, and last version I see was:
lsb/lsb-50.txt · Last modified: 2016/07/19 01:24 (external edit).

21

Sure, but there are a bunch of older programs that just check for “am I root”. Theoretically those could reasonably be separate. Then, over time, we’d presumably have fewer and fewer of them.

It might be a separate issue now, but it definitely didn’t used to be. /usr/bin/httpd, and so on. In the interest of petty but satisfying organization, maybe these things should be gradually moved to libexec. But I don’t really care very strongly.

22

I think we’re just “arguing” for the sake of Saturday evening discussion :wink: I don’t think there’s any real disagreement…

Sure, but there are a bunch of older programs that just check for “am I root”. Theoretically those could reasonably be separate. Then, over time, we’d presumably have fewer and fewer of them.

OK. Then what will happen with them over time is that they’ll either become more versatile, e.g. growing --dry-run options and non-root operation, or they will be moved out of the $PATH. In the first case, moving them eagerly to /usr/bin is OK, because that’s they’d end up anyway, and in the second case, it doesn’t matter.