Probably the dnf update && dnf clean all && flatpak update broke some packages
Workarounds
Although the DNF package manager was broken also, I managed to restore it just by manually downloading two packages from pkgs.org
and installing them with rpm -ivh --nodeps --force /path/packagename.rpm
Also I fully REMOVED the NVIDIA PROPRIETARY DRIVER in an attempt to troubleshoot the system.
Afer that, at some point when I re-rerun dnf update, the open-source diver nouveau got automatically installed. xorg-x11-drv-nouveau-1:1.0.17-6.fc39.x86_64
That was a method I found myself, I could not find another workaround…
Is it bad? Trying to rescue/restore the system, so it seems legit and it worked!
For sure the system is still in trouble. Still I cannot boot into graphical mode and the services fail to start. Just fixed, even with this “weird”, method, the DFN package manager…
Any thoughts about the services? Can they be forced to START?
Which one is the most basic one to be started first?
Probably the systemd-homed.service?
systemd[1]: dbus.socket: Failed with result 'service-start-limit-hit'.
is a symptom, but only tells us that repeated attempts to start some service have failed. Look for messages about specific services failing to start, maybe by comparing systemctl status for successful and failed boots.
This points to either vendor firmware that doesn’t support linux or a hardware problem. Linux has many workarounds problems with vendor firmware, either via kernel command-line parameters or a user-generated driver package, but to uses these you need to pin down the problem.
Auto repair shops are often guilty of the “parts cannon” approach where they just replace every part mentioned in the diagnostics without attempting to understand the causal path. In linux, “package cannon” approaches rarely work. You end up with a unique configuration that can’t be duplicated by others, making it nearly impossible for others to understand the problem.
Use journalctl to identify differences between successful and failed boots.
Thanks for the guidance @gnwiii
Will try to debug more with systemctl status.
Also with journalctl where I am looking for errors inside… already…
BTW I am not a SysAdmin. Maybe advanced user, but I use this laptop for work, thus I am in trouble now… Unfortunately, even if I read the logs and/or the journalctl, most probably it will be really hard to find a solution.
Probably it would be even better to copy all files to an external drive and then re-install Linux…
p.s. as for the manual downloads from pkgs dot org, I repeat that I had to try something, somehow to fix my work system… fully agree it is not the recommended or correct method. What would I do? Nothing?
I have to ask if you have tried to downgrade the system back to before you did a dnf update
You can do several things, You can start by doing
dnf history to find which update caused you issues.
After you identified the update you can do dnf history undo to revert the update.
optional, You can do dnf history info <transaction_id>to see more details of the packages installed in that update.
If you know which package broke the system, you can choose to dnf downgrade package_name[-version] to a known build of the package that is working properly.
So, if you still have boot issues and need to get back to multi-user.target (teminal) like you did adding the 3 option, you should be able to check the system with the above commands.
future edit
You can repair dnf if you can boot in with a Live environment.
Boot in to your Live USb,
Identify your root partition (usually /dev/sdaX where X is a letter).
Open a terminal (Terminal icon).
Run lsblk to list block devices.
Look for the partition mounted as / .
Run the commands, mount /dev/sdaX /mnt (replace sdaX with your identified root partition).
You have tried multiple linux distroa already. Reinstall has the advantage that you will have the same packages as users trying to help, but you should try to capture some details of the failures (comparign boots that fial to those that succeed) so you know what to look for if a fresh install still has a problem.
Because i typically like to live dangerously, I would go further back to a build using 730 as @grumpey posted and revert to it. As I posted in #3 , you can peruse the update and see if anything related to systemd, or grub was done. Maybe finding an update that was done far enough back to build off of.
future editor :
Do some inspection there, and see if the recent update relates to grub or systemd.
I think you can do a vimdiff <(dnf history info <transaction_id>) <(dnf history info <transaction_id>)
or diff <(dnf history info <transaction_id> | grep -A 15 '^Packages Altered:') <(dnf history info <transaction_id> | grep -A 15 '^Packages Altered:')
if you feel like you want to see the comparisons.
So a fundamental question: doesn’t DNF (and YUM before it, or APT in debian) check for disk space BEFORE committing changes? I would definitely expect them to be coded with that checks implemented.
Package managers like DNF (and YUM in the past) are designed to check for available disk space before committing changes to the system. They have safety mechanisms in place to prevent operations that could exhaust available disk space, as running out of disk space can lead to system instability and corruption.
However, there can be certain scenarios where these checks may not work as expected, or the disk space situation changes after the initial check
Now I just booted by using the Fedora live USB.
Image name, automatically downloaded: Fedora-Workstation-Live-x86_64-39-1.5.iso
Ofcourse I can access my disk, enter the passphrase, view the files and rsync them to another medium/disk e.t.c.
Thus there is one POSSIBLY IMPORTANT trick:
Is there any useful tool provided with the live-USB, or separately installable that can help recover/fix/repair the initial(installed) system ?
Would you recommend such an approach?
p.s. did not perform any rollback after using the dnf history since it is not clear which package broke. We cannot be sure it is the dracut right?
