5 hours passed after the installation, here are the errors from journalctl:
BTRFS error (device sdb3): parent transid verify failed on logical 681705472 mirror 1 wanted 369 found 343
BTRFS error (device sdb3): parent transid verify failed on logical 681705472 mirror 2 wanted 369 found 343
BTRFS: error (device sdb3) in btrfs_replay_log:2104: errno=-5 IO failure (Failed to recover log tree)
BTRFS error (device sdb3 state E): open_ctree failed: -5
Failed to mount sysroot.mount - /sysrooot
and these repeating 2 times
Both copies of metadata have a mismatch on the transaction id, which changes at least once every 30 seconds (or every fsync). So this is > 50 transactions off, which is a pretty big difference.
I’m a little suspicious the drive is not correctly honoring flush or fua.
Ordinarily what happens is the data is written, then metadata written, then flush, then super block written to point to the latest metadata. When they don’t match transactions, usually the drive did something wrong.
The log tree is only present when fsync is used. The metadata is written in abbreviated form to a log kept per subvolume. And then replayed following a crash, to permit the reconstruction of the current metadata. That it’s present means fsync was being used and then interrupted by a crash or power fail. After successful replay, it’s cleared.
Since replay isn’t helping, clearing the log doesn’t directly hurt. But as a consequence it means going back in time before the most recent writes, meaning they will be lost. But if that’s the only thing that went wrong, the file system would be OK.
Note that this ony applies to messages involving failure to recover or replay the log tree.
# btrfs rescue zero-log /dev/sdX3
You’ll need to replace X with the correct node, which can vary between reboots - especially when booting from a USB stick. This command won’t work on a mounted file system.
I advise also running a btrfs check --readonly on this file system and reporting any errors before deciding what to do next.
A safe alternative is to mount using mount -o ro,rescue=nologreplay,usebackuproot,ibadroots /dev/sdX3 /mnt
This will prevent the replay of the bad log tree, see if a backup root is available that will work, and ignore any other b-trees that are damaged. It will retain checksum verification of data and metadata, therefore if the mount works, you can reliably copy out important data to another drive for safekeeping (freshen backups) before attempting any repairs.
Transid/generation in the 300’s is a pretty young file system, maybe only days old. So perhaps you don’t have much important user data on it and you can just reformat and install instead of all this hassle. But I would keep current backups in case this happens again. Btrfs is intentionally doing these kinds of checks to make sure your data is safe. In the case of either software or firmware bugs, it will complain sooner than other file systems.
2 Likes
