On my desktop running Fedora 37, after a power loss (3 to be exact, the first 2 apparently did nothing as I could boot and use it) I can no longer boot into the system. Here is the dmsg output after trying to mount the partition from a live system:
dmesg
[ 1570.021961] BTRFS info (device dm-3): use zstd compression, level 1
[ 1570.021964] BTRFS info (device dm-3): using free space tree
[ 1570.021965] BTRFS info (device dm-3): has skinny extents
[ 1570.062301] BTRFS info (device dm-3): enabling ssd optimizations
[ 1570.062303] BTRFS info (device dm-3): start tree-log replay
[ 1570.346395] ------------[ cut here ]------------
[ 1570.346396] BTRFS: Transaction aborted (error -17)
[ 1570.346402] WARNING: CPU: 2 PID: 11894 at fs/btrfs/delayed-inode.c:1092 __btrfs_run_delayed_items+0x16d/0x270
[ 1570.346407] Modules linked in: dm_crypt uinput exfat vfat fat rfcomm snd_seq_dummy snd_hrtimer nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink qrtr bnep iwlmvm amdgpu mac80211 libarc4 snd_hda_codec_realtek snd_hda_codec_generic uvcvideo ledtrig_audio snd_hda_codec_hdmi btusb videobuf2_vmalloc iwlwifi snd_usb_audio snd_hda_intel videobuf2_memops btrtl videobuf2_v4l2 intel_rapl_msr snd_intel_dspcfg intel_rapl_common iwlmei snd_intel_sdw_acpi videobuf2_common btbcm snd_usbmidi_lib snd_hda_codec iommu_v2 cfg80211 videodev btintel snd_rawmidi gpu_sched snd_hda_core edac_mce_amd btmtk mc joydev drm_ttm_helper snd_hwdep kvm_amd snd_seq mei ttm bluetooth snd_seq_device kvm drm_display_helper snd_pcm cec snd_timer irqbypass snd eeepc_wmi soundcore asus_wmi rapl sparse_keymap
[ 1570.346436] platform_profile rfkill i2c_piix4 k10temp video pcspkr gpio_amdpt wmi_bmof gpio_generic acpi_cpufreq zram isofs squashfs cdc_ether usbnet r8152 nvme uas mii nvme_core usb_storage r8169 crct10dif_pclmul crc32_pclmul crc32c_intel ccp ghash_clmulni_intel sp5100_tco wmi sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi loop ip6_tables ip_tables fuse
[ 1570.346453] Unloaded tainted modules: amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 amd64_edac():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 pcc_cpufreq():1 fjes():1 fjes():1 pcc_cpufreq():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1 fjes():1
[ 1570.346480] CPU: 2 PID: 11894 Comm: mount Tainted: G W 5.19.7-300.fc37.x86_64 #1
[ 1570.346481] Hardware name: ASUS System Product Name/TUF GAMING B550M-PLUS, BIOS 2803 04/27/2022
[ 1570.346482] RIP: 0010:__btrfs_run_delayed_items+0x16d/0x270
[ 1570.346484] Code: 48 0f ba 2a 03 72 25 41 83 ff fb 0f 84 07 01 00 00 41 83 ff e2 0f 84 fd 00 00 00 44 89 fe 48 c7 c7 78 bd 66 83 e8 98 d3 78 00 <0f> 0b 44 89 f9 ba 44 04 00 00 48 c7 c6 70 0a 27 83 4c 89 e7 e8 8d
[ 1570.346485] RSP: 0018:ffffb639970b7778 EFLAGS: 00010296
[ 1570.346486] RAX: 0000000000000026 RBX: ffff96dcf1bef168 RCX: 0000000000000000
[ 1570.346487] RDX: 0000000000000001 RSI: ffffffff836810b8 RDI: 00000000ffffffff
[ 1570.346488] RBP: ffff96dcf1bef110 R08: 0000000000000000 R09: ffffb639970b7630
[ 1570.346488] R10: 0000000000000003 R11: ffffffff83f45d88 R12: ffff96dcbe7dbe38
[ 1570.346489] R13: ffff96dce64225b0 R14: ffff96db45817000 R15: 00000000ffffffef
[ 1570.346489] FS: 00007f03ea49e800(0000) GS:ffff96e63ea80000(0000) knlGS:0000000000000000
[ 1570.346490] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1570.346491] CR2: 00007f8e1345c5a8 CR3: 000000074809c000 CR4: 0000000000750ee0
[ 1570.346492] PKRU: 55555554
[ 1570.346492] Call Trace:
[ 1570.346493] <TASK>
[ 1570.346495] drop_one_dir_item+0x116/0x130
[ 1570.346498] add_inode_ref+0x81a/0x1110
[ 1570.346501] replay_one_buffer+0x11f/0x490
[ 1570.346503] ? find_extent_buffer+0x54/0x80
[ 1570.346505] ? find_extent_buffer+0x54/0x80
[ 1570.346506] ? btrfs_get_64+0x5b/0xf0
[ 1570.346507] walk_down_log_tree+0x18b/0x360
[ 1570.346509] walk_log_tree+0xc6/0x1e0
[ 1570.346511] btrfs_recover_log_trees+0x1cb/0x590
[ 1570.346513] ? replay_one_extent+0x780/0x780
[ 1570.346514] open_ctree+0x12eb/0x1576
[ 1570.346517] btrfs_mount_root.cold+0x13/0xee
[ 1570.346519] legacy_get_tree+0x27/0x50
[ 1570.346521] vfs_get_tree+0x25/0xc0
[ 1570.346523] vfs_kern_mount.part.0+0x73/0xb0
[ 1570.346526] btrfs_mount+0x11a/0x3c0
[ 1570.346527] ? cred_has_capability.isra.0+0x65/0xf0
[ 1570.346530] ? legacy_get_tree+0x27/0x50
[ 1570.346531] ? btrfs_show_options+0x650/0x650
[ 1570.346532] legacy_get_tree+0x27/0x50
[ 1570.346533] vfs_get_tree+0x25/0xc0
[ 1570.346535] path_mount+0x434/0xa70
[ 1570.346537] __x64_sys_mount+0xe5/0x130
[ 1570.346538] do_syscall_64+0x5b/0x80
[ 1570.346540] ? do_syscall_64+0x67/0x80
[ 1570.346541] ? bpf_lsm_task_kill+0x10/0x10
[ 1570.346543] ? security_task_prctl+0x4e/0x90
[ 1570.346545] ? __do_sys_prctl+0x3a/0x6d0
[ 1570.346547] ? syscall_exit_to_user_mode+0x17/0x40
[ 1570.346549] ? syscall_exit_to_user_mode+0x17/0x40
[ 1570.346551] ? do_syscall_64+0x67/0x80
[ 1570.346552] ? do_syscall_64+0x67/0x80
[ 1570.346553] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 1570.346555] RIP: 0033:0x7f03ea681b0e
[ 1570.346570] Code: 48 8b 0d 15 53 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d e2 52 0c 00 f7 d8 64 89 01 48
[ 1570.346571] RSP: 002b:00007fff049cd9f8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
[ 1570.346572] RAX: ffffffffffffffda RBX: 000055909c3e6550 RCX: 00007f03ea681b0e
[ 1570.346573] RDX: 000055909c3e6810 RSI: 000055909c3e68b0 RDI: 000055909c3e77c0
[ 1570.346573] RBP: 0000000000000000 R08: 000055909c3e67b0 R09: 0000000000000060
[ 1570.346574] R10: 0000000000000000 R11: 0000000000000246 R12: 000055909c3e77c0
[ 1570.346574] R13: 000055909c3e6810 R14: 00000000ffffffff R15: 00007f03ea7b5076
[ 1570.346575] </TASK>
[ 1570.346576] ---[ end trace 0000000000000000 ]---
[ 1570.346577] BTRFS: error (device dm-3: state A) in __btrfs_run_delayed_items:1092: errno=-17 Object already exists
[ 1570.346587] BTRFS: error (device dm-3: state EA) in btrfs_replay_log:2500: errno=-17 Object already exists (Failed to recover log tree)
[ 1570.354467] BTRFS error (device dm-3: state EA): open_ctree failed
The partition (nvme0n1p3) is encrypted with LUKS, I ran the luksOpen successfully, mapping it to /dev/mapper/nvme0n1p3-open, then I ran sudo btrfs check --check-data-csum -p /dev/mapper/nvme0n1p3-open, which found no errors:
btrfs check
[liveuser@localhost-live /]$ sudo btrfs check --check-data-csum -p /dev/mapper/nvme0n1p3-open
Opening filesystem to check...
Checking filesystem on /dev/mapper/nvme0n1p3-open
UUID: 43cb14de-53ad-40a1-9c73-c0d74da6cd57
[1/7] checking root items (0:00:02 elapsed, 2059532 items checked)
[2/7] checking extents (0:00:14 elapsed, 207779 items checked)
[3/7] checking free space tree (0:00:00 elapsed, 241 items checked)
[4/7] checking fs roots (0:00:05 elapsed, 177421 items checked)
[5/7] checking csums against data (0:03:09 elapsed, 666364 items checked)
[6/7] checking root refs (0:00:00 elapsed, 6 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 237282832384 bytes used, no error found
total csum bytes: 228035468
total tree bytes: 3404054528
total fs tree bytes: 2916843520
total extent tree bytes: 201539584
btree space waste bytes: 479331422
file data blocks allocated: 388668747776
referenced 291103481856
This is the output from lsblk -f:
lsblk
[liveuser@localhost-live ~]$ lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squashfs 4.0
loop1 ext4 1.0 Anaconda 10d7fd4a-b807-4211-bde9-defca27328cd
├─live-rw ext4 1.0 Anaconda 10d7fd4a-b807-4211-bde9-defca27328cd 1.6G 79% /
└─live-base ext4 1.0 Anaconda 10d7fd4a-b807-4211-bde9-defca27328cd
loop2
└─live-rw ext4 1.0 Anaconda 10d7fd4a-b807-4211-bde9-defca27328cd 1.6G 79% /
sda
└─sda1 btrfs HDD 626e24c2-a30b-40e4-9f84-d99ea8aef484
sdb
├─sdb1
└─sdb2 ntfs 423E08A53E089457
sdc
sdd
├─sdd1 exfat 1.0 Ventoy 0D7E-7AD2
└─sdd2 vfat FAT16 VTOYEFI 3A2F-294B 5M 84% /run/media/liveuser/VTOYEFI
zram0 [SWAP]
nvme0n1
├─nvme0n1p1 vfat FAT32 D10C-D9C8
├─nvme0n1p2 ext4 1.0 131fd67f-cc8b-4d07-94ac-667ce1020634
└─nvme0n1p3 crypto_LUKS 2 34c9f75a-076c-4521-a2cc-886e9138047b
└─nvme0n1p3-open btrfs fedora_localhost-live 43cb14de-53ad-40a1-9c73-c0d74da6cd57
And this is the output when trying to mount:
mount
[liveuser@localhost-live mnt]$ sudo mount -t btrfs -o subvol=root,compress=zstd:1 UUID=43cb14de-53ad-40a1-9c73-c0d74da6cd57 /mnt
mount: /mnt: mount(2) system call failed: File exists.
dmesg(1) may have more information after failed mount system call.
I tried to follow this question, but in my case I can mount the other 2 partitions (nvme0n1p1 and nvme0n1p2) and smartctl output seems fine.
smartctl
[liveuser@localhost-live ~]$ smartctl -a /dev/nvme0n1
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.19.7-300.fc37.x86_64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
Smartctl open device: /dev/nvme0n1 failed: Permission denied
[liveuser@localhost-live ~]$ sudo smartctl -a /dev/nvme0n1
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.19.7-300.fc37.x86_64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: ADATA FALCON
Serial Number: 2K5029A1562L
Firmware Version: V9002s69
PCI Vendor/Subsystem ID: 0x10ec
IEEE OUI Identifier: 0x00e04c
Controller ID: 1
NVMe Version: 1.3
Number of Namespaces: 1
Namespace 1 Size/Capacity: 512,110,190,592 [512 GB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Mon Mar 13 17:04:21 2023 EDT
Firmware Updates (0x02): 1 Slot
Optional Admin Commands (0x0007): Security Format Frmw_DL
Optional NVM Commands (0x0014): DS_Mngmt Sav/Sel_Feat
Log Page Attributes (0x03): S/H_per_NS Cmd_Eff_Lg
Maximum Data Transfer Size: 32 Pages
Warning Comp. Temp. Threshold: 115 Celsius
Critical Comp. Temp. Threshold: 120 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 8.00W - - 0 0 0 0 0 50000
1 + 4.00W - - 1 1 1 1 0 50000
2 + 3.00W - - 2 2 2 2 0 50000
3 - 0.0500W - - 3 3 3 3 4000 50000
4 - 0.0080W - - 4 4 4 4 8000 100000
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 37 Celsius
Available Spare: 100%
Available Spare Threshold: 32%
Percentage Used: 4%
Data Units Read: 39,691,472 [20.3 TB]
Data Units Written: 48,405,788 [24.7 TB]
Host Read Commands: 472,016,750
Host Write Commands: 531,403,792
Controller Busy Time: 0
Power Cycles: 1,028
Power On Hours: 3,850
Unsafe Shutdowns: 135
Media and Data Integrity Errors: 0
Error Information Log Entries: 1,459
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Error Information (NVMe Log 0x01, 8 of 8 entries)
No Errors Logged
The btrfs troubleshooting docs doesn’t say what to do about this type of error.
I’m lost now, what else can I do? Do I need to run the infamous btrfs repair command?