hobbes1069
August 28, 2024, 4:32pm
1
I’m about to have to reinstall F40 for a 2nd time due to BTRFS corruption. It’s unclear why I’m having an issue that does not seem to be wide spread.
I don’t remember the original version I installed on my HP ENVY X360 w/ AMD Ryzen 4500U processor and NVME storage but it was several releases ago and I’ve had zero issues until upgrading to F40.
The first even seemed to occur shortly after an update but I left it on until the battery died so while BTRFS should be resilient to that, I kind of chalked it up to that.
After installing fresh F40 things seemed fine until after I did a dnf update. It would sometimes lock up at the login screen or if I logged in from the command line it would lock up shortly there after. So far the system seemed to be protecting itself from FS corruption as btrfs scrub came back clean.
I just recently tried kernel 6.10.6 and shortly after logging in and starting a Youtube video Chrome locked up but I could still get to other apps, at least for a few seconds.
Now my FS won’t mount and since I didn’t have anything I cared about on it I tried the usual btrfs check and the notorious btrfs check --repair since I had nothing to lose, and the system is unrecoverable.
I have had zero issues with the stock F40 or rescue kernels.
Before I reinstall again, how the heck am I supposed to know when it’s safe to use a new kernel?
I suspect its not the kernel that is the problem but an issue with the hardware.
I run btrfs and update to the new kernels soon after they hit stable repo.
What would be interesting to see is what the kernel reported when there where errors.
I am assuming that the failures mean that the journal is not written to.
You may want to always boot without the rhgb and quiet options removed from the boot command line. Then you may get to see the kernel error messages that may point to a hardware issue.
maybe f40 changes the I/O load on the drive?
hobbes1069
August 28, 2024, 5:39pm
3
My first thought was a hardware issue as well and was prepared to start looking for a new laptop, but I literally just got done installing F40 again with the default kernel and am having no issues.
I built the Rawhide kernel on my desktop which is 6.11.x and plan to see if that helps at all.
Currently no useful messages from dmesg or journalctl.
It’s easier for other to compare to your experience if you use the stable kernels.
hobbes1069
August 28, 2024, 10:06pm
5
Well, stock F40 kernel up for 4 1/2 hours without issue. Nothing of note in dmesg or journalctl -b0 | grep i btrfs.
The other kernels mentioned have already resulted in btrfs fs corruption which usually locks up the system so it could be difficult to capture the errors unless I go old school with a serial based terminal.
hobbes1069
August 28, 2024, 10:44pm
6
Well interesting… It may not be the kernels per se.
I’ve been running 6.10.6 for a bit and monitoring journalctl remotely to try and capture the error and no issues, but I have not done a full system update yet.
gnwiii
August 28, 2024, 11:47pm
7
Over the years I noticed that drives too often fail during upgrades. Upgrading requires a lot of disk activity, so I expect a drive that is nearing end-of-life may achieve death while updating. Gnome Disks supports S.M.A.R.T (Self-Monitoring, Analysis, and Reporting Technology) and provides a quick “health” assessment but can also run tests that often reveal problems with “healthy” drives.
BTRFS tells you about bitrot that lesser filesystems won’t detect. With larger drive and RAM capacity, encouraging us to store more data, bitrot has become more of an issue.
It is more likely a hardware issue than the new kernel. RAM and storage devices are the prime suspects, so you should use the available tests for both.
2 Likes
hobbes1069
August 30, 2024, 10:41pm
8
SMART came back clean when I first checked about a month ago and now recently. Only about 4TB written total.
Memtest86+ ran 7 times without issue when I first had an issue but running again now just to be sure.
hobbes1069
September 1, 2024, 10:29pm
9
So:
15 memtest86 and 10 continuous BIOS full system checks later (all passed)
Running for two days without issue (single boot)
BIOS update (had nothing to lose so why not?)
And now rebooting I got BTRFS corruption 36 seconds into my latest boot
[ 36.669900] BTRFS info (device nvme0n1p6): leaf 37421056 gen 5653 total ptrs 162 free space 314 owner 257
[ 36.669907] item 0 key (332722 108 0) itemoff 16230 itemsize 53
[ 36.669909] extent data disk bytenr 2746179584 nr 4096
[ 36.669910] extent data offset 0 nr 8192 ram 8192
[ 36.669911] item 1 key (332723 1 0) itemoff 16070 itemsize 160
[ 36.669912] inode generation 5650 size 200 mode 100644
[ 36.669913] item 2 key (332723 12 158977) itemoff 16025 itemsize 45
[ 36.669914] item 3 key (332723 24 3817753667) itemoff 15938 itemsize 87
[ 36.669915] item 4 key (332723 108 0) itemoff 15885 itemsize 53
[ 36.669916] extent data disk bytenr 2746216448 nr 4096
[ 36.669917] extent data offset 0 nr 4096 ram 4096
[ 36.669918] item 5 key (332724 1 0) itemoff 15725 itemsize 160
[ 36.669919] inode generation 5650 size 464 mode 100644
[ 36.669919] item 6 key (332724 12 158977) itemoff 15678 itemsize 47
[ 36.669920] item 7 key (332724 24 3817753667) itemoff 15591 itemsize 87
[ 36.669921] item 8 key (332724 108 0) itemoff 15538 itemsize 53
[ 36.669922] extent data disk bytenr 2746220544 nr 4096
[ 36.669923] extent data offset 0 nr 4096 ram 4096
[ 36.669923] item 9 key (332725 1 0) itemoff 15378 itemsize 160
[ 36.669924] inode generation 5650 size 200 mode 100644
[ 36.669925] item 10 key (332725 12 158977) itemoff 15328 itemsize 50
[ 36.669926] item 11 key (332725 24 3817753667) itemoff 15241 itemsize 87
[ 36.669927] item 12 key (332725 108 0) itemoff 15188 itemsize 53
[ 36.669928] extent data disk bytenr 2746228736 nr 4096
[ 36.669928] extent data offset 0 nr 4096 ram 4096
[ 36.669929] item 13 key (332726 1 0) itemoff 15028 itemsize 160
[ 36.669930] inode generation 5650 size 200 mode 100644
[ 36.669931] item 14 key (332726 12 158977) itemoff 14977 itemsize 51
[ 36.669931] item 15 key (332726 24 3817753667) itemoff 14890 itemsize 87
[ 36.669932] item 16 key (332726 108 0) itemoff 14837 itemsize 53
[ 36.669933] extent data disk bytenr 2746867712 nr 4096
[ 36.669934] extent data offset 0 nr 4096 ram 4096
[ 36.669934] item 17 key (332737 1 0) itemoff 14677 itemsize 160
[ 36.669935] inode generation 5650 size 68 mode 100644
[ 36.669936] item 18 key (332737 12 142176) itemoff 14647 itemsize 30
[ 36.669937] item 19 key (332737 24 3817753667) itemoff 14555 itemsize 92
[ 36.669938] item 20 key (332737 108 0) itemoff 14502 itemsize 53
[ 36.669939] extent data disk bytenr 2746880000 nr 4096
[ 36.669939] extent data offset 0 nr 4096 ram 4096
[ 36.669940] item 21 key (332739 1 0) itemoff 14342 itemsize 160
[ 36.669941] inode generation 5650 size 70 mode 100644
[ 36.669942] item 22 key (332739 12 142176) itemoff 14321 itemsize 21
[ 36.669942] item 23 key (332739 24 3817753667) itemoff 14229 itemsize 92
[ 36.669943] item 24 key (332739 108 0) itemoff 14176 itemsize 53
[ 36.669944] extent data disk bytenr 2746892288 nr 4096
[ 36.669945] extent data offset 0 nr 4096 ram 4096
[ 36.669945] item 25 key (332740 1 0) itemoff 14016 itemsize 160
[ 36.669946] inode generation 5653 size 8388608 mode 100640
[ 36.669947] item 26 key (332740 12 158913) itemoff 13992 itemsize 24
[ 36.669948] item 27 key (332740 24 843765919) itemoff 13938 itemsize 54
[ 36.669949] item 28 key (332740 24 2038346239) itemoff 13833 itemsize 105
[ 36.669950] item 29 key (332740 24 3817753667) itemoff 13756 itemsize 77
[ 36.669951] item 30 key (332740 108 0) itemoff 13703 itemsize 53
[ 36.669952] extent data disk bytenr 18661376 nr 4096
[ 36.669952] extent data offset 0 nr 4096 ram 4096
[ 36.669953] item 31 key (332740 108 4096) itemoff 13650 itemsize 53
[ 36.669954] extent data disk bytenr 1573978112 nr 8384512
[ 36.669955] extent data offset 0 nr 4251648 ram 8384512
[ 36.669955] item 32 key (332740 108 4255744) itemoff 13597 itemsize 53
[ 36.669956] extent data disk bytenr 1573978112 nr 8384512
[ 36.669957] extent data offset 4251648 nr 4096 ram 8384512
[ 36.669958] item 33 key (332740 108 4259840) itemoff 13544 itemsize 53
[ 36.669959] extent data disk bytenr 1573978112 nr 8384512
[ 36.669959] extent data offset 4255744 nr 49152 ram 8384512
[ 36.669960] item 34 key (332740 108 4308992) itemoff 13491 itemsize 53
[ 36.669961] extent data disk bytenr 1573978112 nr 8384512
[ 36.669961] extent data offset 4304896 nr 4096 ram 8384512
[ 36.669962] item 35 key (332740 108 4313088) itemoff 13438 itemsize 53
[ 36.669963] extent data disk bytenr 1573978112 nr 8384512
[ 36.669964] extent data offset 4308992 nr 24576 ram 8384512
[ 36.669964] item 36 key (332740 108 4337664) itemoff 13385 itemsize 53
[ 36.669965] extent data disk bytenr 1573978112 nr 8384512
[ 36.669966] extent data offset 4333568 nr 4096 ram 8384512
[ 36.669967] item 37 key (332740 108 4341760) itemoff 13332 itemsize 53
[ 36.669967] extent data disk bytenr 1573978112 nr 8384512
[ 36.669968] extent data offset 4337664 nr 757760 ram 8384512
[ 36.669969] item 38 key (332740 108 5099520) itemoff 13279 itemsize 53
[ 36.669970] extent data disk bytenr 1573978112 nr 8384512
[ 36.669970] extent data offset 5095424 nr 4096 ram 8384512
[ 36.669971] item 39 key (332740 108 5103616) itemoff 13226 itemsize 53
[ 36.669972] extent data disk bytenr 1573978112 nr 8384512
[ 36.669972] extent data offset 5099520 nr 49152 ram 8384512
[ 36.669973] item 40 key (332740 108 5152768) itemoff 13173 itemsize 53
[ 36.669974] extent data disk bytenr 1573978112 nr 8384512
[ 36.669974] extent data offset 5148672 nr 3235840 ram 8384512
[ 36.669975] item 41 key (332741 1 0) itemoff 13013 itemsize 160
[ 36.669976] inode generation 5653 size 8388608 mode 100640
[ 36.669977] item 42 key (332741 12 158913) itemoff 12922 itemsize 91
[ 36.669978] item 43 key (332741 24 843765919) itemoff 12868 itemsize 54
[ 36.669979] item 44 key (332741 24 2038346239) itemoff 12763 itemsize 105
[ 36.669979] item 45 key (332741 24 3817753667) itemoff 12686 itemsize 77
[ 36.669980] item 46 key (332741 108 0) itemoff 12633 itemsize 53
[ 36.669981] extent data disk bytenr 19664896 nr 4096
[ 36.669982] extent data offset 0 nr 131072 ram 131072
[ 36.669983] item 47 key (332741 108 131072) itemoff 12580 itemsize 53
[ 36.669983] extent data disk bytenr 18665472 nr 4096
[ 36.669984] extent data offset 0 nr 131072 ram 131072
[ 36.669985] item 48 key (332741 108 262144) itemoff 12527 itemsize 53
[ 36.669986] extent data disk bytenr 2235768832 nr 4096
[ 36.669986] extent data offset 0 nr 131072 ram 131072
[ 36.669987] item 49 key (332741 108 393216) itemoff 12474 itemsize 53
[ 36.669988] extent data disk bytenr 2209714176 nr 4096
[ 36.669988] extent data offset 0 nr 131072 ram 131072
[ 36.669989] item 50 key (332741 108 524288) itemoff 12421 itemsize 53
[ 36.669990] extent data disk bytenr 2235772928 nr 4096
[ 36.669990] extent data offset 0 nr 131072 ram 131072
[ 36.669991] item 51 key (332741 108 655360) itemoff 12368 itemsize 53
[ 36.669992] extent data disk bytenr 2209718272 nr 4096
[ 36.669993] extent data offset 0 nr 131072 ram 131072
[ 36.669993] item 52 key (332741 108 786432) itemoff 12315 itemsize 53
[ 36.669994] extent data disk bytenr 2249777152 nr 4096
[ 36.669995] extent data offset 0 nr 131072 ram 131072
[ 36.669995] item 53 key (332741 108 917504) itemoff 12262 itemsize 53
[ 36.669996] extent data disk bytenr 2249781248 nr 4096
[ 36.669997] extent data offset 0 nr 131072 ram 131072
[ 36.669998] item 54 key (332741 108 1048576) itemoff 12209 itemsize 53
[ 36.669998] extent data disk bytenr 2235777024 nr 4096
[ 36.669999] extent data offset 0 nr 131072 ram 131072
[ 36.670000] item 55 key (332741 108 1179648) itemoff 12156 itemsize 53
[ 36.670001] extent data disk bytenr 2249908224 nr 4096
[ 36.670001] extent data offset 0 nr 131072 ram 131072
[ 36.670002] item 56 key (332741 108 1310720) itemoff 12103 itemsize 53
[ 36.670003] extent data disk bytenr 2249912320 nr 4096
[ 36.670003] extent data offset 0 nr 131072 ram 131072
[ 36.670004] item 57 key (332741 108 1441792) itemoff 12050 itemsize 53
[ 36.670005] extent data disk bytenr 2249916416 nr 4096
[ 36.670005] extent data offset 0 nr 131072 ram 131072
[ 36.670006] item 58 key (332741 108 1572864) itemoff 11997 itemsize 53
[ 36.670007] extent data disk bytenr 2235781120 nr 4096
[ 36.670007] extent data offset 0 nr 131072 ram 131072
[ 36.670008] item 59 key (332741 108 1703936) itemoff 11944 itemsize 53
[ 36.670009] extent data disk bytenr 2249920512 nr 4096
[ 36.670010] extent data offset 0 nr 131072 ram 131072
[ 36.670010] item 60 key (332741 108 1835008) itemoff 11891 itemsize 53
[ 36.670011] extent data disk bytenr 2249924608 nr 4096
[ 36.670012] extent data offset 0 nr 131072 ram 131072
[ 36.670012] item 61 key (332741 108 1966080) itemoff 11838 itemsize 53
[ 36.670013] extent data disk bytenr 2253266944 nr 4096
[ 36.670014] extent data offset 0 nr 131072 ram 131072
[ 36.670014] item 62 key (332741 108 2097152) itemoff 11785 itemsize 53
[ 36.670015] extent data disk bytenr 2235785216 nr 4096
[ 36.670016] extent data offset 0 nr 131072 ram 131072
[ 36.670016] item 63 key (332741 108 2228224) itemoff 11732 itemsize 53
[ 36.670017] extent data disk bytenr 2253271040 nr 4096
[ 36.670018] extent data offset 0 nr 131072 ram 131072
[ 36.670018] item 64 key (332741 108 2359296) itemoff 11679 itemsize 53
[ 36.670019] extent data disk bytenr 2253303808 nr 4096
[ 36.670020] extent data offset 0 nr 131072 ram 131072
[ 36.670021] item 65 key (332741 108 2490368) itemoff 11626 itemsize 53
[ 36.670021] extent data disk bytenr 2253647872 nr 4096
[ 36.670022] extent data offset 0 nr 131072 ram 131072
[ 36.670023] item 66 key (332741 108 2621440) itemoff 11573 itemsize 53
[ 36.670024] extent data disk bytenr 2235789312 nr 4096
[ 36.670024] extent data offset 0 nr 131072 ram 131072
[ 36.670025] item 67 key (332741 108 2752512) itemoff 11520 itemsize 53
[ 36.670026] extent data disk bytenr 2253651968 nr 4096
[ 36.670026] extent data offset 0 nr 131072 ram 131072
[ 36.670027] item 68 key (332741 108 2883584) itemoff 11467 itemsize 53
[ 36.670028] extent data disk bytenr 2253656064 nr 4096
[ 36.670028] extent data offset 0 nr 131072 ram 131072
[ 36.670029] item 69 key (332741 108 3014656) itemoff 11414 itemsize 53
[ 36.670030] extent data disk bytenr 2253660160 nr 4096
[ 36.670031] extent data offset 0 nr 131072 ram 131072
[ 36.670031] item 70 key (332741 108 3145728) itemoff 11361 itemsize 53
[ 36.670032] extent data disk bytenr 2235793408 nr 4096
[ 36.670033] extent data offset 0 nr 131072 ram 131072
[ 36.670033] item 71 key (332741 108 3276800) itemoff 11308 itemsize 53
[ 36.670034] extent data disk bytenr 2254028800 nr 4096
[ 36.670035] extent data offset 0 nr 131072 ram 131072
[ 36.670035] item 72 key (332741 108 3407872) itemoff 11255 itemsize 53
[ 36.670036] extent data disk bytenr 2254217216 nr 4096
[ 36.670037] extent data offset 0 nr 131072 ram 131072
[ 36.670037] item 73 key (332741 108 3538944) itemoff 11202 itemsize 53
[ 36.670038] extent data disk bytenr 2254495744 nr 4096
[ 36.670039] extent data offset 0 nr 131072 ram 131072
[ 36.670039] item 74 key (332741 108 3670016) itemoff 11149 itemsize 53
[ 36.670040] extent data disk bytenr 2235797504 nr 20480
[ 36.670041] extent data offset 0 nr 131072 ram 131072
[ 36.670042] item 75 key (332741 108 3801088) itemoff 11096 itemsize 53
[ 36.670042] extent data disk bytenr 2177900544 nr 40960
[ 36.670043] extent data offset 0 nr 131072 ram 131072
[ 36.670044] item 76 key (332741 108 3932160) itemoff 11043 itemsize 53
[ 36.670045] extent data disk bytenr 2310635520 nr 28672
[ 36.670045] extent data offset 0 nr 131072 ram 131072
[ 36.670046] item 77 key (332741 108 4063232) itemoff 10990 itemsize 53
[ 36.670047] extent data disk bytenr 2177941504 nr 36864
[ 36.670047] extent data offset 0 nr 131072 ram 131072
[ 36.670048] item 78 key (332741 108 4194304) itemoff 10937 itemsize 53
[ 36.670049] extent data disk bytenr 2235817984 nr 28672
[ 36.670049] extent data offset 0 nr 131072 ram 131072
[ 36.670050] item 79 key (332741 108 4325376) itemoff 10884 itemsize 53
[ 36.670051] extent data disk bytenr 2311032832 nr 28672
[ 36.670051] extent data offset 0 nr 131072 ram 131072
[ 36.670052] item 80 key (332741 108 4456448) itemoff 10831 itemsize 53
[ 36.670053] extent data disk bytenr 2257072128 nr 20480
[ 36.670054] extent data offset 0 nr 90112 ram 90112
[ 36.670054] item 81 key (332742 1 0) itemoff 10671 itemsize 160
[ 36.670055] inode generation 5653 size 6 mode 40700
[ 36.670056] item 82 key (332742 12 143819) itemoff 10585 itemsize 86
[ 36.670057] item 83 key (332742 24 3817753667) itemoff 10512 itemsize 73
[ 36.670058] item 84 key (332742 84 3284084670) itemoff 10479 itemsize 33
[ 36.670059] dir oid 332743 flags 2
[ 36.670059] item 85 key (332742 96 2) itemoff 10446 itemsize 33
[ 36.670060] item 86 key (332743 1 0) itemoff 10286 itemsize 160
[ 36.670061] inode generation 5653 size 0 mode 41777
[ 36.670062] item 87 key (332743 12 332742) itemoff 10273 itemsize 13
[ 36.670063] item 88 key (332743 24 3817753667) itemoff 10200 itemsize 73
[ 36.670064] item 89 key (332744 1 0) itemoff 10040 itemsize 160
[ 36.670065] inode generation 5653 size 6 mode 40700
[ 36.670065] item 90 key (332744 12 143819) itemoff 9950 itemsize 90
[ 36.670066] item 91 key (332744 24 3817753667) itemoff 9877 itemsize 73
[ 36.670067] item 92 key (332744 84 3284084670) itemoff 9844 itemsize 33
[ 36.670068] dir oid 332745 flags 2
[ 36.670069] item 93 key (332744 96 2) itemoff 9811 itemsize 33
[ 36.670069] item 94 key (332745 1 0) itemoff 9651 itemsize 160
[ 36.670070] inode generation 5653 size 0 mode 41777
[ 36.670071] item 95 key (332745 12 332744) itemoff 9638 itemsize 13
[ 36.670072] item 96 key (332745 24 3817753667) itemoff 9565 itemsize 73
[ 36.670073] item 97 key (332746 1 0) itemoff 9405 itemsize 160
[ 36.670074] inode generation 5653 size 6 mode 40700
[ 36.670074] item 98 key (332746 12 143819) itemoff 9320 itemsize 85
[ 36.670075] item 99 key (332746 24 3817753667) itemoff 9247 itemsize 73
[ 36.670076] item 100 key (332746 84 3284084670) itemoff 9214 itemsize 33
[ 36.670077] dir oid 332747 flags 2
[ 36.670078] item 101 key (332746 96 2) itemoff 9181 itemsize 33
[ 36.670078] item 102 key (332747 1 0) itemoff 9021 itemsize 160
[ 36.670079] inode generation 5653 size 0 mode 41777
[ 36.670080] item 103 key (332747 12 332746) itemoff 9008 itemsize 13
[ 36.670081] item 104 key (332747 24 3817753667) itemoff 8935 itemsize 73
[ 36.670082] item 105 key (332748 1 0) itemoff 8775 itemsize 160
[ 36.670083] inode generation 5653 size 6 mode 40700
[ 36.670083] item 106 key (332748 12 143819) itemoff 8696 itemsize 79
[ 36.670084] item 107 key (332748 24 3817753667) itemoff 8623 itemsize 73
[ 36.670085] item 108 key (332748 84 3284084670) itemoff 8590 itemsize 33
[ 36.670086] dir oid 332749 flags 2
[ 36.670086] item 109 key (332748 96 2) itemoff 8557 itemsize 33
[ 36.670087] item 110 key (332749 1 0) itemoff 8397 itemsize 160
[ 36.670088] inode generation 5653 size 0 mode 41777
[ 36.670089] item 111 key (332749 12 332748) itemoff 8384 itemsize 13
[ 36.670090] item 112 key (332749 24 3817753667) itemoff 8311 itemsize 73
[ 36.670091] item 113 key (332750 1 0) itemoff 8151 itemsize 160
[ 36.670091] inode generation 5653 size 6 mode 40700
[ 36.670092] item 114 key (332750 12 143819) itemoff 8068 itemsize 83
[ 36.670093] item 115 key (332750 24 3817753667) itemoff 7995 itemsize 73
[ 36.670094] item 116 key (332750 84 3284084670) itemoff 7962 itemsize 33
[ 36.670095] dir oid 332751 flags 2
[ 36.670095] item 117 key (332750 96 2) itemoff 7929 itemsize 33
[ 36.670096] item 118 key (332751 1 0) itemoff 7769 itemsize 160
[ 36.670097] inode generation 5653 size 0 mode 41777
[ 36.670098] item 119 key (332751 12 332750) itemoff 7756 itemsize 13
[ 36.670099] item 120 key (332751 24 3817753667) itemoff 7683 itemsize 73
[ 36.670100] item 121 key (332752 1 0) itemoff 7523 itemsize 160
[ 36.670100] inode generation 5653 size 6 mode 40700
[ 36.670101] item 122 key (332752 12 143819) itemoff 7442 itemsize 81
[ 36.670102] item 123 key (332752 24 3817753667) itemoff 7369 itemsize 73
[ 36.670103] item 124 key (332752 84 3284084670) itemoff 7336 itemsize 33
[ 36.670104] dir oid 332753 flags 2
[ 36.670104] item 125 key (332752 96 2) itemoff 7303 itemsize 33
[ 36.670105] item 126 key (332753 1 0) itemoff 7143 itemsize 160
[ 36.670106] inode generation 5653 size 0 mode 41777
[ 36.670107] item 127 key (332753 12 332752) itemoff 7130 itemsize 13
[ 36.670108] item 128 key (332753 24 3817753667) itemoff 7057 itemsize 73
[ 36.670108] item 129 key (332754 1 0) itemoff 6897 itemsize 160
[ 36.670109] inode generation 5653 size 6 mode 40700
[ 36.670110] item 130 key (332754 12 143819) itemoff 6805 itemsize 92
[ 36.670111] item 131 key (332754 24 3817753667) itemoff 6732 itemsize 73
[ 36.670112] item 132 key (332754 84 3284084670) itemoff 6699 itemsize 33
[ 36.670113] dir oid 332755 flags 2
[ 36.670113] item 133 key (332754 96 2) itemoff 6666 itemsize 33
[ 36.670114] item 134 key (332755 1 0) itemoff 6506 itemsize 160
[ 36.670115] inode generation 5653 size 0 mode 41777
[ 36.670116] item 135 key (332755 12 332754) itemoff 6493 itemsize 13
[ 36.670116] item 136 key (332755 24 3817753667) itemoff 6420 itemsize 73
[ 36.670117] item 137 key (332756 1 0) itemoff 6260 itemsize 160
[ 36.670118] inode generation 5653 size 6 mode 40700
[ 36.670119] item 138 key (332756 12 143819) itemoff 6180 itemsize 80
[ 36.670120] item 139 key (332756 24 3817753667) itemoff 6107 itemsize 73
[ 36.670121] item 140 key (332756 84 3284084670) itemoff 6074 itemsize 33
[ 36.670121] dir oid 332757 flags 2
[ 36.670122] item 141 key (332756 96 2) itemoff 6041 itemsize 33
[ 36.670123] item 142 key (332757 1 0) itemoff 5881 itemsize 160
[ 36.670124] inode generation 5653 size 0 mode 41777
[ 36.670124] item 143 key (332757 12 332756) itemoff 5868 itemsize 13
[ 36.670125] item 144 key (332757 24 3817753667) itemoff 5795 itemsize 73
[ 36.670126] item 145 key (332758 1 0) itemoff 5635 itemsize 160
[ 36.670127] inode generation 5653 size 6 mode 40700
[ 36.670128] item 146 key (332758 12 143819) itemoff 5540 itemsize 95
[ 36.670129] item 147 key (332758 24 3817753667) itemoff 5467 itemsize 73
[ 36.670129] item 148 key (332758 84 3284084670) itemoff 5434 itemsize 33
[ 36.670130] dir oid 332759 flags 2
[ 36.670131] item 149 key (332758 96 2) itemoff 5401 itemsize 33
[ 36.670132] item 150 key (332759 1 0) itemoff 5241 itemsize 160
[ 36.670133] inode generation 5653 size 0 mode 41777
[ 36.670133] item 151 key (332759 12 332758) itemoff 5228 itemsize 13
[ 36.670134] item 152 key (332759 24 3817753667) itemoff 5155 itemsize 73
[ 36.670135] item 153 key (332760 1 0) itemoff 4995 itemsize 160
[ 36.670136] inode generation 5653 size 6 mode 40700
[ 36.670137] item 154 key (332760 12 143819) itemoff 4909 itemsize 86
[ 36.670137] item 155 key (332760 24 3817753667) itemoff 33559268 itemsize -16777216
[ 36.670139] item 156 key (216172782164115455 0 3284074496) itemoff 4803 itemsize 33
[ 36.670140] item 157 key (332760 96 2) itemoff 4770 itemsize 33
[ 36.670141] item 158 key (332761 1 0) itemoff 4610 itemsize 160
[ 36.670142] inode generation 5653 size 0 mode 41777
[ 36.670142] item 159 key (332761 12 332760) itemoff 4597 itemsize 0
[ 36.670143] item 160 key (216172782113783808 0 3817734144) itemoff 4524 itemsize 73
[ 36.670144] item 161 key (332762 1 0) itemoff 4364 itemsize 160
[ 36.670145] inode generation 5653 size 6 mode 40700
[ 36.670146] BTRFS error (device nvme0n1p6): block=37421056 write time tree block corruption detected
[ 36.673394] BTRFS: error (device nvme0n1p6) in btrfs_commit_transaction:2505: errno=-5 IO failure (Error while writing out transaction)
[ 36.673399] BTRFS info (device nvme0n1p6 state E): forced readonly
[ 36.673401] BTRFS warning (device nvme0n1p6 state E): Skipping commit of aborted transaction.
[ 36.673403] BTRFS error (device nvme0n1p6 state EA): Transaction aborted (error -5)
[ 36.673404] BTRFS: error (device nvme0n1p6 state EA) in cleanup_transaction:1999: errno=-5 IO failure
[ 36.673792] systemd-journald[653]: /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: IO error, rotating.
[ 36.673859] systemd-journald[653]: Failed to rotate /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: Read-only file system
[ 36.674613] systemd-journald[653]: Failed to write entry to /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal (10 items, 278 bytes) despite vacuuming, ignoring: Input/output error
[ 36.674643] systemd-journald[653]: Failed to rotate /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: Read-only file system
[ 36.675265] systemd-journald[653]: /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: IO error, rotating.
[ 36.675270] systemd-journald[653]: Suppressing rotation, as we already rotated immediately before write attempt. Giving up.
[ 36.675294] systemd-journald[653]: Failed to rotate /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: Read-only file system
[ 36.675904] systemd-journald[653]: /var/log/journal/79426a8f3d2348d5873e7ed1d6cd01b7/system.journal: IO error, rotating.
[ 36.675908] systemd-journald[653]: Suppressing rotation, as we already rotated immediately before write attempt. Giving up.
[ 36.676542] systemd-journald[653]: Suppressing rotation, as we already rotated immediately before write attempt. Giving up.
casey128
September 1, 2024, 11:50pm
10
Do you get similar corruption when you install using ext4 instead of btrfs?
hobbes1069
September 2, 2024, 1:16am
11
Haven’t tried, but my gut is telling me it’s not a BTRFS issue, it’s just catching it. Most of the time it locks things down to RO before there’s permanent corruption.
Still have yet to reproduce the issue on the stock F40 kernel. Because it’s so random, I can’t say it IS newer kernels, but it seems to be part of the problem.
barryascott
September 2, 2024, 6:19am
12
Ext4 may be corrupted as well but as it lacks the checksums that btrfs has you may be able to run of a while before the corruption manifests.
barryascott
September 2, 2024, 6:22am
13
It may be worth unplugging and reseating ram modules and data cables.
If there are some specks of dust in the connectors it may make a intermittent connection based on vibration of the PC.
1 Like
casey128
September 2, 2024, 10:25am
14
Newer kernels shouldn’t matter if the root cause is a hardware error. The btrfs code would likely differ between kernels though. Have you tried Fedora 39 or other distros (like Ubuntu LTS) with older kernels?
hobbes1069
September 2, 2024, 11:39am
15
It’s a laptop so that’s not a lot of fun. I’m assuming the RAM is good after 15+ Memtest86 passes. Not much in the way of cables. HD is NVME.
hobbes1069
September 2, 2024, 11:40am
16
I don’t disagree, just reporting my results. I may just stick to the stock kernel for a while and see if it eventually reproduces the same problem.
gnwiii
September 2, 2024, 12:15pm
17
I would reseat the NVME after “cleaning” the socket with canned air and carefully checking the pins on the card. You should be careful to guard against potential static electricity damage – always use a grounded wrist strap and ideally a grounded conductive mat. Some vendors provide NVME test software as a bootable USB image.
It isn’t clear what you mean by “stock” kernel: is it the latest updated kernel or an earlier version? You should check for firmware updates from the SSD vendor and also problem reports for that model on other forums. I would also try some other memory test programs .
Try other filters in journalctl. Searching for btrfs problems is looking at symptoms, but there may be messages closer to the causal mechanism. If you make a list of the dates and times when an error is detected you can carefully review the hours leading up to the error.
hobbes1069
September 2, 2024, 1:06pm
18
While possible, I don’t think it’s very likely that the factory installed SSD has a “bad connection” Doing that will be a last resort because I have to take the rubber strips off the bottom and I don’t think they’ll go back again (already done once).
By stock, I mean the kernel that came with F40 (6.8.x).
barryascott
September 2, 2024, 1:19pm
19
Dust can make its way in-between the drive and the connector over time.
That is a ‘stock’ kernel for that time (f40 release kernel), but should probably be referred to as a fedora kernel by version number.