Note also I forgot to mention that the sda disk on which I am getting errors has sda3 as mentioned merged with two smaller partitions on a 256 GB older SSD. Above error reported on newer sda3.
Adding a lyer of complexity is that the merged btrfs volume is luks1 encrypted and automounted by fstab, i.e.
/dev/mapper/luks-dd129b86-469d-4042-8b90-00d1acb2a1bf /srv/lpssd/archives btrfs defaults,compress=zstd:9,discard=async,x-systemd.device-timeout=0 0 2
with /etc/crypttab
$ sudo cat /etc/crypttab
...
# RK 19/4/2023: btrfs, archive <- archive2 <-archive3 refer /etc/fstab
luks-dd129b86-469d-4042-8b90-00d1acb2a1bf UUID=dd129b86-469d-4042-8b90-00d1acb2a1bf /root/Documents/encrypted-disks/archive-key luks,nofail,discard
luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f UUID=9b746590-b83b-4e35-bb7c-1c07d5130a4f /root/Documents/encrypted-disks/archive2-key luks,nofail,discard
luks-1e32597b-d73f-48de-a899-af443d96b150 UUID=1e32597b-d73f-48de-a899-af443d96b150 /root/Documents/encrypted-disks/archive3-key luks,nofail,discard
...
While the lsblk disk structure for btrfs backup volume is merged sda3, sdb1, sdb2
sda
├─sda1 vfat FAT32 ESP 3826-001F 1.4G 2% /boot/efi
├─sda2 ext4 1.0 boot 667dcfd4-ff09-4f49-a7a6-08e6c7b49639 722.2M 44% /boot
└─sda3 crypto_LUKS 1 dd129b86-469d-4042-8b90-00d1acb2a1bf
└─luks-dd129b86-469d-4042-8b90-00d1acb2a1bf
btrfs archive 2d85cf85-6165-49a8-8fd1-68651fe12fdc /srv/lpssd/archives
sdb
├─sdb1 crypto_LUKS 1 1e32597b-d73f-48de-a899-af443d96b150
│ └─luks-1e32597b-d73f-48de-a899-af443d96b150
│ btrfs archive 2d85cf85-6165-49a8-8fd1-68651fe12fdc
└─sdb2 crypto_LUKS 1 9b746590-b83b-4e35-bb7c-1c07d5130a4f
└─luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f
btrfs archive 2d85cf85-6165-49a8-8fd1-68651fe12fdc
NOW re the above comment re dmesg, for the disk T-FORCE I get:
$ dmesg | grep error
[ 5.775907] sas: ata7: end_device-7:0: dev error handler
[ 5.840280] sas: ata8: end_device-8:0: dev error handler
[ 6.095538] sas: ata8: end_device-8:0: cmd error handler
[ 6.095566] sas: ata8: end_device-8:0: dev error handler
[ 6.308785] sas: ata8: end_device-8:0: cmd error handler
[ 6.308843] sas: ata8: end_device-8:0: dev error handler
[ 6.333809] sas: ata8: end_device-8:0: cmd error handler
[ 6.333869] sas: ata8: end_device-8:0: dev error handler
[ 6.392722] sas: ata8: end_device-8:0: cmd error handler
[ 6.392784] sas: ata8: end_device-8:0: dev error handler
[ 11.416595] sas: ata8: end_device-8:0: cmd error handler
[ 11.416647] sas: ata8: end_device-8:0: dev error handler
[ 11.445685] sas: ata8: end_device-8:0: cmd error handler
[ 11.445723] sas: ata8: end_device-8:0: dev error handler
[ 11.490737] sas: ata8: end_device-8:0: cmd error handler
[ 11.490776] sas: ata8: end_device-8:0: dev error handler
[ 11.547742] sas: ata8: end_device-8:0: cmd error handler
[ 11.547794] sas: ata8: end_device-8:0: dev error handler
[ 11.612727] sas: ata8: end_device-8:0: cmd error handler
[ 11.612791] sas: ata8: end_device-8:0: dev error handler
[ 24.084697] sas: ata8: end_device-8:0: cmd error handler
[ 24.084708] sas: ata8: end_device-8:0: dev error handler
[ 28.326576] audit: type=1338 audit(1691104184.198:56): module=crypt op=ctr ppid=1 pid=1422 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="systemd-cryptse" exe="/usr/lib/systemd/systemd-cryptsetup" subj=system_u:system_r:lvm_t:s0 dev=253:11 error_msg='success' res=1
[robertk@earth:~]$ dmesg | grep Error
[ 2.584766] ERST: Error Record Serialization Table (ERST) support is initialized.
[ 2.989070] RAS: Correctable Errors collector initialized.
[robertk@earth:~]$ dmesg | grep Warning
[ 0.012202] ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20230331/tbfadt-669)
[robertk@earth:~]$ dmesg | grep warning
[robertk@earth:~]$ dmesg | grep T-F
[ 2.929265] ata1.00: ATA-9: T-FORCE 1TB, V0718B0, max UDMA/133
[ 2.937528] scsi 0:0:0:0: Direct-Access ATA T-FORCE 1TB 8B0 PQ: 0 ANSI: 5
[robertk@earth:~]$ dmesg | grep T-FORCE
[ 2.929265] ata1.00: ATA-9: T-FORCE 1TB, V0718B0, max UDMA/133
[ 2.937528] scsi 0:0:0:0: Direct-Access ATA T-FORCE 1TB 8B0 PQ: 0 ANSI: 5
[robertk@earth:~]$ dmesg | grep '/dev'
[ 0.000000] Command line: BOOT_IMAGE=(hd1,gpt2)/vmlinuz-6.4.7-200.fc38.x86_64 root=/dev/mapper/VG01_nvme_pcie-rootfs ro rd.driver.blacklist=nouveau modprobe.blacklist=nouveau rd.lvm.lv=VG01_nvme_pcie/rootfs rd.luks.uuid=luks-160cee22-ab53-47b2-a48a-382fca72928a rhgb nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init rd.driver.blacklist=nouveau modprobe.blacklist=nouveau
[ 0.309212] Kernel command line: BOOT_IMAGE=(hd1,gpt2)/vmlinuz-6.4.7-200.fc38.x86_64 root=/dev/mapper/VG01_nvme_pcie-rootfs ro rd.driver.blacklist=nouveau modprobe.blacklist=nouveau rd.lvm.lv=VG01_nvme_pcie/rootfs rd.luks.uuid=luks-160cee22-ab53-47b2-a48a-382fca72928a rhgb nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init rd.driver.blacklist=nouveau modprobe.blacklist=nouveau
[ 2.554497] input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input0
[ 2.554792] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[ 3.587877] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.4/3-1.4:1.0/0003:046B:FF10.0001/input/input2
[ 3.591407] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.4/3-1.4:1.1/0003:046B:FF10.0002/input/input3
[ 3.720017] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 3.808870] input: CHICONY HP Basic USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2.2/3-1.2.2:1.0/0003:03F0:0024.0003/input/input4
[ 3.840845] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 4.043629] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2.3/3-1.2.3:1.0/0003:046D:C016.0004/input/input5
[ 22.669586] systemd[1]: Relabelled /dev, /dev/shm, /run, /sys/fs/cgroup in 90.460ms.
[ 23.478730] systemd[1]: Reached target blockdev@dev-mapper-luks\x2d160cee22\x2dab53\x2d47b2\x2da48a\x2d382fca72928a.target - Block Device Preparation for /dev/mapper/luks-160cee22-ab53-47b2-a48a-382fca72928a.
[ 24.082370] input: PC Speaker as /devices/platform/pcspkr/input/input6
[ 24.119728] Adding 8388604k swap on /dev/zram0. Priority:100 extents:1 across:8388604k SSDscFS
[ 24.432118] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:02.0/0000:05:00.1/sound/card1/input7
[ 24.432218] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:02.0/0000:05:00.1/sound/card1/input8
[ 24.432304] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:02.0/0000:05:00.1/sound/card1/input9
[ 24.432386] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:02.0/0000:05:00.1/sound/card1/input10
[ 24.478501] input: HDA Intel PCH Front Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input11
[ 24.478604] input: HDA Intel PCH Rear Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input12
[ 24.478684] input: HDA Intel PCH Line as /devices/pci0000:00/0000:00:1b.0/sound/card0/input13
[ 24.478756] input: HDA Intel PCH Line Out Front as /devices/pci0000:00/0000:00:1b.0/sound/card0/input14
[ 24.478837] input: HDA Intel PCH Line Out Surround as /devices/pci0000:00/0000:00:1b.0/sound/card0/input15
[ 24.478887] input: HDA Intel PCH Line Out CLFE as /devices/pci0000:00/0000:00:1b.0/sound/card0/input16
[ 24.478940] input: HDA Intel PCH Line Out Side as /devices/pci0000:00/0000:00:1b.0/sound/card0/input17
[ 24.478998] input: HDA Intel PCH Front Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input18
[ 28.341728] BTRFS: device label archive devid 1 transid 44806 /dev/dm-11 scanned by (udev-worker) (1286)
[ 28.882776] BTRFS: device label archive devid 3 transid 44806 /dev/dm-12 scanned by (udev-worker) (1286)
[ 29.048837] BTRFS: device label archive devid 2 transid 44806 /dev/dm-14 scanned by (udev-worker) (1297)
[ 29.364894] BTRFS info (device dm-11): bdev /dev/mapper/luks-dd129b86-469d-4042-8b90-00d1acb2a1bf errs: wr 0, rd 0, flush 0, corrupt 106, gen 0
[ 29.392200] BTRFS info: devid 3 device path /dev/mapper/luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f changed to /dev/dm-12 scanned by (udev-worker) (1307)
[ 29.395337] BTRFS info: devid 3 device path /dev/dm-12 changed to /dev/mapper/luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f scanned by (udev-worker) (1307)
[ 29.568860] BTRFS info: devid 3 device path /dev/mapper/luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f changed to /dev/dm-12 scanned by (udev-worker) (1326)
[ 29.571065] BTRFS info: devid 3 device path /dev/dm-12 changed to /dev/mapper/luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f scanned by (udev-worker) (1326)
[robertk@earth:~]$ dmesg | grep sda
[ 2.939343] sd 0:0:0:0: [sda] 2000409264 512-byte logical blocks: (1.02 TB/954 GiB)
[ 2.940217] sd 0:0:0:0: [sda] Write Protect is off
[ 2.940811] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 2.940845] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 2.942074] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 2.947042] sda: sda1 sda2 sda3
[ 2.947789] sd 0:0:0:0: [sda] Attached SCSI disk
[ 24.761271] EXT4-fs (sda2): mounted filesystem 667dcfd4-ff09-4f49-a7a6-08e6c7b49639 r/w with ordered data mode. Quota mode: none.
Nothing stand out as obvious.
Now I was chasing a kernel boot failure just after the BTRFS setup that forced me to power down computer suddenly numerous times. That can be a problem for SSD’s and I am wondering if this crc problem relates to that. Refer to that incidence: Refer Fedora hangs on boot after upgrading to kernel 6.3.4
Though probably not meaningfull to this problem other then forced power downs which is known to upset SSD without relevant circuitry which I am sure applies to the T-FORCE as it was the cheap end of SSDs.
Please note in my rebuild of this computer for F38, I used BTRFS for first time. My experience was not pleasant with this Filesystem prompting my critique. In that case timeshift did not play nicely with BTRFS and I am wondering if borg falls also into this problem. Refer BTRFS - I surrender! Good features BUT Complex, wastes time, not KISS, does not play well - #41 by rkoppelh
I’ll try next to explore what I can do with borg itself to fix the checksum and file 213 given suggestions so far is that disk maybe OK???