A mkfs.btrfs error: "ERROR: error during mkfs: Operation not permitted" when formating an encrypted (LUKS) disk with BTRFS format

Thanks for the info. However, in my case, I haven’t succeeded to format the SSD with any file formats yet.

Interestingly, I could directly format the external SSD disk with BTRFS without using LUKS and mount the disk. What does this result mean? The command log is below.

$ rpm -qf /usr/sbin/mkfs.btrfs
btrfs-progs-6.11-1.fc39.x86_64

$ sudo mkfs.btrfs -L my-passport-ssd-2tb /dev/sda1
btrfs-progs v6.11
See https://btrfs.readthedocs.io for more information.

ERROR: /dev/sda1 appears to contain an existing filesystem (crypto_LUKS)
ERROR: use the -f option to force overwrite of /dev/sda1

$ sudo mkfs.btrfs -L my-passport-ssd-2tb -f /dev/sda1
btrfs-progs v6.11
See https://btrfs.readthedocs.io for more information.

Performing full device TRIM /dev/sda1 (1.82TiB) ...
NOTE: several default settings have changed in version 5.15, please make sure
      this does not affect your deployments:
      - DUP for metadata (-m dup)
      - enabled no-holes (-O no-holes)
      - enabled free-space-tree (-R free-space-tree)

Label:              my-passport-ssd-2tb
UUID:               fe126d85-a9d3-44be-a934-1d7e50cdc1fe
Node size:          16384
Sector size:        4096	(CPU page size: 4096)
Filesystem size:    1.82TiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP               1.00GiB
  System:           DUP               8.00MiB
SSD detected:       yes
Zoned device:       no
Features:           extref, skinny-metadata, no-holes, free-space-tree
Checksum:           crc32c
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1     1.82TiB  /dev/sda1

$ echo $?
0

$ sudo mkdir /mnt/my-passport-ssd-2tb

$ sudo mount /dev/sda1 /mnt/my-passport-ssd-2tb

$ df -hT /mnt/my-passport-ssd-2tb
Filesystem     Type   Size  Used Avail Use% Mounted on
/dev/sda1      btrfs  1.9T  5.8M  1.9T   1% /mnt/my-passport-ssd-2tb

$ sudo touch /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB

$ ls -l /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB
-rw-r--r--. 1 root root 0 Nov  2 16:50 /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB

Then I tried to set LUKS up and format the disk with BTRFS again. But it failed as my past post. The command log is below.

$ sudo cryptsetup luksFormat /dev/sda1
WARNING: Device /dev/sda1 already contains a 'btrfs' superblock signature.

WARNING!
========
This will overwrite data on /dev/sda1 irrevocably.

Are you sure? (Type 'yes' in capital letters): YES
Enter passphrase for /dev/sda1:
Verify passphrase:

$ sudo cryptsetup luksUUID /dev/sda1
d520129d-d028-46ae-b721-7c0f3c47c815

$ sudo cryptsetup luksOpen /dev/sda1 luks-d520129d-d028-46ae-b721-7c0f3c47c815
Enter passphrase for /dev/sda1:

$ sudo dmsetup info luks-d520129d-d028-46ae-b721-7c0f3c47c815
Name:              luks-d520129d-d028-46ae-b721-7c0f3c47c815
State:             ACTIVE
Read Ahead:        4096
Tables present:    LIVE
Open count:        0
Event number:      0
Major, minor:      253, 1
Number of targets: 1
UUID: CRYPT-LUKS2-d520129dd02846aeb7217c0f3c47c815-luks-d520129d-d028-46ae-b721-7c0f3c47c815

$ ls -l /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815
lrwxrwxrwx. 1 root root 7 Nov  2 17:19 /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 -> ../dm-1

$ sudo mkfs.btrfs -L my-passport-ssd-2tb /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815
btrfs-progs v6.11
See https://btrfs.readthedocs.io for more information.

Performing full device TRIM /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 (1.82TiB) ...
NOTE: several default settings have changed in version 5.15, please make sure
      this does not affect your deployments:
      - DUP for metadata (-m dup)
      - enabled no-holes (-O no-holes)
      - enabled free-space-tree (-R free-space-tree)

ERROR: error during mkfs: Operation not permitted

Good news! I was able to format the SSD with BTRFS on the LUKS!

The solution was that I upgraded the Fedora Linux version from 39 to 40. So, the command log is below.

$ rpm -qf /usr/sbin/cryptsetup
cryptsetup-2.7.5-1.fc40.x86_64

$ rpm -qf /usr/sbin/mkfs.btrfs
btrfs-progs-6.11-1.fc40.x86_64

$ uname -r
6.11.5-200.fc40.x86_64

Maybe previously the kernel kernel-6.11.5-100.fc39.x86_64 was executed on Fedora Linux 39. Now the kernel kernel-6.11.5-200.fc40.x86_64 is executed on Fedora Linux 40.

$ rpm -q kernel
kernel-6.10.12-100.fc39.x86_64
kernel-6.11.5-100.fc39.x86_64
kernel-6.11.5-200.fc40.x86_64

I ran the following commands to open LUKS as usual.

$ lsblk -fp
NAME                                                      FSTYPE      FSVER LABEL                 UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
/dev/sda
└─/dev/sda1                                               crypto_LUKS 2                           d520129d-d028-46ae-b721-7c0f3c47c815
/dev/zram0                                                                                                                                            [SWAP]
/dev/nvme0n1
├─/dev/nvme0n1p1                                          vfat        FAT32                       D0B0-4455                             545.6M     9% /boot/efi
├─/dev/nvme0n1p2                                          ext4        1.0                         1072bb16-2ca1-4502-8bda-750ad467f25f    605M    31% /boot
└─/dev/nvme0n1p3                                          crypto_LUKS 2                           09538e5a-f802-40a6-bc22-328604dc2312
  └─/dev/mapper/luks-09538e5a-f802-40a6-bc22-328604dc2312 btrfs             fedora_localhost-live be430ab5-ebda-4f4f-a01e-427d18fb12f1      1T    45% /home
                                                                                                                                                      /
$ sudo cryptsetup luksUUID /dev/sda1
d520129d-d028-46ae-b721-7c0f3c47c815

$ sudo cryptsetup luksOpen /dev/sda1 luks-d520129d-d028-46ae-b721-7c0f3c47c815

$ ls -l /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815
lrwxrwxrwx. 1 root root 7 Nov  4 19:23 /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 -> ../dm-1

The following mkfs.btrfs command succeeded showing the error message ERROR: superblock magic doesn't match.

$ sudo mkfs.btrfs -L my-passport-ssd-2tb /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815
btrfs-progs v6.11
See https://btrfs.readthedocs.io for more information.

ERROR: superblock magic doesn't match
Performing full device TRIM /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 (1.82TiB) ...
NOTE: several default settings have changed in version 5.15, please make sure
      this does not affect your deployments:
      - DUP for metadata (-m dup)
      - enabled no-holes (-O no-holes)
      - enabled free-space-tree (-R free-space-tree)

Label:              my-passport-ssd-2tb
UUID:               4dcfdd12-668b-4bf8-acab-edbed6afa58d
Node size:          16384
Sector size:        4096>-(CPU page size: 4096)
Filesystem size:    1.82TiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP               1.00GiB
  System:           DUP               8.00MiB
SSD detected:       yes
Zoned device:       no
Features:           extref, skinny-metadata, no-holes, free-space-tree
Checksum:           crc32c
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1     1.82TiB  /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815

$ echo $?
0

Then the block device was regoznied by lsblk.

$ lsblk -fp
NAME                                                      FSTYPE      FSVER LABEL                 UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
/dev/sda
└─/dev/sda1                                               crypto_LUKS 2                           d520129d-d028-46ae-b721-7c0f3c47c815
  └─/dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 btrfs             my-passport-ssd-2tb   4dcfdd12-668b-4bf8-acab-edbed6afa58d
/dev/zram0                                                                                                                                            [SWAP]
/dev/nvme0n1
├─/dev/nvme0n1p1                                          vfat        FAT32                       D0B0-4455                             545.6M     9% /boot/efi
├─/dev/nvme0n1p2                                          ext4        1.0                         1072bb16-2ca1-4502-8bda-750ad467f25f    605M    31% /boot
└─/dev/nvme0n1p3                                          crypto_LUKS 2                           09538e5a-f802-40a6-bc22-328604dc2312
  └─/dev/mapper/luks-09538e5a-f802-40a6-bc22-328604dc2312 btrfs             fedora_localhost-live be430ab5-ebda-4f4f-a01e-427d18fb12f1      1T    45% /home
                                                                                                                                                      /
$ lsblk -fpr
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
/dev/sda
/dev/sda1 crypto_LUKS 2  d520129d-d028-46ae-b721-7c0f3c47c815
/dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 btrfs  my-passport-ssd-2tb 4dcfdd12-668b-4bf8-acab-edbed6afa58d
/dev/zram0       [SWAP]
/dev/nvme0n1
/dev/nvme0n1p1 vfat FAT32  D0B0-4455 545.6M 9% /boot/efi
/dev/nvme0n1p2 ext4 1.0  1072bb16-2ca1-4502-8bda-750ad467f25f 605M 31% /boot
/dev/nvme0n1p3 crypto_LUKS 2  09538e5a-f802-40a6-bc22-328604dc2312
/dev/mapper/luks-09538e5a-f802-40a6-bc22-328604dc2312 btrfs  fedora_localhost-live be430ab5-ebda-4f4f-a01e-427d18fb12f1 1T 45% /home\x0a/

I tested with the mount command. And I could mount the disk.

$ sudo mkdir /mnt/my-passport-ssd-2tb

$ sudo mount /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 /mnt/my-passport-ssd-2tb

$ echo $?
0

$ df -hT /mnt/my-passport-ssd-2tb
Filesystem                                            Type   Size  Used Avail Use% Mounted on
/dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 btrfs  1.9T  5.8M  1.9T   1% /mnt/my-passport-ssd-2tb

$ sudo touch /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB

$ ls -l /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB
-rw-r--r--. 1 root root 0 Nov  4 19:39 /mnt/my-passport-ssd-2tb/MY_PASSPORT_SSD_2TB

$ sudo umount /mnt/my-passport-ssd-2tb

$ echo $?
0

$ sudo cryptsetup luksClose /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815

I also tested the mounting the disk by the udisksctl command by a regular user. This is a practical use case. And it worked!

$ udisksctl unlock --block-device /dev/sda1
Passphrase:
Unlocked /dev/sda1 as /dev/dm-1.

$ udisksctl mount -b /dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815
Mounted /dev/dm-1 at /run/media/jaruga/my-passport-ssd-2tb

$ df -hT /run/media/jaruga/my-passport-ssd-2tb
Filesystem                                            Type   Size  Used Avail Use% Mounted on
/dev/mapper/luks-d520129d-d028-46ae-b721-7c0f3c47c815 btrfs  1.9T  5.8M  1.9T   1% /run/media/jaruga/my-passport-ssd-2tb

$ ls -l /run/media/jaruga/my-passport-ssd-2tb
total 0
-rw-r--r--. 1 root root 0 Nov  4 19:39 MY_PASSPORT_SSD_2TB

$ udisksctl unmount -b /dev/dm-1
Unmounted /dev/dm-1.

So, I suppose the upgraded kernel version or something on Fedora Linux 40 fixed the error that I reported.

2 Likes