Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #226347

LUKS and bit rot

Show all headers | View raw

On 2020-08-20 14:30, Dan Ritter wrote:
 > David Christensen wrote:
 >> On 2020-08-20 08:32, rhkramer@gmail.com wrote:
 >> I have been pondering bit-rot mitigation on non-checksumming 
filesystems.
 >>
 >>
 >> Some people have mentioned md RAID.  tomas has mentioned LUKS.  I 
believe
 >> both of them add checksums to the contained contents.  So, bit-rot 
within a
 >> container should be caught by the container driver.
 >
 > This is incorrect. The systems that checksum every write and
 > recalculate and match on every read are BTRFS and ZFS.
 >
 > LVM, LUKS, and mdadm do not.


Let's find out for LUKS:

2020-08-20 23:53:41 root@tinkywinky ~
# cat /etc/debian_version ; uname -a
9.13
Linux tinkywinky 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1+deb9u1 
(2020-06-07) x86_64 GNU/Linux


Start with a blank USB flash drive:

2020-08-21 00:18:32 root@tinkywinky ~
# hexdump /dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
3ba300000


Put an MBR partition table on it:

2020-08-21 00:26:49 root@tinkywinky ~
# parted /dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0 mklabel msdos
Information: You may need to update /etc/fstab.


Create a ~100 Megabyte primary partition:

2020-08-21 00:29:34 root@tinkywinky ~
# parted /dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0 mkpart 
primary 1 100
Information: You may need to update /etc/fstab.

2020-08-21 00:33:09 root@tinkywinky ~
# parted /dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0 u s p
Model: SanDisk Ultra Fit (scsi)
Disk /dev/sdb: 31266816s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start  End      Size     Type     File system  Flags
  1      2048s  194559s  192512s  primary


Put a LUKS partition into the MBR primary partition 1:

2020-08-21 00:33:14 root@tinkywinky ~
# cryptsetup luksFormat 
/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0-part1

WARNING!
========
This will overwrite data on 
/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0:0-part1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:


Find the end of the LUKS header within the LUKS partition:

2020-08-21 00:43:32 root@tinkywinky ~
# dd if=/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0-part1 bs=1M 
status=none | hexdump | tail -n 4
000203f0  32 f4 c3 5e 76 a7 06 80  5b 25 db 4a 21 da 97 e4 
|2..^v...[%.J!...|
00020400  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
05e00000


So, the LUKS header is 0x20400 = 129 KiB = 132,096 bytes.


Open the LUKS partition:

2020-08-21 00:45:48 root@tinkywinky ~
# cryptsetup luksOpen 
/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0-part1 usb100_crypt
Enter passphrase for 
/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0:0-part1:


Create an ext4 filesystem:

2020-08-21 00:47:37 root@tinkywinky ~
# mkfs.ext4 /dev/mapper/usb100_crypt
mke2fs 1.43.4 (31-Jan-2017)
Creating filesystem with 94208 1k blocks and 23616 inodes
Filesystem UUID: 6a8f1680-224b-4bf8-9e10-9fd627c4c0fc
Superblock backups stored on blocks:
	8193, 24577, 40961, 57345, 73729

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done


Look for the start of the filesystem ciphertext within the LUKS partition:

2020-08-21 00:53:36 root@tinkywinky ~
# dd if=/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0-part1 bs=1M 
status=none | hexdump -s 0x20400 | head -n 3
00020400  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
00200000  3c ea 19 de 0d f3 81 a6  e2 ef 35 88 d2 68 1a 0e 
|<.........5..h..|


So, the ciphertext starts at 0x200000 = 2 MiB = 2097152 bytes into the 
LUKS partition.


Mount the filesystem:
2020-08-21 00:53:45 root@tinkywinky ~
# mkdir /mnt/usb100

2020-08-21 00:55:47 root@tinkywinky ~
# mount /dev/mapper/usb100_crypt /mnt/usb100


Create a file containing zeros that fills the filesystem:

2020-08-21 00:56:07 root@tinkywinky ~
# dd if=/dev/zero bs=1M of=/mnt/usb100/zero.bin && sync
dd: error writing '/mnt/usb100/zero.bin': No space left on device
82+0 records in
81+0 records out
85696512 bytes (86 MB, 82 MiB) copied, 1.16472 s, 73.6 MB/s

2020-08-21 00:58:02 root@tinkywinky ~
# df | grep usb100
/dev/mapper/usb100_crypt       86M     84M        0M 100% /mnt/usb100


Verify the contents of the file:

2020-08-21 00:58:31 root@tinkywinky ~
# hexdump /mnt/usb100/zero.bin
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
051ba000


Zero a disk block near the middle of the ciphertext to simulate bit rot 
(more appropriately, device failure):

2020-08-21 01:04:55 root@tinkywinky ~
# dd if=/dev/zero 
of=/dev/disk/by-id/usb-SanDisk_Ultra_Fit_********-0\:0-part1 count=1 
seek=100k
1+0 records in
1+0 records out
512 bytes copied, 0.00108718 s, 471 kB/s


Examine the file:

2020-08-21 01:07:54 root@tinkywinky ~
# hexdump /mnt/usb100/zero.bin | head -n 5
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
01dffc00  75 90 b9 63 df 0d 5a 8f  81 9c 1a 09 ad be 14 7d 
|u..c..Z........}|
01dffc10  e4 ec e1 27 f3 f6 fb af  49 3c 54 1d 79 9c 89 fe 
|...'....I<T.y...|
01dffc20  00 93 50 4f ea 6d d2 62  0c 24 88 5c 43 f5 aa 66 
|..PO.m.b.$.\C..f|

2020-08-21 01:08:27 root@tinkywinky ~
# hexdump /mnt/usb100/zero.bin | tail -n 5
01dffde0  e6 6c ae 11 81 28 bf 48  9f 6e c7 f2 2d 55 f2 13 
|.l...(.H.n..-U..|
01dffdf0  b3 75 e4 42 a1 9a 08 f5  1f 16 10 03 6e 59 b8 d3 
|.u.B........nY..|
01dffe00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
051ba000


There is now a block of bad data near the middle of the file.


Therefore, you are right -- LUKS does not detect bit rot.


I'll leave the md and LVM exercises to people who actually use them (I 
use ZFS).


David

Back to linux.debian.user | Previous | Next | Find similar | Unroll thread

Thread

LUKS and bit rot David Christensen <dpchrist@holgerdanske.com> - 2020-08-21 10:40 +0200

csiph-web