Softraid on OpenBSD

Page content

Softraid

Inspired by a book from MWL - OpenBSD Mastery Filesystems, here some Notes ..

Target

build a RAID with 3 Disks, add some Data, destroy one Disk, and rebuild the Raid (and it’s Data).

Requirements

  • OpenBSD 7.2 Running
  • added 3 Disk with 20G each: sd0, sd1, sd2

Find Disks

root@puffy # dmesg |grep -i sec

wd0: 64-sector PIO, LBA, 20480MB, 41943040 sectors
sd0: 20480MB, 512 bytes/sector, 41943040 sectors
sd1: 20480MB, 512 bytes/sector, 41943040 sectors
sd2: 20480MB, 512 bytes/sector, 41943040 sectors

sd0, sd1, sd2 are New Disks for RAID

Write GPT Table

fdisk -gy sd0
fdisk -gy sd1
fdisk -gy sd2

Set Filesystem to Raid

root@puffy # disklabel -E sd0

Label editor (enter '?' for help at any prompt)
sd0> a
partition: [a]
offset: [64]
size: [41942943]
FS type: [4.2BSD] raid
sd0*> w
sd0> q
No label changes.

Dump Disklabel, apply the Label to other Disks

if your disks have different sizes, you need to find the smallest one and apply that schema to the other disks

disklabel sd0 > disklabel.sd0.softraid
disklabel -R sd1 disklabel.sd0.softraid
disklabel -R sd2 disklabel.sd0.softraid

Create Raid5

root@puffy # bioctl -c 5 -l sd0a,sd1a,sd2a softraid0

softraid0: RAID 5 volume attached as sd3

Zeroize Raid

root@puffy # dd if=/dev/zero bs=1m count=1 of=/dev/sd3c

1+0 records in
1+0 records out
1048576 bytes transferred in 0.068 secs (15227879 bytes/sec)

make Partition on Raid

root@puffy # fdisk -gy sd3
Writing GPT.
# disklabel -E sd3

Label editor (enter '?' for help at any prompt)
sd3> a
partition: [a]
offset: [64]
size: [83884703]
FS type: [4.2BSD]
sd3*> w
sd3> q
No label changes.

Build new FileSystem

root@puffy # newfs sd3a
/dev/rsd3a: 40959.3MB in 83884672 sectors of 512 bytes
203 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
super-block backups (for fsck -b #) at:
 160, 414880, 829600, 1244320, 1659040, 2073760, 2488480, 2903200, 3317920, 3732640, 4147360, 4562080, 4976800, 5391520,
 5806240, 6220960, 6635680, 7050400, 7465120, 7879840, 8294560, 8709280, 9124000, 9538720, 9953440, 10368160, 10782880,
 11197600, 11612320, 12027040, 12441760, 12856480, 13271200, 13685920, 14100640, 14515360, 14930080, 15344800, 15759520,
 16174240, 16588960, 17003680, 17418400, 17833120, 18247840, 18662560, 19077280, 19492000, 19906720, 20321440, 20736160,
 21150880, 21565600, 21980320, 22395040, 22809760, 23224480, 23639200, 24053920, 24468640, 24883360, 25298080, 25712800,
 26127520, 26542240, 26956960, 27371680, 27786400, 28201120, 28615840, 29030560, 29445280, 29860000, 30274720, 30689440,
 31104160, 31518880, 31933600, 32348320, 32763040, 33177760, 33592480, 34007200, 34421920, 34836640, 35251360, 35666080,
 36080800, 36495520, 36910240, 37324960, 37739680, 38154400, 38569120, 38983840, 39398560, 39813280, 40228000, 40642720,
 41057440, 41472160, 41886880, 42301600, 42716320, 43131040, 43545760, 43960480, 44375200, 44789920, 45204640, 45619360,
 46034080, 46448800, 46863520, 47278240, 47692960, 48107680, 48522400, 48937120, 49351840, 49766560, 50181280, 50596000,
 51010720, 51425440, 51840160, 52254880, 52669600, 53084320, 53499040, 53913760, 54328480, 54743200, 55157920, 55572640,
 55987360, 56402080, 56816800, 57231520, 57646240, 58060960, 58475680, 58890400, 59305120, 59719840, 60134560, 60549280,
 60964000, 61378720, 61793440, 62208160, 62622880, 63037600, 63452320, 63867040, 64281760, 64696480, 65111200, 65525920,
 65940640, 66355360, 66770080, 67184800, 67599520, 68014240, 68428960, 68843680, 69258400, 69673120, 70087840, 70502560,
 70917280, 71332000, 71746720, 72161440, 72576160, 72990880, 73405600, 73820320, 74235040, 74649760, 75064480, 75479200,
 75893920, 76308640, 76723360, 77138080, 77552800, 77967520, 78382240, 78796960, 79211680, 79626400, 80041120, 80455840,
 80870560, 81285280, 81700000, 82114720, 82529440, 82944160, 83358880, 83773600,

Mount Raid

root@puffy # mkdir /raid
root@puffy # mount /dev/sd3a /raid

make mount persistent

echo "/dev/$(bioctl softraid0 |awk '/softraid0/{ print $5 }')a /raid ffs rw,softdep,noatime,nodev,nosuid 1 2" >> /etc/fstab

df

root@puffy # df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/wd0a      600M    224M    346M    39%    /
/dev/wd0l      3.7G   63.7M    3.4G     2%    /home
/dev/wd0d      848M    1.7M    804M     0%    /tmp
/dev/wd0f      2.3G    1.2G    1.0G    54%    /usr
/dev/wd0g      643M    271M    339M    44%    /usr/X11R6
/dev/wd0h      2.3G    1.7G    493M    78%    /usr/local
/dev/wd0k      5.2G    2.0K    4.9G     0%    /usr/obj
/dev/wd0j      1.6G    2.0K    1.5G     0%    /usr/src
/dev/wd0e      1.2G   49.8M    1.1G     4%    /var
/dev/sd3a     38.7G    2.0K   36.8G     0%    /raid

Create Data

root@puffy # tar czf /raid/all.tar.gz / &

root@puffy # df -h /raid/;
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd3a     38.7G    2.2G   34.6G     6%    /raid

root@puffy# ls -al /raid/
drwxr-xr-x   2 root  wheel         512 Feb  4 11:18 .
drwxr-xr-x  14 root  wheel         512 Feb  4 11:18 ..
-rw-r--r--   1 root  wheel  2374055353 Feb  4 11:22 all.tar.gz

Raid Status

root@puffy 130 # bioctl softraid0
Volume      Status               Size Device
softraid0 0 Online        42949017600 sd3     RAID5
          0 Online        21474516480 0:0.0   noencl <sd0a>
          1 Online        21474516480 0:1.0   noencl <sd1a>
          2 Online        21474516480 0:2.0   noencl <sd2a>

let’s reboot, remove one disk and replace it with an empty, bigger one …

New Disk

root@puffy # dmesg |grep -i sect
wd0: 64-sector PIO, LBA, 20480MB, 41943040 sectors
sd0: 20480MB, 512 bytes/sector, 41943040 sectors
sd1: 20480MB, 512 bytes/sector, 41943040 sectors
sd2: 30720MB, 512 bytes/sector, 62914560 sectors
sd3: 40959MB, 512 bytes/sector, 83884800 sectors

Check Raid Status

root@puffy # bioctl softraid0
Volume      Status               Size Device
softraid0 0 Degraded      42949017600 sd3     RAID5
          0 Online        21474516480 0:0.0   noencl <sd0a>
          1 Offline                 0 0:1.0   noencl <>
          2 Online        21474516480 0:2.0   noencl <sd1a>

Data still Safe ?

root@puffy # ll /raid/
total 4639136
-rw-r--r--  1 root  wheel  2374055353 Feb  4 11:22 all.tar.gz

Rebuild Raid

root@puffy # bioctl -R /dev/sd2a sd3
softraid0: sd2a partition too large, wasting 10737418240 bytes
softraid0: rebuild of sd3 started on sd2a

Rebuild Progress

root@puffy # bioctl sd3
Volume      Status               Size Device
softraid0 0 Rebuild       42949017600 sd3     RAID5 10% done
          0 Online        21474516480 0:0.0   noencl <sd0a>
          1 Rebuild       32211934720 0:1.0   noencl <sd2a>
          2 Online        21474516480 0:2.0   noencl <sd1a>

seems working fine !

Problems

  • created more than one Bluescreen while reading and writing to the RAID

  • added 4.th Disk to VM -> RAID Volume (softraid0 -> sd3) got renamed to sd4, while the new Disk got inserted as sd3

-> RAID was broken until i remove the new Disk and did a reboot :(

softraid0: volume sd4 is roaming, it used to be sd3, updating metadata

Any Comments ?

sha256: d69126086aa0db837e03b8ad8b02f2e54a1642a265dc5d4d1f2ddc852b728d76