Meeting: 10th September 2012 – Linux RAID


  • Topic: Home Server Part 1 – Linux RAID, NFS & Samba
  • Host: Tim Gibbon

Protecting your data using RAID1 on a root filesystem and sharing it using NFS and Samba

RAID introduction: http://www.prepressure.com/library/technology/raid

The aim of this talk is to demonstrate Software RAID 1 (mdadm) under GNU/Linux and how easy protecting important data can be. It also does something potentially risky, using Software RAID1 for the Root filesystem post installation. Here be dragons.

ideal_raid_setupnot_ideal_raid_setup

I would always recommend using a 3 disk setup – buy a 30 UKP drive for the root filesystem and then use Software RAID 1 to protect the 2 (bigger) data disks. All the grub problems then go away, you don’t have to worry about software upgrades/grub re-installs impacting the data and you can change O/S with minimal impact. If you absolutely need to do this, then I would advise you setting up RAID on install. You will still have the usual problems with grub updates etc, but at least you’ll have some support when you realise it is borked again.

The Bibliography links to the mdadm cheat sheet which explains how to build a data volume without using the root filesystem (Ideal RAID setup picture above). The rest of this talk deals with a Non ideal RAID1 setup.

A question was asked about performance – see below:

http://www.prepressure.com/library/technology/raid

In RAID1: Read performance will increase (the more disks in the mirror, the greater the performance). Write performance will decrease slightly – should be comparable to a normal disk.

Software RAID1 for the Root Filesystem

You have backups?

We are booting off a pre-installed Debian. 1GB O/S in /dev/sda1. We first of all setup a RAID array on /dev/sdb saying there is a missing disk, we copy the O/S over onto /dev/sdb, we boot off it. We then add /dev/sda back into the RAID array.

First we install the software:

root@debian:/# apt-get install mdadm rsync initramfs-tools
root@debian:/# fdisk /dev/sda
n for new partition
p for primary
1 for the number
t for type
L to list codes
fd (Linux RAID autodetect)
w to write
q to quit

Copy the partition table from /dev/sda to /dev/sdb. Change the

root@debian:/# sfdisk -d /dev/sda | sfdisk /dev/sdb

Now setup the first mdadm device (/dev/md0) using /dev/sdb1 and saying there is a missing disk (/dev/sda1).

root@debian:/# mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.9
0 missing /dev/sdb1

Make a filesystem

root@debian:/# mkfs.ext4 /dev/md0

Make sure mdadm knows where to look for it’s disks when the system is booting

root@debian:/# vi /etc/mdadm/mdadm.conf

Add to line starting DEVICE

DEVICE /dev/sda* /dev/sdb*

Append the mdadm configuration to the bottom of that file

root@debian:/# mdadm --detail --scan >>/etc/mdadm/mdadm.conf

We can set lots of monitoring preferences at this stage

root@debian:/# dpkg-reconfigure mdadm
root@debian:/# cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdb1[1]
      1023936 blocks [2/1] [_U]

unused devices: <none>

Reinstall grub. Here be dragons. This is why the metadata needs to be 0.90

root@debian:/# dpkg-reconfigure grub-pc
Installation finished. No error reported.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-3-686-pae
Found initrd image: /boot/initrd.img-3.2.0-3-686-pae
done

Time to copy our root filesystem data (from /dev/sda to /dev/sdb to be able to boot off /dev/sdb).

root@debian:/# mount /dev/md0 /mnt
root@debian:/# rsync -auHxv --exclude=/proc/* --exclude=/mnt --exclude=/tmp/* --exclude=/sys/* /* /mnt/

We need to modify fstab on /dev/sdb to say that we are not booting off the /dev/sda disk, but are booting off an mdadm device.

root@debian:/# vi /mnt/etc/fstab

FROM:

UUID=55676119-82a0-400b-a0dc-3b3f28e20ec7	/	ext4    errors=remount-r
o 0	1

TO:

/dev/md0 					/  	ext4	errors=remount-r
o 0	1
root@debian:/# shutdown -r now

Fingers crossed.


Add the grub prompt, type e to edit

		set root='(md/0)' 
		linux /boot/vmlinuz-3.2.0-3-686-pae root=/dev/md0 ro

Check that all is well with our world and that we have booted off /dev/sdb (/dev/md0)

root@debian:/home/bobbins# mount | grep md0
/dev/md0 on / type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data
=ordered)

Add /dev/sda back into the RAID array

root@debian:/home/bobbins# mdadm /dev/md0 --add /dev/sda1
mdadm: added /dev/sda1
root@debian:/home/bobbins# watch cat /proc/mdstat

You wait – time passes.

Personalities : [raid1] 
md0 : active raid1 sda1[0] sdb1[1]
      1023936 blocks [2/2] [UU]

unused devices: <none>

Update grub, hoping for the best

root@debian:/home/bobbins# update-grub

error: disk missing.
error: disk missing.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-3-686-pae
Found initrd image: /boot/initrd.img-3.2.0-3-686-pae
Found Debian GNU/Linux (wheezy/sid) on /dev/sda1
Found Debian GNU/Linux (wheezy/sid) on /dev/sdb1
done

Reboot to check all is well with the world

root@debian:/home/bobbins# shutdown -r now

DISK PULLING TEST

To check it works. After pulling disk and reinserting

Edit /etc/default/grub:

Uncomment

GRUB_DISABLE_LINUX_UUID=true
root@debian:/home/bobbins# update-grub
root@debian:/home/bobbins# dpkg-reconfigure grub-pc

Add the failed disk back in to sync it.

root@debian:/home/bobbins# mdadm /dev/md0 --add /dev/sda1

Make a directory that we will be using for our data:

root@debian:/home/bobbins# mkdir /data_md0

Sharing data using Samba

Install samba:

root@debian:/home/bobbins# apt-get install samba

Configure it:

root@debian:/home/bobbins# cd /etc/samba

root@debian:/home/bobbins# cp smb.conf smb.conf.10Sep2012

Edit smb.conf adding the following at the bottom:

[data_md0]
   comment = RAID1 Datastore
   path = /data_md0
   read only = no
   public = yes
;   hosts allow = 192.168.1.1/24

For more security, the hosts allow can be tied down to a subnet. It is currently commented in the code above. See smb.conf for more examples.

Restart samba

root@debian:/home/bobbins# /etc/init.d/samba restart

Sharing data using NFS

root@debian:/home/bobbins# apt-get install nfs-kernel-server

root@debian:/home/bobbins# cd /etc/

root@debian:/home/bobbins# cp exports.conf exports.conf.10Sep2012

Edit exports adding the following at the bottom:

/data_md0  *(rw,sync,no_subtree_check,insecure)
rw = read/write. (Use ro for read only mounts).
sync = Reply to requests only after the changes have been committed to stable storage.
no_subtree_check = 
              If  a  subdirectory  of  a filesystem is exported, but the whole
              filesystem isn't then whenever a NFS request arrives, the server
              must check not only that the accessed file is in the appropriate
              filesystem (which is easy) but also that it is in  the  exported
              tree (which is harder). This check is called the subtree_check.
insecure = Required for Mac OS X. Port number used < 1024. This is insecure as it makes it easier to sniff the traffic. Don't use this on non-secured networks.

or more securely:

/data_md0 192.168.1.0/24(rw,sync,no_subtree_check,insecure)

Re/start NFS

root@debian:/home/bobbins# /etc/init.d/nfs-kernel-server restart

Check all is well

root@debian:/home/bobbins# showmount -e localhost

Bibliography:

Leave a comment