As the title of the post states I have recently decommissioned my Open Solaris ZFS based file server. I have now combined my file server with my workstation. There are several reasons why I have done this:
As I run Arch Linux on my workstation and did not want to resort to using ZFS through fuse I had to say good bye to ZFS. I will definitely miss ZFS. Managing storage is really a no brainer with ZFS compared to setting up a comparable Linux based layered software raid > LVM > file systems type setup. I salvaged the disks from my file server, after backing up the data, and along with the disk from my workstation and a spare I had on hand, this gave me 4 x 250G drives. I partitioned each drive with a 128M boot partition, 2G swap partition and a 248G main partition.
[mike@mercury|~] $ sudo parted /dev/sda print
Model: ATA ST3250820AS (scsi)
Disk /dev/sda: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 132MB 132MB primary ext2 boot, raid
2 132MB 2130MB 1999MB primary linux-swap(v1)
3 2130MB 250GB 248GB primary raid
The four boot partitions were combined into a RAID1 array while the 4 large partitions were combined into a RAID10 array.
[mike@mercury|~] $ cat /proc/mdstat
Personalities : [raid1] [raid10]
md1 : active raid10 sda3 sdd3 sdc3 sdb3
484231040 blocks 64K chunks 2 near-copies [4/4] [UUUU]
md0 : active raid1 sda1 sdd1 sdc1 sdb1
128384 blocks [4/4] [UUUU]
unused devices: <none>
I was tossing up between RAID5 or RAID10 and finally came down on the side of RAID10. With RAID10 you sacrifice a drive, storage wise, though you do get the benefit of no parity calculations and no RAID5 write hole. Having said that I'm not sure how much of an issue the parity calculations would have been after seeing ZFS perform fine on some very modest hardware. Linux can perform RAID10 as a single layer which gives you a few more options when setting things up and also allows you to do things like setup a RAID10 array on 3 drives.
On top of the RAID10 array I setup LVM with a separate home logical volume. The final file system layout looks like:
[mike@mercury|~] $ pydf / /boot /home
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg0-root 63G 24G 36G 38.3 [#########...............] /
/dev/md0 121M 16M 99M 13.2 [###.....................] /boot
/dev/mapper/vg0-home 350G 176G 173G 50.4 [############............] /home
I chose the following file systems:
[mike@mercury|~] $ mount | grep 'root\|home\|boot' | cut -d ' ' -f 1,3,5 | column -t
/dev/mapper/vg0-root / ext4
/dev/md0 /boot ext2
/dev/mapper/vg0-home /home xfs
There was no real reasoning behind the selection of ext4 and xfs for root and home respectively. I have used both before and found them to be solid. I left around 50G spare in my volume group which I will probably end up adding to my home file system. Xfs will make this easy as it allows online resizing.
I didn't want to have to reinstall Arch Linux so I used my usual technique of transfering a Linux install using the Arch Linux live CD:
I wanted things to go smoothly so before I backed up my root files system I made sure I had added mdadm to my mkinitcpio hooks array, I already had lvm in there, and recreated my initrd. I also changed my /etc/fstab and /boot/grub/menu.lst to reflect the new filesystem layout. The last step was to chroot into my Arch Linux install and reinstall GRUB and generate a /etc/mdadm.conf. I hadn't realised I needed a /etc/mdadm.conf until trying to boot. I had been roughly following the Arch Linux wiki article Installing with Software RAID or LVM and it set me straight. It also included a neat trick with sfdisk to make partitioning multiple disks easy.
All in all it went well. I managed to get the whole thing done during the Oscar's broadcast. I did catch bits and pieces though. Steve Martin and Alec Baldwin did a fantastic job hosting and it was great to see an underdog get up in the form of Hurt Locker.
Next I need to work out a backup system. I would really like something similar to the rsync/ZFS snapshot system I used in Open Solaris. I will probably end up going with something similar using rsync and hard links or I might try btrfs and rsync. Btrfs seems to be coming along and it may be stable enough for use as a backup file system.