This document attempts to provide step-by-step procedures that can be followed to:
• Set up software RAID in RHEL 6
• Migrate existing storage from Non-RAID to Software RAID
• Resize RAID volumes
• Recover from a broken RAID
This document uses RAID-1 as an example while working with Software RAID. The procedure can however be applied to other RAID types as applicable. Please consult the mdadm(8) man page for details on exact options for various RAID types.
We used a Dell PowerEdgeTM R510 server with a Dell PERC H200 storage controller on the system for this procedure. All the storage volumes were exported directly to the OS without using any controller hardware RAID features.
Note: Ensure that all data backed up before performing any of these procedures
Setting up Software RAID in RHEL 6
Setup during Installation
The RHEL 6 installer (anaconda) has functionality that enables the OS to be installed on a software RAID partition. This section describes the steps to install RHEL 6 on a RAID-1 partition.
1. Start the RHEL 6 installer and follow the on-screen installation instructions and select the “Custom” layout for installation.
2. Create a minimum of two partitions to create RAID-1 device type and set the File System Type as software raid.
3. Create a RAID-1 device from RAID members created in Step2, select the filesystem and RAID-1 level
Here is the minimum number of software RAID partitions required for each RAID level:
• RAID 0,1,10 – 2 partitions
• RAID 4,5 – 3 partitions
• RAID 6 – 4 partitions
4. After creating all the necessary partitions (/boot, /, swap, etc.) on RAID-1 volume, proceed with the installation.
5. Once the installation is completed, the OS will boot successfully from the partitions on the RAID volume.
Note: Ensure that the boot-loader is installed on the first disk and not on the RAID device.
Installing boot-loader on the RAID device may result in failure to boot the OS after installation.
Setup after Installation
Software RAID volumes can be created on a running system post install as well. Ensure that the partition/s on which the OS is installed are not used for creating software RAID partitions, failure to do that may results in re-installing OS on the system.
Following section describes steps to create RAID-1 partition on the system.
1. Create the raid-1 md device using the mdadm command with /dev/sdj1 and /dev/sdl1. sdj1 and sdl1 are un-used partitions on this system
Note: "--metadata=0.90" is only used when you create /boot partition, because CentOS 6.x (6.4) still does not support boot from default metadata 1.2
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdj1 /dev/sdl1 --metadata=0.90
mdadm: array /dev/md9 started.
2. Create ext4 filesystem layout on the md device and add array details mdadm –-detail –scan to /etc/mdadm.conf file. Mount the device /dev/md9 on the system to use it.
[root@Dell-PowerEdge-R510 ~]# mkfs.ext4 /dev/md9
[root@Dell-PowerEdge-R510 ~]# mdadm --detail --scan >> /etc/mdadm.conf
[root@Dell-PowerEdge-R510 ~]# mount /dev/md9 /data/
3. Add a new entry in /etc/fstab file to auto mount the md raid partition, whenever system reboots.
[root@Dell-PowerEdge-R510 ~]# cat /etc/fstab
/dev/md9 /data ext4 defaults 1 1
Migration of Storage from Non-RAID to RAID Configurations
It is possible to migrate to software raid, the “/” partition without having to re-install the operating system if you installed RHEL 6 OS without software raid volumes. This section explains how migration of storage from non-raid to raid configuration can be achieved.
At a high level, here is how it can be achieved:
• Prepare the new storage volume
• Update fstab and grub configuration to boot from newly created storage volume
• Sync the data from the old partitions to the new storage volume
• Install the boot loader on new storage volume
• Add the old partition volume to the md raid-1 set
Prepare the new storage volume
1. Create the partition layout on the /dev/sdj volume similar to /dev/sdh.
[root@Dell-PowerEdge-R510 ~]# sfdisk -d /dev/sdh | sfdisk --force /dev/sdj
Device Boot Start End #sectors Id System
/dev/sdj1 * 2048 20482047 20480000 83 Linux
/dev/sdj2 20482048 24578047 4096000 82 Linux swap / Solaris
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table
2. Set the partition id of /dev/sdj1 to Linux RAID
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdj 1 fd
Done
Create RAID-1 using mdadm utility:
1. Create the raid-1 md device using the mdadm command with /dev/sdj1. Mark the first volume as “missing”, which will be sdh volume, and it will be added later in the steps. Since sdh has OS installed, we have to add this to raid array after copying the contents from sdj to sdh drive.
Note: "--metadata=0.90" is only used when you create /boot partition, because CentOS 6.x (6.4) still does not support boot from default metadata 1.2
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 missing /dev/sdj1 --metadata=0.90
mdadm: array /dev/md9 started.
2. Create ext4 filesystem layout on the md device and add the array details mdadm –-detail –scan to /etc/mdadm.conf file.
[root@Dell-PowerEdge-R510 ~]# mkfs.ext4 /dev/md9
[root@Dell-PowerEdge-R510 ~]# mdadm --detail --scan >> /etc/mdadm.conf
Update fstab and grub configuration to boot from newly created storage volume:
1. Modify the /etc/fstab and /boot/grub/menu.lst with md device.
[root@Dell-PowerEdge-R510 ~]# blkid | grep –i md9
/dev/md9: UUID="016db049-6802-4369-bf66-bd48aad15395" TYPE="ext4"
[root@Dell-PowerEdge-R510 ~]# cat /etc/fstab
#UUID=38b56dff-c6d2-434f-bb48-25efb97f3a58 / 1 1 ext4 defaults
UUID=016db049-6802-4369-bf66-bd48aad15395 / 1 1 ext4 defaults
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/menu.lst
2. Add sdj to the device map entry to install the grub on the sdj device.
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/device.map
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
Sync the data from the old partitions to the new storage volume:
1. Since we are trying to replicate the contents from currently running partition. It is recommended that you execute the following steps in run level1
2. Mount the array volume and copy the contents from sdh1 to md9.
[root@Dell-PowerEdge-R510 ~]# mount /dev/md9 //
[root@Dell-PowerEdge-R510 ~]# rsync -aqxP / //
Install the boot loader on new storage volume:
1. Install the boot loader on the sdj device.
[root@Dell-PowerEdge-R510 ~]# grub-install /dev/sdj
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'.
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
2. Reboot the system and verify the system has booted with the md device using the mount command.
Add the old partition volume to the md raid-1 set:
1. Change the partition of sdh1 and add the sdh1 device to md9 array and allow the resync to complete from sdh1 to sdj1.
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdh 1 fd
Done
[root@Dell-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdh1
mdadm: added /dev/sdh1
[root@Dell-PowerEdge-R510 ~]# watch cat /proc/mdstat
2. Reboot the system to verify the md migration completed successfully.
3. Run the cat /proc /mdstat command to check the status of the running array.
Resizing an existing RAID Partition
The Linux software-RAID solution allows us to resize (increase or decrease) the RAID partition size. Following steps explain how to increase the size of existing software RAID partition (data and OS partitions).
Here is what is required:
• Prepare partitions of the new size desired
• Replace both RAID members with newly created partitions by breaking existing RAID
• Resize the RAID array
• If dealing with OS partitions, Prepare the new RAID volume to be bootable
Prepare partitions of the new size desired:
1. Initially create the md9 raid-1 level with a size approximately 100 GB comprised of both the sdh1 and sdj1 volumes. The md-0 array RAID set will be increased to approximately 200 GB by using the sdl1 and sdm1 volumes.
2. Create the new RAID partition of increased size on sdl and sdm. In this example, we created a new RAID partition of approximately 200GB in size on the sdl volume.
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdl 1 fd
Done
[root@Dell-PowerEdge-R510 ~]# fdisk -l /dev/sdl
Disk /dev/sdl: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4c0a9054
Device Boot Start End Blocks Id System
/dev/sdl1 1 25000 200812468+ fd Linux raid autodetect
Replace both RAID members with newly created partitions by breaking existing RAID:
1. Set the sdj1 volume to faulty and remove the volume from the RAID set.
[root@Dell-PowerEdge-R510 ~]# mdadm /dev/md9 –-fail /dev/sdj1 –remove /dev/sdj1
mdadm: set /dev/sdj1 faulty in /dev/md9
mdadm: hot removed /dev/sdj1 from /dev/md9
2. Add the new partition to the RAID set and allow the resync to complete on the new partition added to the RAID set. Run cat /proc/mdstat to show the status of resynchronization.
[root@Dell-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
mdadm: added /dev/sdl1
[root@Dell-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdl1[2] sdh1[0]
102399928 blocks super 1.0 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
unused devices:
3. Repeat the above steps adding the /dev/sdm1 partition. Remove the /dev/sdh1 partition from the RAID set. Allow the resynchronization to complete on the /dev/sdm1 partition.
[root@Dell-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdm1[3] sdl1[2]
102399928 blocks super 1.0 [2/1] [_U]
[>....................]
recovery =
finish=20.7min speed=81894K/sec
bitmap: 1/1 pages [4KB], 65536KB chunk
0.3%
(409472/102399928)
unused devices:
Resize the RAID array:
1. Set the /dev/md9 partition size to use the new volume partition size and allow the resync to complete.
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --size=max
mdadm: Cann[ot set device size for /dev/md9: Device or resource busy
Bitmap must be removed before size can be changed
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --bitmap none
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --size=max
mdadm: component size of /dev/md9 has been set to 200812396K
2. Resize the file system of the /dev/md9 partition to increase the file system size. Now the df –h command shows the increased md RAID set.
[root@Dell-PowerEdge-R510 ~]# resize2fs /dev/md9
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/md9 is mounted on /; on-line resizing required old desc_blocks = 7, new_desc_blocks = 12
Performing an on-line resize of /dev/md9 to 50203099 (4k) blocks.
The filesystem on /dev/md9 is now 50203099 blocks long.
[root@Dell-PowerEdge-R510 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md9 189G 1.4G 178G 1% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
If dealing with OS partitions, prepare the new RAID volume to be bootable:
Note: Following steps are required to boot the Linux system, if you are replacing the drive from RAID array which has boot loader and file system already installed.
Before resizing the RAID partition, ensure that the md-raid set has two active drives, sdh1 and sdj1 and that grub is installed in /dev/sdh. After the resizing operation, the md-raid set has been replaced with sdl1 and sdm1 volumes and boot loader has been installed on sdl volume.
1. Add the sdl and sdm entries in device.map file.
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/device.map
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
(hd2) /dev/sdl
(hd3) /dev/sdm
2. Install the grub boot loader on the /dev/sdl volume and then remove the older drives from the system.
[root@Dell-PowerEdge-R510 ~]# grub-install /dev/sdl
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map. Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'.
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
(hd2) /dev/sdl
(hd3) /dev/sdm
3. Reboot the system and enter the storage controller BIOS configuration.
4. Change the boot drive to new drive where the boot loader is now installed.
5. Save the configuration and restart the system to boot from new RAID set partition.
Recovery from a Broken RAID
In case of RAID failures on a system running Linux Software RAID (md) solution (for example, media failure or disk driver failure), it is possible to recover the system by any of these methods
• replacing the faulty disk
• adding a new disk
• using the spare disk
In Linux software raid, recovery is achieved through “failover” mechanisms. Failover mechanism ensures data protection by providing additional drives (spares) and can be automatic or manual.
Automatic Failover
Linux md-raid solution has an intelligent monitor mechanism to detect hardware failure in RAID arrays.
If any disk in the RAID array fails, the monitors sets the failed drive to faulty and starts using one of the available spare drives for regeneration. To check the status of the RAID array, look at /proc/mdstat file.
To replicate a raid failure to check how automatic failover scenario works, follow the steps.
1. Create a RAID-1 setup with three raid partition. Minimum two raid partitions are required to create RAID-1 device and third raid partition is used as a spare disk, will be used as a replacement if one of the active RAID partition fails.
2. Simply pull out one of the disks which are active in the raid array and check the status of the array using the cat /proc/mdstat command, will show the removed drive as faulty and use the spare drive as replacement for date re-generation.
Manual Failover
Faulty drives in the raid array can be replaced manually. Following steps discuss how to manually replace the faulty drive from the RAID-1 array:
1. Raid-1 setup is created with sdh1 and sdj1 partitions. Set the sdh1 drive to faulty.
[root@DELL-PowerEdge-R510 ~]# mdadm -f /dev/md9 /dev/sdh1
mdadm: set /dev/sdh1 faulty in /dev/md9
2. Remove the faulty drive sdh1 from the array.
[root@DELL-PowerEdge-R510 ~]# mdadm -r /dev/md9 /dev/sdh1
mdadm: hot removed /dev/sdh1 from /dev/md9
3. Replace the faulty drive with adding a new one to the array.
[root@DELL-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
Device Added
4. Look at status of the RAID array by executing the cat /proc/mdstat command, showing /dev/sdl1 added to the RAID array. Also check that the resynchronization is complete.
[root@DELL-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdl1[2] sdj1[0]
40958908 blocks super 1.1 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
Adding a Spare Disk to the Array
Spare disks provide additional protection to a raid configuration. If a disk fails in a raid array, the spare disk automatically replaces the failed drive; also the raid can be rebuilt automatically in the background. Spare drives can be added to the RAID array during the time of creation of the array or later.
Adding spare disk during raid array creation
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdj1 /dev/sdl1 --metadata=0.90 –-spare-devices=1 /dev/sdm1
mdadm: array /dev/md9 started.
Adding spare disk to an existing array.
[root@DELL-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
mdadm: added /dev/sdl1
[root@DELL-Poweredge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md8 : active RAID-1 sdl1[2](S) sdh1[0] sdj1[1] sdl
References
• http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/index.html
• http://www.spinics.net/lists/raid/
• https://raid.wiki.kernel.org/index.php/Linux_Raid
-------------------------------------------------------------------------------------------
Stopping resync-a:
echo "idle" > /sys/block/md0/md/sync_action
Rename:
Last time I did this it was from a later distribution --- either a RHEL-4 or FC-5 rescue CD, I cannot recall which at this point. I did it precisely for the reason you give, to move an existing /dev/md0 to another RHEL-4 box. But it wasn't hard, using mdadm. Knowing the devices belonging to the array (sda5 and sdb5 in this case), I just did:
# mdadm --stop /dev/md0
# mdadm -A /dev/md6 -m0 --update=super-minor /dev/sda5 /dev/sdb5
which stops the array as /dev/md0 and then reassembles it as /dev/md6.
The reassembly looks for devices which have an existing minor number of 0, not 6 (-m0), and then updates the minors in the superblocks to the new number.
I believe the same options are present in taroon's mdadm.
mdadm ---create --level=1 --raid-devices=2 /dev/sdh1 /dev/sdg1
(Repair filesystem) Writing not allowed. Solution:
mount -w -o remount /
Linux console scroll : Shift + PgDn ili Shift + PgUp
Creating Boot RAID1:
[root@kancelarija yum]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdd1 /dev/sde1 --metadata=0.90
Creating RAID10 far,2:
[root@kancelarija yum]# mdadm --create /dev/md9 --level=10 --layout=f2 --raid-disks=2 /dev/sdd9 /dev/sde9
6.3. Activate RAID Devices
The installation media doesn't include a default mdadm.conf file, so it must be created. An easy method is...
mdadm --examine --scan > /etc/mdadm.conf
Next, activate all discovered RAID devices with...
mdadm --assemble --scan
cat /proc/sys/dev/raid/speed_limit_max
200000
cat /proc/sys/dev/raid/speed_limit_min
1000
50MB/s:
echo 50000 >/proc/sys/dev/raid/speed_limit_min
• Set up software RAID in RHEL 6
• Migrate existing storage from Non-RAID to Software RAID
• Resize RAID volumes
• Recover from a broken RAID
This document uses RAID-1 as an example while working with Software RAID. The procedure can however be applied to other RAID types as applicable. Please consult the mdadm(8) man page for details on exact options for various RAID types.
We used a Dell PowerEdgeTM R510 server with a Dell PERC H200 storage controller on the system for this procedure. All the storage volumes were exported directly to the OS without using any controller hardware RAID features.
Note: Ensure that all data backed up before performing any of these procedures
Setting up Software RAID in RHEL 6
Setup during Installation
The RHEL 6 installer (anaconda) has functionality that enables the OS to be installed on a software RAID partition. This section describes the steps to install RHEL 6 on a RAID-1 partition.
1. Start the RHEL 6 installer and follow the on-screen installation instructions and select the “Custom” layout for installation.
2. Create a minimum of two partitions to create RAID-1 device type and set the File System Type as software raid.
3. Create a RAID-1 device from RAID members created in Step2, select the filesystem and RAID-1 level
Here is the minimum number of software RAID partitions required for each RAID level:
• RAID 0,1,10 – 2 partitions
• RAID 4,5 – 3 partitions
• RAID 6 – 4 partitions
4. After creating all the necessary partitions (/boot, /, swap, etc.) on RAID-1 volume, proceed with the installation.
5. Once the installation is completed, the OS will boot successfully from the partitions on the RAID volume.
Note: Ensure that the boot-loader is installed on the first disk and not on the RAID device.
Installing boot-loader on the RAID device may result in failure to boot the OS after installation.
Setup after Installation
Software RAID volumes can be created on a running system post install as well. Ensure that the partition/s on which the OS is installed are not used for creating software RAID partitions, failure to do that may results in re-installing OS on the system.
Following section describes steps to create RAID-1 partition on the system.
1. Create the raid-1 md device using the mdadm command with /dev/sdj1 and /dev/sdl1. sdj1 and sdl1 are un-used partitions on this system
Note: "--metadata=0.90" is only used when you create /boot partition, because CentOS 6.x (6.4) still does not support boot from default metadata 1.2
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdj1 /dev/sdl1 --metadata=0.90
mdadm: array /dev/md9 started.
2. Create ext4 filesystem layout on the md device and add array details mdadm –-detail –scan to /etc/mdadm.conf file. Mount the device /dev/md9 on the system to use it.
[root@Dell-PowerEdge-R510 ~]# mkfs.ext4 /dev/md9
[root@Dell-PowerEdge-R510 ~]# mdadm --detail --scan >> /etc/mdadm.conf
[root@Dell-PowerEdge-R510 ~]# mount /dev/md9 /data/
3. Add a new entry in /etc/fstab file to auto mount the md raid partition, whenever system reboots.
[root@Dell-PowerEdge-R510 ~]# cat /etc/fstab
/dev/md9 /data ext4 defaults 1 1
Migration of Storage from Non-RAID to RAID Configurations
It is possible to migrate to software raid, the “/” partition without having to re-install the operating system if you installed RHEL 6 OS without software raid volumes. This section explains how migration of storage from non-raid to raid configuration can be achieved.
At a high level, here is how it can be achieved:
• Prepare the new storage volume
• Update fstab and grub configuration to boot from newly created storage volume
• Sync the data from the old partitions to the new storage volume
• Install the boot loader on new storage volume
• Add the old partition volume to the md raid-1 set
Prepare the new storage volume
1. Create the partition layout on the /dev/sdj volume similar to /dev/sdh.
[root@Dell-PowerEdge-R510 ~]# sfdisk -d /dev/sdh | sfdisk --force /dev/sdj
Device Boot Start End #sectors Id System
/dev/sdj1 * 2048 20482047 20480000 83 Linux
/dev/sdj2 20482048 24578047 4096000 82 Linux swap / Solaris
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table
2. Set the partition id of /dev/sdj1 to Linux RAID
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdj 1 fd
Done
Create RAID-1 using mdadm utility:
1. Create the raid-1 md device using the mdadm command with /dev/sdj1. Mark the first volume as “missing”, which will be sdh volume, and it will be added later in the steps. Since sdh has OS installed, we have to add this to raid array after copying the contents from sdj to sdh drive.
Note: "--metadata=0.90" is only used when you create /boot partition, because CentOS 6.x (6.4) still does not support boot from default metadata 1.2
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 missing /dev/sdj1 --metadata=0.90
mdadm: array /dev/md9 started.
2. Create ext4 filesystem layout on the md device and add the array details mdadm –-detail –scan to /etc/mdadm.conf file.
[root@Dell-PowerEdge-R510 ~]# mkfs.ext4 /dev/md9
[root@Dell-PowerEdge-R510 ~]# mdadm --detail --scan >> /etc/mdadm.conf
Update fstab and grub configuration to boot from newly created storage volume:
1. Modify the /etc/fstab and /boot/grub/menu.lst with md device.
[root@Dell-PowerEdge-R510 ~]# blkid | grep –i md9
/dev/md9: UUID="016db049-6802-4369-bf66-bd48aad15395" TYPE="ext4"
[root@Dell-PowerEdge-R510 ~]# cat /etc/fstab
#UUID=38b56dff-c6d2-434f-bb48-25efb97f3a58 / 1 1 ext4 defaults
UUID=016db049-6802-4369-bf66-bd48aad15395 / 1 1 ext4 defaults
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/menu.lst
2. Add sdj to the device map entry to install the grub on the sdj device.
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/device.map
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
Sync the data from the old partitions to the new storage volume:
1. Since we are trying to replicate the contents from currently running partition. It is recommended that you execute the following steps in run level1
2. Mount the array volume and copy the contents from sdh1 to md9.
[root@Dell-PowerEdge-R510 ~]# mount /dev/md9 //
[root@Dell-PowerEdge-R510 ~]# rsync -aqxP / //
Install the boot loader on new storage volume:
1. Install the boot loader on the sdj device.
[root@Dell-PowerEdge-R510 ~]# grub-install /dev/sdj
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'.
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
2. Reboot the system and verify the system has booted with the md device using the mount command.
Add the old partition volume to the md raid-1 set:
1. Change the partition of sdh1 and add the sdh1 device to md9 array and allow the resync to complete from sdh1 to sdj1.
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdh 1 fd
Done
[root@Dell-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdh1
mdadm: added /dev/sdh1
[root@Dell-PowerEdge-R510 ~]# watch cat /proc/mdstat
2. Reboot the system to verify the md migration completed successfully.
3. Run the cat /proc /mdstat command to check the status of the running array.
Resizing an existing RAID Partition
The Linux software-RAID solution allows us to resize (increase or decrease) the RAID partition size. Following steps explain how to increase the size of existing software RAID partition (data and OS partitions).
Here is what is required:
• Prepare partitions of the new size desired
• Replace both RAID members with newly created partitions by breaking existing RAID
• Resize the RAID array
• If dealing with OS partitions, Prepare the new RAID volume to be bootable
Prepare partitions of the new size desired:
1. Initially create the md9 raid-1 level with a size approximately 100 GB comprised of both the sdh1 and sdj1 volumes. The md-0 array RAID set will be increased to approximately 200 GB by using the sdl1 and sdm1 volumes.
2. Create the new RAID partition of increased size on sdl and sdm. In this example, we created a new RAID partition of approximately 200GB in size on the sdl volume.
[root@Dell-PowerEdge-R510 ~]# sfdisk -c /dev/sdl 1 fd
Done
[root@Dell-PowerEdge-R510 ~]# fdisk -l /dev/sdl
Disk /dev/sdl: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4c0a9054
Device Boot Start End Blocks Id System
/dev/sdl1 1 25000 200812468+ fd Linux raid autodetect
Replace both RAID members with newly created partitions by breaking existing RAID:
1. Set the sdj1 volume to faulty and remove the volume from the RAID set.
[root@Dell-PowerEdge-R510 ~]# mdadm /dev/md9 –-fail /dev/sdj1 –remove /dev/sdj1
mdadm: set /dev/sdj1 faulty in /dev/md9
mdadm: hot removed /dev/sdj1 from /dev/md9
2. Add the new partition to the RAID set and allow the resync to complete on the new partition added to the RAID set. Run cat /proc/mdstat to show the status of resynchronization.
[root@Dell-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
mdadm: added /dev/sdl1
[root@Dell-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdl1[2] sdh1[0]
102399928 blocks super 1.0 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
unused devices:
3. Repeat the above steps adding the /dev/sdm1 partition. Remove the /dev/sdh1 partition from the RAID set. Allow the resynchronization to complete on the /dev/sdm1 partition.
[root@Dell-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdm1[3] sdl1[2]
102399928 blocks super 1.0 [2/1] [_U]
[>....................]
recovery =
finish=20.7min speed=81894K/sec
bitmap: 1/1 pages [4KB], 65536KB chunk
0.3%
(409472/102399928)
unused devices:
Resize the RAID array:
1. Set the /dev/md9 partition size to use the new volume partition size and allow the resync to complete.
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --size=max
mdadm: Cann[ot set device size for /dev/md9: Device or resource busy
Bitmap must be removed before size can be changed
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --bitmap none
[root@Dell-PowerEdge-R510 ~]# mdadm --grow /dev/md9 --size=max
mdadm: component size of /dev/md9 has been set to 200812396K
2. Resize the file system of the /dev/md9 partition to increase the file system size. Now the df –h command shows the increased md RAID set.
[root@Dell-PowerEdge-R510 ~]# resize2fs /dev/md9
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/md9 is mounted on /; on-line resizing required old desc_blocks = 7, new_desc_blocks = 12
Performing an on-line resize of /dev/md9 to 50203099 (4k) blocks.
The filesystem on /dev/md9 is now 50203099 blocks long.
[root@Dell-PowerEdge-R510 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md9 189G 1.4G 178G 1% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
If dealing with OS partitions, prepare the new RAID volume to be bootable:
Note: Following steps are required to boot the Linux system, if you are replacing the drive from RAID array which has boot loader and file system already installed.
Before resizing the RAID partition, ensure that the md-raid set has two active drives, sdh1 and sdj1 and that grub is installed in /dev/sdh. After the resizing operation, the md-raid set has been replaced with sdl1 and sdm1 volumes and boot loader has been installed on sdl volume.
1. Add the sdl and sdm entries in device.map file.
[root@Dell-PowerEdge-R510 ~]# cat /boot/grub/device.map
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
(hd2) /dev/sdl
(hd3) /dev/sdm
2. Install the grub boot loader on the /dev/sdl volume and then remove the older drives from the system.
[root@Dell-PowerEdge-R510 ~]# grub-install /dev/sdl
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map. Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'.
# this device map was generated by anaconda
(hd0) /dev/sdh
(hd1) /dev/sdj
(hd2) /dev/sdl
(hd3) /dev/sdm
3. Reboot the system and enter the storage controller BIOS configuration.
4. Change the boot drive to new drive where the boot loader is now installed.
5. Save the configuration and restart the system to boot from new RAID set partition.
Recovery from a Broken RAID
In case of RAID failures on a system running Linux Software RAID (md) solution (for example, media failure or disk driver failure), it is possible to recover the system by any of these methods
• replacing the faulty disk
• adding a new disk
• using the spare disk
In Linux software raid, recovery is achieved through “failover” mechanisms. Failover mechanism ensures data protection by providing additional drives (spares) and can be automatic or manual.
Automatic Failover
Linux md-raid solution has an intelligent monitor mechanism to detect hardware failure in RAID arrays.
If any disk in the RAID array fails, the monitors sets the failed drive to faulty and starts using one of the available spare drives for regeneration. To check the status of the RAID array, look at /proc/mdstat file.
To replicate a raid failure to check how automatic failover scenario works, follow the steps.
1. Create a RAID-1 setup with three raid partition. Minimum two raid partitions are required to create RAID-1 device and third raid partition is used as a spare disk, will be used as a replacement if one of the active RAID partition fails.
2. Simply pull out one of the disks which are active in the raid array and check the status of the array using the cat /proc/mdstat command, will show the removed drive as faulty and use the spare drive as replacement for date re-generation.
Manual Failover
Faulty drives in the raid array can be replaced manually. Following steps discuss how to manually replace the faulty drive from the RAID-1 array:
1. Raid-1 setup is created with sdh1 and sdj1 partitions. Set the sdh1 drive to faulty.
[root@DELL-PowerEdge-R510 ~]# mdadm -f /dev/md9 /dev/sdh1
mdadm: set /dev/sdh1 faulty in /dev/md9
2. Remove the faulty drive sdh1 from the array.
[root@DELL-PowerEdge-R510 ~]# mdadm -r /dev/md9 /dev/sdh1
mdadm: hot removed /dev/sdh1 from /dev/md9
3. Replace the faulty drive with adding a new one to the array.
[root@DELL-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
Device Added
4. Look at status of the RAID array by executing the cat /proc/mdstat command, showing /dev/sdl1 added to the RAID array. Also check that the resynchronization is complete.
[root@DELL-PowerEdge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md9 : active RAID-1 sdl1[2] sdj1[0]
40958908 blocks super 1.1 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
Adding a Spare Disk to the Array
Spare disks provide additional protection to a raid configuration. If a disk fails in a raid array, the spare disk automatically replaces the failed drive; also the raid can be rebuilt automatically in the background. Spare drives can be added to the RAID array during the time of creation of the array or later.
Adding spare disk during raid array creation
[root@Dell-PowerEdge-R510 ~]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdj1 /dev/sdl1 --metadata=0.90 –-spare-devices=1 /dev/sdm1
mdadm: array /dev/md9 started.
Adding spare disk to an existing array.
[root@DELL-PowerEdge-R510 ~]# mdadm --add /dev/md9 /dev/sdl1
mdadm: added /dev/sdl1
[root@DELL-Poweredge-R510 ~]# cat /proc/mdstat
Personalities : [RAID-1]
md8 : active RAID-1 sdl1[2](S) sdh1[0] sdj1[1] sdl
References
• http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/index.html
• http://www.spinics.net/lists/raid/
• https://raid.wiki.kernel.org/index.php/Linux_Raid
-------------------------------------------------------------------------------------------
Stopping resync-a:
echo "idle" > /sys/block/md0/md/sync_action
Rename:
Last time I did this it was from a later distribution --- either a RHEL-4 or FC-5 rescue CD, I cannot recall which at this point. I did it precisely for the reason you give, to move an existing /dev/md0 to another RHEL-4 box. But it wasn't hard, using mdadm. Knowing the devices belonging to the array (sda5 and sdb5 in this case), I just did:
# mdadm --stop /dev/md0
# mdadm -A /dev/md6 -m0 --update=super-minor /dev/sda5 /dev/sdb5
which stops the array as /dev/md0 and then reassembles it as /dev/md6.
The reassembly looks for devices which have an existing minor number of 0, not 6 (-m0), and then updates the minors in the superblocks to the new number.
I believe the same options are present in taroon's mdadm.
mdadm ---create --level=1 --raid-devices=2 /dev/sdh1 /dev/sdg1
(Repair filesystem) Writing not allowed. Solution:
mount -w -o remount /
Linux console scroll : Shift + PgDn ili Shift + PgUp
Creating Boot RAID1:
[root@kancelarija yum]# mdadm --create /dev/md9 --level=1 --raid-disks=2 /dev/sdd1 /dev/sde1 --metadata=0.90
Creating RAID10 far,2:
[root@kancelarija yum]# mdadm --create /dev/md9 --level=10 --layout=f2 --raid-disks=2 /dev/sdd9 /dev/sde9
6.3. Activate RAID Devices
The installation media doesn't include a default mdadm.conf file, so it must be created. An easy method is...
mdadm --examine --scan > /etc/mdadm.conf
Next, activate all discovered RAID devices with...
mdadm --assemble --scan
cat /proc/sys/dev/raid/speed_limit_max
200000
cat /proc/sys/dev/raid/speed_limit_min
1000
50MB/s:
echo 50000 >/proc/sys/dev/raid/speed_limit_min
No comments:
Post a Comment