Wednesday 3 February 2010

Data migration using EMC OpenMigrator

Few weeks ago I had to migrate some data to new storage array. As EMC released some software for free I decided to use one of them - OpenMigrator. You can find a step by step procedure to perform such migration. I hope it helps.


----------------------------------------


Test performed on centos 5.4 x86_64 running on VM.
Let's install OpenMigrator:

[root@centos54 OpenMigrator]# ls
EMComom.LINUX_3.12.tar
[root@centos54 OpenMigrator]# pwd
/root/OpenMigrator
[root@centos54 OpenMigrator]# tar xf EMComom.LINUX_3.12.tar
[root@centos54 OpenMigrator]# ls
EMComom.LINUX_3.12.tar EMCom-V3.12.0.7-sles-10-ga-i386.rpm
EMCom-V3.12.0.7-rhel-3-i386-as.rpm EMCom-V3.12.0.7-sles-10-ga-x86_64.rpm
EMCom-V3.12.0.7-rhel-3-x86_64-as.rpm EMCom-V3.12.0.7-sles-10-sp1-i386.rpm
EMCom-V3.12.0.7-rhel-4-i386-as.rpm EMCom-V3.12.0.7-sles-10-sp1-x86_64.rpm
EMCom-V3.12.0.7-rhel-4-x86_64-as.rpm EMCom-V3.12.0.7-sles-10-sp2-i386.rpm
EMCom-V3.12.0.7-rhel-5-i386-as.rpm EMCom-V3.12.0.7-sles-10-sp2-s390x.rpm
EMCom-V3.12.0.7-rhel-5-s390x-as.rpm EMCom-V3.12.0.7-sles-10-sp2-x86_64.rpm
EMCom-V3.12.0.7-rhel-5-x86_64-as.rpm
[root@centos54 OpenMigrator]# rpm -ivh EMCom-V3.12.0.7-rhel-5-x86_64-as.rpm
Preparing... ########################################### [100%]
--------------------------------------------------------

EMC Open Migrator/LM V3.12.0.7 Linux rhel-5-x86_64-as x86_64

preinstall...

Target, host machine type check ...

target machine is type x86_64, host machine is type x86_64.

target, host machine types agree, continuing installation

...architecture check done.

Target, host version check...

target os version is rhel 5, host os version is rhel 5

target, host os versions agree, continuing installation

...version check done.

Checking for previously installed package versions...

no previously installed version

...installed check done

preinstall done.
1:EMCom ########################################### [100%]
--------------------------------------------------------
post-install...


COPYRIGHT (C) EMC CORPORATION, 1997-2009. ALL
RIGHTS RESERVED. UNPUBLISHED RIGHTS RESERVED UNDER
THE COPYRIGHT LAWS OF THE UNITED STATES.

THE SOFTWARE CONTAINED ON THIS MEDIA IS PROPRIETARY
TO AND EMBODIES THE CONFIDENTIAL TECHNOLOGY OF EMC
CORPORATION. POSSESSION, USE, DUPLICATION OR
DISSEMINATION OF THE SOFTWARE AND MEDIA IS
AUTHORIZED ONLY PURSUANT TO A VALID WRITTEN LICENSE
FROM EMC CORPORATION.

RESTRICTED RIGHTS LEGEND USE, DUPLICATION, OR
DISCLOSURE BY THE U.S. GOVERNMENT IS SUBJECT TO
RESTRICTIONS AS SET FORTH IN SUBPARAGRAPH (C)(1)(II)
OF DFARS 252.227-7013, OR IN FAR 52.227-19, AS
APPLICABLE.


initializing ...
creating EMCom records:
created.
initialized.

installing kernel module...
creating symbolic link /lib/modules/2.6.18-164.6.1.el5/kernel/drivers/emc/emcom.ko ...
created.
loading emcom kernel module ...
loaded.
creating default statefile ...
created.
installed.

installing symlinks...
creating symbolic link /usr/sbin/stormigrate
created.
creating symbolic link /usr/share/man/man8/stormigrate.8.gz
created.
creating symbolic link /usr/share/man/man4/emcom.4.gz
created.
installed.

installing init rc run level scripts...
making run state 1 rc script
making shutdown scripts
installed.

done.
[root@centos54 OpenMigrator]#
[root@centos54 OpenMigrator]# rpm -qa |grep -i emc
EMCom-V3.12.0.7-1
[root@centos54 OpenMigrator]# rpm -qil EMCom-V3.12.0.7-1
Name : EMCom Relocations: (not relocatable)
Version : V3.12.0.7 Vendor: EMC
Release : 1 Build Date: Sat 26 Sep 2009 02:53:12 AM CEST
Install Date: Wed 02 Dec 2009 09:56:41 PM CET Build Host: rhel5-x64-pkg
Group : Utilities/System Source RPM: EMCom-V3.12.0.7-1.src.rpm
Size : 5795634 License: Commercial
Signature : (none)
Summary : EMC Open Migrator/LM
Description :
EMC Open Migrator/LM
/etc/opt/EMCom
/etc/opt/EMCom/bin
/etc/opt/EMCom/bin/omx
/etc/opt/EMCom/bin/stormigrate
/etc/opt/EMCom/driver
/etc/opt/EMCom/driver/emcom
/etc/opt/EMCom/driver/emcom.conf
/etc/opt/EMCom/log
/etc/opt/EMCom/man
/etc/opt/EMCom/man/man4
/etc/opt/EMCom/man/man4/emcom.4.gz
/etc/opt/EMCom/man/man8
/etc/opt/EMCom/man/man8/stormigrate.8.gz
/etc/opt/EMCom/properties
/etc/opt/EMCom/scripts
/etc/opt/EMCom/scripts/addmod.sh
/etc/opt/EMCom/scripts/addrcinit.sh
/etc/opt/EMCom/scripts/addsymlink.sh
/etc/opt/EMCom/scripts/copyright.emc
/etc/opt/EMCom/scripts/initdb.sh
/etc/opt/EMCom/scripts/remmod.sh
/etc/opt/EMCom/scripts/remrcinit.sh
/etc/opt/EMCom/scripts/remsymlink.sh
/var/EMCom
/var/EMCom/lic
/var/EMCom/log
[root@centos54 OpenMigrator]#

Let's create LVM group (/dev/sda) and filesystem on it:

[root@centos54 OpenMigrator]# pvs
PV VG Fmt Attr PSize PFree
/dev/hda2 rootvg lvm2 a- 7.89G 20.00M
/dev/sda lvm2 -- 2.00G 2.00G
/dev/sdb lvm2 -- 2.00G 2.00G
[root@centos54 OpenMigrator]# vgcreate testvg /dev/sda
Volume group "testvg" successfully created
[root@centos54 OpenMigrator]# lvcreate -L 500M -n test1lv testvg
Logical volume "testlv" created
[root@centos54 OpenMigrator]# mkfs.ext3 /dev/testvg/test1lv
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@centos54 OpenMigrator]# mkdir /test1_migr
[root@centos54 OpenMigrator]# mount /dev/testvg/test1lv /test1_migr
[root@centos54 OpenMigrator]# df -k /test1_migr
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/testvg-testlv
495844 10544 459700 3% /test_migr

Let' put some files to new filesystem.

[root@centos54 OpenMigrator]# rsync -a /usr/lib/* /test1_migr/
[root@centos54 OpenMigrator]# df -k /test1_migr/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/testvg-testlv
495844 453553 16691 97% /test1_migr
[root@centos54 OpenMigrator]#

Let's create another lvm volume and create filesystem:

....
[root@centos54 OpenMigrator]# df -k /test2_migr
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/testvg-test2lv
495844 10544 459700 3% /test2_migr

Current configuration is as follows:

[root@centos54 OpenMigrator]# pvs
PV VG Fmt Attr PSize PFree
/dev/hda2 rootvg lvm2 a- 7.89G 20.00M
/dev/sda testvg lvm2 a- 2.00G 1.02G
/dev/sdb lvm2 -- 2.00G 2.00G
[root@centos54 OpenMigrator]# lvs testvg
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv testvg -wi-a- 500.00M
test2lv testvg -wi-ao 500.00M

It is recommended to migrate LVM volumes instead of disks. Whole lun migration can occur in PVS duplication and in this case we migrate two volumes of 500MB each instead of one 2GB lun, which is more efficient.
Migrating the whole disc does not change current data placement, but does not allow to perform any reconfigurations (lun size change, placement, disk size etc).

Let's create new volumes, it is required to migrate all volumes in a group to it is better to create a script that does it automatically:

[root@centos54 OpenMigrator]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created
[root@centos54 OpenMigrator]# vgcreate destvg /dev/sdb
Volume group "destvg" successfully created
[root@centos54 OpenMigrator]# lvs testvg|grep testvg|awk '{printf "lvcreate -L %s -n %s destvg\n",$NF,$1}'|sh
Logical volume "test1lv" created
Logical volume "test2lv" created
[root@centos54 OpenMigrator]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv destvg -wi-a- 500.00M
test2lv destvg -wi-a- 500.00M
rootlv rootvg -wi-ao 7.38G
swaplv rootvg -wi-ao 512.00M
test1lv testvg -wi-a- 500.00M
test2lv testvg -wi-ao 500.00M

Now we must create file with source/target pairs:

[root@centos54 OpenMigrator]# lvs testvg|grep testvg|awk '{printf "/dev/testvg/%s\t/dev/destvg/%s\n",$1,$1}' > file_session.txt
[root@centos54 OpenMigrator]# cat file_session.txt
/dev/testvg/test1lv /dev/destvg/test1lv
/dev/testvg/test2lv /dev/destvg/test2lv

And create copy session - this is just a definition, no copy process is started:

[root@centos54 OpenMigrator]# stormigrate create -session testmigr -file file_session.txt

Let's check what is the current status:

[root@centos54 OpenMigrator]# stormigrate show -session testmigr

MB
Source Size Target
---------------------------------- --------- ----------------------------------
/dev/testvg/test1lv 500 /dev/destvg/test1lv
/dev/testvg/test2lv 500 /dev/destvg/test2lv


Session Name: testmigr
Session State: Inactive
Action Plan: N/A
Task States: N/A
Number of Pairs: 2
Total Source Size: 1000 MB

Now we can activate session, which means to establish a relationship between source and destination:

[root@centos54 OpenMigrator]# stormigrate query -session testmigr
S D
y o
Current Task n Rate n MB
Source Target Action State c MB/s % e Left
----------------------- ----------------------- -------- ----- - ---- --- -----
/dev/testvg/test1lv /dev/destvg/test1lv None Idle N N/A N/A 0
/dev/testvg/test2lv /dev/destvg/test2lv None Idle N N/A N/A 0
---- --- -----
Total Copy: 0 0 0
Total Compare: 0 0 0

Session Name: testmigr
Session State: Active
Action Plan: None
Task States: Idle
Number of Pairs: 2
Total Source Size: 1000 MB
System I/O Processing: PASSTHRU
Target Devices State: RW
Time: 23:03:03

Destination volumes are now opened by kernel, but still no copying was started:

[root@centos54 OpenMigrator]# lvs destvg
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv destvg -wi-ao 500.00M
test2lv destvg -wi-ao 500.00M

Let's start the copy process:

[root@centos54 OpenMigrator]# stormigrate copy -session testmigr
[root@centos54 OpenMigrator]# stormigrate query -session testmigr
S D
y o
Current Task n Rate n MB
Source Target Action State c MB/s % e Left
----------------------- ----------------------- -------- ----- - ---- --- -----
/dev/testvg/test1lv /dev/destvg/test1lv Copy Run N 1 0 500
/dev/testvg/test2lv /dev/destvg/test2lv Copy Run N 3 0 500
---- --- -----
Total Copy: 3 0 1000
Total Compare: 0 0 0

Session Name: testmigr
Session State: Active
Action Plan: Copy
Task States: Running
Number of Pairs: 2
Total Source Size: 1000 MB
System I/O Processing: MIRRORING
Target Devices State: NR
Time: 23:05:11

Let's verify the status of migration:

[root@centos54 OpenMigrator]# stormigrate verify -session testmigr
The volume pair has not completed the Copy operation.

Copy process tuning considerations:

Based on stormigrate query it is clear that copy Rate is 1MB.

root@centos54 OpenMigrator]# stormigrate query -session testmigr
S D
y o
Current Task n Rate n MB
Source Target Action State c MB/s % e Left
----------------------- ----------------------- -------- ----- - ---- --- -----
/dev/testvg/test1lv /dev/destvg/test1lv Copy Run N 1 0 500
/dev/testvg/test2lv /dev/destvg/test2lv Copy Run N 1 0 500

We can speed it up:

[root@centos54 OpenMigrator]# stormigrate tune -ceiling 10

Warning: Changing this setting can seriously impact running applications. Are you sure [Y/[N]]?y


System wide Open Migrator Tune Parameters

Parameter Value
----------------------------------- --------
I/O Size 64 kb
Ceiling 10 MB/s
Mover threads 16
Checkpoint Size 256 MB
Mirror I/O Throttle Performance Ratio 200 %
Mirror I/O Throttle Recovery Action Terminate


[root@centos54 OpenMigrator]# stormigrate tune -iosize 128

Warning: Changing this setting can seriously impact running applications. Are you sure [Y/[N]]?y


System wide Open Migrator Tune Parameters

Parameter Value
----------------------------------- --------
I/O Size 128 kb
Ceiling 10 MB/s
Mover threads 16
Checkpoint Size 256 MB
Mirror I/O Throttle Performance Ratio 200 %
Mirror I/O Throttle Recovery Action Terminate


Finally the copy process is finished:

[root@centos54 OpenMigrator]# stormigrate query -session testmigr
S D
y o
Current Task n Rate n MB
Source Target Action State c MB/s % e Left
----------------------- ----------------------- -------- ----- - ---- --- -----
/dev/testvg/test1lv /dev/destvg/test1lv Copy Done Y 0 100 0
/dev/testvg/test2lv /dev/destvg/test2lv Copy Done Y 0 100 0
---- --- -----
Total Copy: 0 100 0
Total Compare: 0 0 0

Session Name: testmigr
Session State: Active
Action Plan: Copy
Task States: Done
Number of Pairs: 2
Total Source Size: 1000 MB
System I/O Processing: MIRRORING
Target Devices State: NR
Time: 23:16:03

[root@centos54 OpenMigrator]# stormigrate verify -session testmigr

All of the volume pairs have completed the Copy operation.

Procedure to switch to new volumes is as follows:

1. Stop application and unmount original filesystems

[root@centos54 OpenMigrator]# umount /test1_migr/
[root@centos54 OpenMigrator]# umount /test2_migr/

2. Deactivate migration process.

[root@centos54 OpenMigrator]# stormigrate deactivate -session testmigr
[root@centos54 OpenMigrator]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv destvg -wi-a- 500.00M
test2lv destvg -wi-a- 500.00M
rootlv rootvg -wi-ao 7.38G
swaplv rootvg -wi-ao 512.00M
test1lv testvg -wi-a- 500.00M
test2lv testvg -wi-a- 500.00M

All source end target volumes are not opened.

3. Change the name of source and destination group

[root@centos54 OpenMigrator]# vgchange -an testvg
0 logical volume(s) in volume group "testvg" now active
[root@centos54 OpenMigrator]# vgchange -an destvg
0 logical volume(s) in volume group "destvg" now active
[root@centos54 OpenMigrator]# vgrename testvg oldvg
Volume group "testvg" successfully renamed to "oldvg"
[root@centos54 OpenMigrator]# vgrename destvg testvg
Volume group "destvg" successfully renamed to "testvg"
[root@centos54 OpenMigrator]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv oldvg -wi--- 500.00M
test2lv oldvg -wi--- 500.00M
rootlv rootvg -wi-ao 7.38G
swaplv rootvg -wi-ao 512.00M
test1lv testvg -wi--- 500.00M
test2lv testvg -wi--- 500.00M
[root@centos54 OpenMigrator]# pvs
PV VG Fmt Attr PSize PFree
/dev/hda2 rootvg lvm2 a- 7.89G 20.00M
/dev/sda oldvg lvm2 a- 2.00G 1.02G
/dev/sdb testvg lvm2 a- 2.00G 1.02G

4. Activate new volumes and mount filesystem from new location

[root@centos54 OpenMigrator]# vgchange -ae testvg
2 logical volume(s) in volume group "testvg" now active

Some checking:

[root@centos54 OpenMigrator]# pvs
PV VG Fmt Attr PSize PFree
/dev/hda2 rootvg lvm2 a- 7.89G 20.00M
/dev/sda oldvg lvm2 a- 2.00G 1.02G
/dev/sdb testvg lvm2 a- 2.00G 1.02G
[root@centos54 OpenMigrator]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
test1lv oldvg -wi--- 500.00M
test2lv oldvg -wi--- 500.00M
rootlv rootvg -wi-ao 7.38G
swaplv rootvg -wi-ao 512.00M
test1lv testvg -wi-a- 500.00M
test2lv testvg -wi-a- 500.00M
[root@centos54 OpenMigrator]# mount /dev/testvg/test1lv /test1_migr
[root@centos54 OpenMigrator]# mount /dev/testvg/test2lv /test2_migr
[root@centos54 OpenMigrator]# df -k /test1_migr /test2_migr
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/testvg-test1lv
495844 453553 16691 97% /test1_migr
/dev/mapper/testvg-test2lv
495844 10544 459700 3% /test2_migr
[root@centos54 OpenMigrator]#

5. Clenup

[root@centos54 OpenMigrator]# stormigrate cleanup

WARNING: Cleanup will delete all the defined sessions

Are you sure you want to cleanup the configuration[Y/N]?y

The configuration was removed successfully
[root@centos54 OpenMigrator]# stormigrate list

No sessions found.

Finally it is possible to remove old volume group and disks from host.

2 comments:

Unknown said...

How does a migration between two hosts work?, does it go by LAN?

Unknown said...

nice....