We have a fairly large ~46Tb database which needed to get off an older san array due to EMC’s\u00a0extortion-like\u00a0support pricing model.<\/p>\n
My goal is to take the shortest outage possible, while getting all 46Tb of data migrated to disks on the new array.<\/p>\n
The minimal outage does require taking the database down and briefly unmounting the filesystems. But, a 5 minute outage to migrate 46Tb of data seems somewhat tolerable given our alternatives.<\/p>\n
For the purpose of this article, I’ll use the simpler CTD disks from a setup with internal only disk, rather than the eyechart CTD disk labels to illustrate the technique. The database server I’m working on has hundreds of luns, all with device names similar to:<\/p>\n
c3t60060480000190100478533031383339d0s2<\/strong><\/pre>\nSo, we’ll go with internal disks\u00a0c1t0<\/span>d0s5<\/strong>, mapped to metadevice d51<\/strong>, mounted as \/d1001<\/strong>, as a source disk that has good data. The destination disk is\u00a0c1t1<\/span>d0s5<\/strong>\u00a0mapped to metadevice d52<\/strong>\u00a0as a migration destination.<\/p>\n
Let’s make sure we know our source disk, so I’m creating a file based on the device name.<\/p>\n
bash-3.00# metastat -ap | grep d51<\/strong>\nd51 1 1 c1t0d0s5\nbash-3.00# mount | grep d51<\/strong>\n\/d1001<\/strong> on \/dev\/md\/dsk\/d51<\/strong> read\/write\/setuid\/devices...\nbash-3.00# touch \/d1001\/I_am_d51<\/strong><\/span>\nbash-3.00# ls \/d1001<\/strong>\nI_am_d51 lost+found ...<\/pre>\nOur original disk is mounted as \/d1001, first unmount it.<\/p>\n
bash-3.00# umount \/d1001<\/strong><\/pre>\nNow create the mirror, and attach d51 with our good data to it.<\/p>\n
bash-3.00# metainit d50 -m d51<\/strong>\nd50: Mirror is setup<\/pre>\nTo see that our data is still there, mount our new mirror and take a look<\/span><\/span><\/p>\n
bash-3.00# mount \/dev\/md\/dsk\/d50 \/d1001<\/strong>\nbash-3.00# ls \/d1001<\/strong>\nI_am_d51 lost+found<\/pre>\nFor my purposes, I would do the above unmount, attach new mirror metadevice for all mountpoints. \u00a0At this point the database could be restarted.<\/p>\n
We will then create the new side of the SDS mirror, then attach to get the data synchronized while the database is running.<\/p>\n
Use format<\/strong> to label the new incoming disks (c1t1d0s5<\/strong>) then use metainit to initialize the new disk.<\/p>\n
bash-3.00# metainit d52 1 1 c1t1d0s5<\/span><\/strong><\/pre>\nWe can now attach the new disk, which would represent our new san array.<\/span>
\n<\/span><\/strong><\/p>\nbash-3.00# metattach d50 d52<\/strong>\nd50: submirror d52 is attached\nbash-3.00# metastat d50<\/strong>\nd50: Mirror\n Submirror 0: d51\n State: Okay\n Submirror 1: d52\n State: Resyncing\n Resync in progress: 0 % done\n Pass: 1\n Read option: roundrobin (default)\n Write option: parallel (default)\n Size: 117726144 blocks (56 GB)\n\nd51: Submirror of d50\n State: Okay\n Size: 117726144 blocks (56 GB)\n Stripe 0:\n Device Start Block Dbase State Reloc Hot Spare\n c1t0d0s5 0 No Okay Yes \n\nd52: Submirror of d50\n State: Resyncing\n Size: 117726144 blocks (56 GB)\n Stripe 0:\n Device Start Block Dbase State Reloc Hot Spare\n c1t1d0s5 0 No Okay Yes<\/pre>\nbash-3.00# metastat d50 | grep progress<\/strong>\nResync in progress: 33 % done<\/pre>\nWhen d50 has completed sync’ing we can detach the original disk which will make our old array decommission possible.<\/p>\n
bash-3.00# metastat d50<\/strong>\nd50: Mirror\n Submirror 0: d51\n State: Okay\n Submirror 1: d52\n State: Okay\n Pass: 1\n Read option: roundrobin (default)\n Write option: parallel (default)\n Size: 117726144 blocks (56 GB)\n\nd51: Submirror of d50\n State: Okay\n Size: 117726144 blocks (56 GB)\n Stripe 0:\n Device Start Block Dbase State Reloc Hot Spare\n c1t0d0s5 0 No Okay Yes \n\nd52: Submirror of d50\n State: Okay\n Size: 117726144 blocks (56 GB)\n Stripe 0:\n Device Start Block Dbase State Reloc Hot Spare\n c1t1d0s5 0 No Okay Yes \n\nDevice Relocation Information:\nDevice Reloc Device ID\nc1t0d0 Yes id1,sd@n500000e0125d2140\nc1t1d0 Yes id1,sd@n500000e0125e77c0<\/pre>\nWe can now disconnect our original disk and retire our array.<\/p>\n
bash-3.00# metadetach d50 d51<\/strong>\nd50: submirror d51 is detached<\/pre>\nbash-3.00# df -h | grep d1001<\/strong>\n\/dev\/md\/dsk\/d50 55G 56M 55G 1% \/d1001\nbash-3.00# ls \/d1001<\/strong>\nI_am_d51 lost+found<\/pre>\nAs a note, if there are dozens of mounpoints to synchronize, it should be done in small groups or serially. Having two dozen metadevices syncing at the same time will certainly affect the performance of the database.<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
I need to migrate a Solaris 10 database server from an old san array to a newer one, while taking as small an outage as we can. I came upon the migration method below by necessity. We have a fairly … Continue reading