When provisioning storage and creating volumes, there are times when you create a volume for a specific workload, and things change after the fact. Veritas volume manager can easily deal with changing requirements, and allows you to convert between volume types ( e.g., convert a RAID5 volume to a striped mirrored volume) on the fly. Veritas performs this operation in most cases with layered volumes, and requires a chunk of free space to complete the relayout operation. The VxVM users guide describes the supported relayout operations, and also provides disk space requirements.
To illustrate just how useful the relayout operation is, let’s say your manager just finished reading a Gartner report that criticizes RAID5. He comes over to your desk and asks you to convert the Oracle data volume from a 4-column RAID5 volume to a 2-column striped-mirror volume. Since you despise software RAID5, you set down UNIX File systems and run vxassist(1m) with the “relayout” keyword, the “layout” to convert to, and the number of columns to use (the ncols option is only used with striped volumes):
$ vxassist -g oof relayout oravol01 layout=stripe-mirror ncol=2
The relayout operation requires a temporary region to copy data to (marked with a state of TMP in vxprint) prior to migrating data it to it’s final destination. If sufficent space isn’t available, vxassist will display an error similar to the following and exit:
VxVM vxassist ERROR V-5-1-6345 Cannot allocate 15728640 blocks of disk space \
required by the relayout operation for column addition
VxVM vxassist ERROR V-5-1-4037 Relayout operation aborted. (7)
Once the relayout begins, the vxrelayout(1m) and vxtask(1m) utilities can be used to monitor the progress of the relayout operations:
$ vxrelayout -g oof status oravol01
RAID5, columns=4, stwidth=32 --> STRIPED-MIRROR, columns=2, stwidth=128
Relayout running, 10.02% completed.
$ vxtask list
TASKID PTID TYPE/STATE PCT PROGRESS
2125 RELAYOUT/R 14.45% 0/41943168/6061184 RELAYOUT oravol01 oof
When the relayout operation completes, the volume will be converted to a 2-column striped-mirrored volume:
$ vxprint -hft
Disk group: oof
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg oof default default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971584 SELECT oravol01-01 fsgen
pl oravol01-01 oravol01 ENABLED ACTIVE 20971776 STRIPE 2/128 RW
sv oravol01-Ds01 oravol01-01 oravol01-d01 1 10485888 0/0 3/3 ENA
sv oravol01-Ds02 oravol01-01 oravol01-d02 1 10485888 1/0 3/3 ENA
v oravol01-d01 - ENABLED ACTIVE 10485888 SELECT - fsgen
pl oravol01-dp01 oravol01-d01 ENABLED ACTIVE 10485888 CONCAT - RW
sd c1t1d0-04 oravol01-dp01 c1t1d0 0 10485888 0 c1t1d0 ENA
pl oravol01-dp02 oravol01-d01 ENABLED ACTIVE 10485888 CONCAT - RW
sd c1t6d0-03 oravol01-dp02 c1t6d0 2096640 10485888 0 c1t6d0 ENA
pl oravol01-dp03 oravol01-d01 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-02 oravol01-dp03 c1t3d0 6990528 528 LOG c1t3d0 ENA
v oravol01-d02 - ENABLED ACTIVE 10485888 SELECT - fsgen
pl oravol01-dp04 oravol01-d02 ENABLED ACTIVE 10485888 CONCAT - RW
sd c1t2d0-04 oravol01-dp04 c1t2d0 0 10485888 0 c1t2d0 ENA
pl oravol01-dp05 oravol01-d02 ENABLED ACTIVE 10485888 CONCAT - RW
sd c1t5d0-04 oravol01-dp05 c1t5d0 2100480 10485888 0 c1t5d0 ENA
pl oravol01-dp06 oravol01-d02 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-03 oravol01-dp06 c1t3d0 6991056 528 LOG c1t3d0 ENA
Once the relayout completes, you pick up the wonderful Linker’s & Libraries guide and begin reading. Just as you get to chapter 7 (Object file formats), your manager comes back to you and asks you to convert the 2-column striped-mirror volume to a 3-column striped-mirror volume. You are eager to use the remaining spindles in your storage array, so you execute vxassist(1m) with the relayout and ncol options:
$ vxassist -b -g oof relayout oravol01 layout=mirror-stripe ncol=3
When adding additional columns to a striped volume with the relayout option, ncol refers to the total number of columns, not the number to add. Once the relayout process completes, you have a 3-column striped-mirrored volume:
$ vxprint -hft
Disk group: oof
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg oof 6 default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971520 SELECT oravol01-03 fsgen
pl oravol01-03 oravol01 ENABLED ACTIVE 20971776 STRIPE 3/128 RW
sv oravol01-S01 oravol01-03 oravol01-L01 1 6990592 0/0 2/2 ENA
sv oravol01-S02 oravol01-03 oravol01-L02 1 6990592 1/0 2/2 ENA
sv oravol01-S03 oravol01-03 oravol01-L03 1 6990592 2/0 2/2 ENA
v oravol01-L01 - ENABLED SYNC 6990592 SELECT - fsgen
pl oravol01-P01 oravol01-L01 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t1d0-02 oravol01-P01 c1t1d0 0 6990592 0 c1t1d0 ENA
pl oravol01-P02 oravol01-L01 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t4d0-02 oravol01-P02 c1t4d0 0 6990592 0 c1t4d0 ENA
v oravol01-L02 - ENABLED SYNC 6990592 SELECT - fsgen
pl oravol01-P03 oravol01-L02 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t2d0-02 oravol01-P03 c1t2d0 0 6990592 0 c1t2d0 ENA
pl oravol01-P04 oravol01-L02 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t5d0-02 oravol01-P04 c1t5d0 0 6990592 0 c1t5d0 ENA
v oravol01-L03 - ENABLED SYNC 6990592 SELECT - fsgen
pl oravol01-P05 oravol01-L03 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t3d0-02 oravol01-P05 c1t3d0 0 6990592 0 c1t3d0 ENA
pl oravol01-P06 oravol01-L03 ENABLED ACTIVE 6990592 CONCAT - RW
sd c1t6d0-02 oravol01-P06 c1t6d0 0 6990592 0 c1t6d0 ENA
While reading through the DTRACE users guide, you notice that the io and syscall providers can be used to determine application I/O patterns. Once you get your D script working, you notice that the Java application running on your server is reading and writing data in 96k chunks. To optimize write performance, you decide to adjust the stripunit size to be a multiple of the application I/O size. This is accomplished with vxassist(1m)‘s stripeunit option:
$ vxassist -g oof relayout oravol01 stripeunit=32k
Once this operation completes, you have a 3-column striped-mirrored volume with a 32k stripe unit size (since we have 3-columns, and the application is reading and writing 96k chunks, a 32k stripeunit is optimal):
$ vxprint -hft
Disk group: oof
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg oof default default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971584 SELECT oravol01-01 fsgen
pl oravol01-01 oravol01 ENABLED ACTIVE 20971584 STRIPE 3/64 RW
sv oravol01-Ds01 oravol01-01 oravol01-d01 1 6990528 0/0 3/3 ENA
sv oravol01-Ds02 oravol01-01 oravol01-d02 1 6990528 1/0 3/3 ENA
sv oravol01-Ds06 oravol01-01 oravol01-d06 1 6990528 2/0 3/3 ENA
v oravol01-d01 - ENABLED ACTIVE 6990528 SELECT - fsgen
pl oravol01-dp01 oravol01-d01 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t1d0-03 oravol01-dp01 c1t1d0 0 6990528 0 c1t1d0 ENA
pl oravol01-dp02 oravol01-d01 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t6d0-01 oravol01-dp02 c1t6d0 2096640 6990528 0 c1t6d0 ENA
pl oravol01-dp03 oravol01-d01 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-01 oravol01-dp03 c1t3d0 0 528 LOG c1t3d0 ENA
v oravol01-d02 - ENABLED ACTIVE 6990528 SELECT - fsgen
pl oravol01-dp04 oravol01-d02 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t2d0-03 oravol01-dp04 c1t2d0 0 6990528 0 c1t2d0 ENA
pl oravol01-dp05 oravol01-d02 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t5d0-01 oravol01-dp05 c1t5d0 2100480 6990528 0 c1t5d0 ENA
pl oravol01-dp06 oravol01-d02 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-02 oravol01-dp06 c1t3d0 528 528 LOG c1t3d0 ENA
v oravol01-d06 - ENABLED ACTIVE 6990528 SELECT - fsgen
pl oravol01-dp16 oravol01-d06 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t4d0-01 oravol01-dp16 c1t4d0 2096640 6990528 0 c1t4d0 ENA
pl oravol01-dp17 oravol01-d06 ENABLED ACTIVE 6990528 CONCAT - RW
sd c1t3d0-04 oravol01-dp17 c1t3d0 6991584 6990528 0 c1t3d0 ENA
pl oravol01-dp18 oravol01-d06 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-03 oravol01-dp18 c1t3d0 1056 528 LOG c1t3d0 ENA
Now a week or two goes by and your manager comes to you and tells you he wants to use concatenated-mirrored volumes since the back-end is already striped with EMC meta volumes. Since you have noted little to no benefit with host side striping, you set down Solaris Systems Programming, and fire up vxassist to convert the striped-mirrored volume to a concat-mirrored volume:
$ vxassist -g oof relayout oravol01 layout=concat-mirror
Once this operation completes, you have a shiny new concat-mirrored volume:
$ vxprint -hft
Disk group: oof
dg oof default default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971520 SELECT - fsgen
pl oravol01-01 oravol01 ENABLED ACTIVE 20971520 CONCAT - RW
sv oravol01-Ds03 oravol01-01 oravol01-d03 1 20971520 0 3/3 ENA
v oravol01-d03 - ENABLED ACTIVE 20971520 SELECT - fsgen
pl oravol01-dp07 oravol01-d03 ENABLED ACTIVE 20971520 CONCAT - RW
sd c1t1d0-04 oravol01-dp07 c1t1d0 0 20971520 0 c1t1d0 ENA
pl oravol01-dp08 oravol01-d03 ENABLED ACTIVE 20971520 CONCAT - RW
sd c1t4d0-04 oravol01-dp08 c1t4d0 2880 20971520 0 c1t4d0 ENA
pl oravol01-dp09 oravol01-d03 ENABLED ACTIVE LOGONLY CONCAT - RW
sd c1t3d0-02 oravol01-dp09 c1t3d0 2097152 528 LOG c1t3d0 ENA
Once the change is implemented, things run smoothly for a few weeks. Then one day your manager comes to you and tells you times are tight, and asks you to convert the mirrored-concatenated volume back to a RAID5 volume. Since this will save the company money (which will keep you employed), you set down the Veritas Volume Manager Troubleshooting Guide and fire up vxassist to perform the conversion:
$ vxassist -g oof relayout oravol01 layout=raid5 ncol=5
Once the relayout process completes, you have a nice 5-column RAID5 volume:
$ vxprint -hft
Disk group: oof
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg oof default default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971520 RAID - raid5
pl oravol01-Dp02 oravol01 ENABLED LOG 1600 CONCAT - RW
sd c1t5d0-02 oravol01-Dp02 c1t5d0 5242880 1600 0 c1t5d0 ENA
pl oravol01-03 oravol01 ENABLED ACTIVE 20971520 RAID 5/32 RW
sd c1t1d0-03 oravol01-03 c1t1d0 0 5242880 0/0 c1t1d0 ENA
sd c1t5d0-03 oravol01-03 c1t5d0 0 5242880 1/0 c1t5d0 ENA
sd c1t6d0-02 oravol01-03 c1t6d0 0 5242880 2/0 c1t6d0 ENA
sd c1t3d0-03 oravol01-03 c1t3d0 2096640 5242880 3/0 c1t3d0 ENA
sd c1t4d0-03 oravol01-03 c1t4d0 2096640 5242880 4/0 c1t4d0 ENA
Now due to some budget tightening, your manager is let go, and a consultant is brought in to provide “performance” recommendations. After reviewing the system configuration, he recommends converting the RAID5 volume to a pure striped volume. Since this provides absolutely NO protection for your data, you question his decision, but are forced to implement the change. Since you are a devoted company guy, you execute vxassist to convert the volume to a pure striped volume:
$ vxassist -g oof relayout oravol01 layout=striped ncol=6
Once this operation completes, you have a non-protected 6-column striped volume:
$ vxprint -hft
Disk group: oof
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg oof default default 21000 1120604922.22.tigger
dm c1t1d0 c1t1d0s2 auto 2048 35365968 -
dm c1t2d0 c1t2d0s2 auto 2048 35521408 -
dm c1t3d0 c1t3d0s2 auto 2048 35521408 -
dm c1t4d0 c1t4d0s2 auto 2048 35521408 -
dm c1t5d0 c1t5d0s2 auto 2048 35521408 -
dm c1t6d0 c1t6d0s2 auto 2048 35521408 -
v oravol01 - ENABLED ACTIVE 20971520 SELECT oravol01-03 fsgen
pl oravol01-03 oravol01 ENABLED ACTIVE 20971776 STRIPE 6/128 RW
sd c1t1d0-03 oravol01-03 c1t1d0 0 3495296 0/0 c1t1d0 ENA
sd c1t5d0-02 oravol01-03 c1t5d0 0 3495296 1/0 c1t5d0 ENA
sd c1t6d0-02 oravol01-03 c1t6d0 0 3495296 2/0 c1t6d0 ENA
sd c1t3d0-02 oravol01-03 c1t3d0 2096640 3495296 3/0 c1t3d0 ENA
sd c1t4d0-02 oravol01-03 c1t4d0 2096640 3495296 4/0 c1t4d0 ENA
sd c1t2d0-03 oravol01-03 c1t2d0 2096640 3495296 5/0 c1t2d0 ENA
Since this configuration makes you extremely nervous, you decide to update your resume, and eventually leave to take a position with a different company. After settling into your new job, you get a frantic call from the consultant at your previous job. He informs you that one of disks failed, and users are unable to get to the MySQL database. Being the nice guy you are, you explain how to create a redundant high performant volume with vxassist, and then walk him through using mysqlimport to restore the data that was being backed up with the mysqldump utility. You also tell him to stay tuned to this website for a future entry on recovery fundamentals (or the lack of them in this case!).
All of the operations described here were performed with an active MySQL database accessing the volume that was used in the examples. It is imperative to test all configuration changes in a non-production environment prior to implementing in production. While the relayout process should work flawlessly in other environments, I accept no responsibility for lost or corrupted data.