Does anyone else remember when XCOPY via VAAI was released on vSphere? How cool it was to offload the copy of a VM to the array. It only got cooler as the storage vendors figured out how to not actually need to copy the data but just point to the data. Clones went from having to come to the host and back down to just the array copying data to the array just saying “ok I copied” the data.
You know it was super cool that first time you cloned a huge VM and it didn’t take forever. Just did it. Well why can’t we do that for other things? The array supports it. Why doesn’t anyone else use the offloads? If you go poking around you’ll find tools like sg_xcopy or ddpt. sq_xcopy has some limitations and doesn’t seem to work and ddpt works if you do it right but since most arrays can only copy small chunks at a time and the tool’s built in bulk requests don’t work you can really only copy 16MiB of data at a time. It takes about 15 minutes to have it run in a loop to copy a 1TB disk. Just too slow. So as per usual on this blog I did a thing….
LVM is a way to allocate volumes on a physical disk in Linux. Go look it up if you haven’t heard of it. LVM is also what a number of different hypervisors use to back VM disks. By using LVM volumes as disks they can get around the clustered filesystem limitations that we’ve run into in everyone’s attempts to move off of vSphere. VMFS is a huge thing that people just took for granted. Now I could go on about this and how we need one for Linux that isn’t gfs2 or ocfs and get on my soap box but enough of you have heard me talk about this and I have some other things to blog about surrounding it when I have some more time. Yes that’s a run on sentence, deal with it.
Back to the point. I didn’t see a reason why offloading the copy of a VM disk backed by a logical volume (LV) on LVM wasn’t possible. So I went ahead and wrote a python script to do it. I stuck it here: https://github.com/PureStorage-OpenConnect/lvm-xcopy
It works for SCSI (FC, iSCSI) and NVME volumes. Both work slightly differently but the result is the same. The script finds the LBAs that the LV is using, creates a new LV on an existing LVM volume, and then clones the data in the source LV to the destination LV via the offloads. For SCSI it can be either the same volume/lun or a different one on the same array for NVME it must be the same namespace/backend volume.
Pretty cool right? So why does anyone care? Well there is also a script in the repo under the hypervisors directory that give an example of a use case. Cloning Proxmox VMs https://github.com/PureStorage-OpenConnect/lvm-xcopy/tree/master/hypervisors.
For those of you running LVM and need to clone it or Proxmox and want fast cloning of VMs take a peek and let me know what you think.
vSkeeBall 