After getting ESXi on PI working I started to wonder what is it really useful for. Outside of just playing with it I wasn’t able to come up with anything besides a VPN client. I have to much x86_64 compute to need extra compute at home. So I decided to do something silly. Running a VeloCloud Edge on my PI came to mind. I thought about unpacking the KVM version and seeing if I could convert it to ARM but I figured there was custom code somewhere and without the source it seemed very unlikely. So instead I went the inception route. I built an Ubuntu VM on my PI and loaded up qemu.
I did have to get the qemu-system-x86 package as well as the regular qemu stuff. First I had to change the netplan on this machine to include a bridge device. I edited /etc/netplan/00-installer-config.yaml to look like this:
network:
ethernets:
ens192:
dhcp4: no
version: 2
renderer: networkd
bridges:
br0:
interfaces:
- ens192
addresses:
- 192.168.0.4/24
- 192.168.2.2/24
gateway4: 192.168.0.1
nameservers:
addresses:
- 192.168.0.1
Notice the 192.168.2.2/24 address. This is the default network of a Velocloud edge inside interface. The edge will assign itself 192.168.2.1/24 and use DHCP for its WAN interface. Run “netplan apply” to apply the config. Next I added some tap devices so they could be assigned to the bridge and ports on the qemu vm.
ip tuntap add tap0 mode tap
ip tuntap add tap1 mode tap
ip tuntap add tap2 mode tap
Then I added the tap interfaces to the bridge:
brctl addif br0 tap0brctl addif br0 tap1
brctl addif br0 tap2
To verify this was all configured correctly run: brctl show. The output should be similar to this:
Now it’s time to copy the edge disk file to the VM and unzip it. You must have access to download the file from VMware at https://my.vmware.com/web/vmware/downloads/details?downloadGroup=SD-WAN-EDGE-322&productId=854. Download the KVM image. Copy the file to the VM and extract it:
gzip -d edge-VC_KVM_GUEST-x86_64-4.1.0-69-R410-20201028-GA-ac590f42a5-updatable-ext4.qcow2.gz
To tell the edge what password you want the root account to be configured with you must make some cloud-init files and put them on a iso. There are two files to make. The user-data file includes the password and VeloCloud Orchestrator (VCO) instance to connect to. You are not required to enter the VCO information. The edge can be activated manually through the web UI. The other required file is the meta-data file. This file included the instance and host name of the edge. There is another file named network-data that allows you to specify a static IP on the WAN interface. I’m relying on DHCP for WAN so I did not configure this file.
The user-data file looks like this:
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: true
The meta-data file looks like this:
name.instance-id: vedge1
local-hostname: vedge1
Once these files are made they must be put into an iso file. One command takes care of that:
genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data
Now time to start the VM.
qemu-system-x86_64 -smp 2 -m 3072 -netdev tap,id=u1,ifname=tap0 -device e1000,netdev=u1,mac=52:55:00:d1:55:01 -netdev tap,id=u2,ifname=tap1 -device e1000,netdev=u2,mac=52:55:00:d1:55:02 -netdev tap,id=u3,ifname=tap2 -device e1000,netdev=u3,mac=52:55:00:d1:55:03 -drive file=edge-VC_KVM_GUEST-x86_64-4.1.0-69-R410-20201028-GA-ac590f42a5-updatable-ext4.qcow2 -drive file=seed.iso,media=cdrom,if=none,id=cdrom1 -device ich9-ahci,id=ahci -device ide-cd,drive=cdrom1,bus=ahci.0 -vnc :1
This command starts up qemu-system-x86_64 with 2 cpus, 3072MB of memory, 3 network interfaces, uses the edge qcow2 file as the hard disk, mounts the seed.iso generated as an IDE drive, and attaches the console to a VNC server on port 1. The edge requires 3 interfaces. The first two are bonded as a LAN and the 3rd is WAN.
After about 40 (yes 40) minutes the edge is up and running. The console will look something like this once boot is complete:
After verifying the inside interface address is ping-able (192.168.2.1) lynx is able to load the activation page from the ubuntu host console:
lynx 192.168.2.1
With verification that it boots and will load up. I put the interfaces into promiscuous mode so i could access the edge from my copmuter.
ip link set tap0 promisc on
ip link set tap1 promisc on
ip link set tap2 promisc on
ip link set br0 promisc on
For some reason activation with hostname doesn’t work. Activating with IP does. The ignore certificate errors advanced option must be enabled since the certificates are based on hostname.
Once activated the edge downloads the firmware associated with the profile assigned in the VCO. Once completed the edge reboots and comes online. The update process takes a VERY long time. It appears to mainly be due to a bzip operation on an emulated x86_64 processor being extremely slow.
Now is this useful? Probably not. Did i configure it correctly? Probably not. Could most of this run as native ARM code? Probably. The last 1% of the code probably wouldn’t without access to the source and that’s the secret sauce. I don’t expect it to preform very well or even at all since this is emulated x86_64 running on ARM. However it’s a neat example of something you can do with a very low cost hypervisor. Maybe in the future VMware will release an ARM image for VeloCloud VMs.
vSkeeBall 