I know eveyone has done one of these but since I went through the process of writing the code and getting it open source approval at VMware I figured I should write about it. In 2019 I got tired of all of the “official/supported” way to deploy k8s with NSX integration. I found them cumbersome and painful. For instance PKS (TKGI now) took forever to deploy and had so many requirements it was easier to just write my own tool.
I really wrote this to help me understand how to deploy Kubernetes with and without NSX. I started with a the idea that there was no good reason to have external dependencies on my deployment method outside of standard vSphere things. The only thing need to use my method is vCenter, PowerCLI, and an Ubuntu template. Of course if you want NSX integration then you need NSX-T as well.
The sources for both the NSX and non NSX version are on my github.
Non NSX: https://github.com/khensler/deploy-k8s-local
NSX: https://github.com/khensler/deploy-k8s-local-NSX
Both of these will allow you to deploy a k8s cluster on vSphere, scale it up, scale it down, and destroy it with a single command. I use VM custom attributes as a key value store for the cluster information. Once deployed you’ll find K8-Cluster, K8-Role, and K8-Join attributes. K8-Cluster is set to the cluster name specified. K8-Role is set to Master or Node depending upon the VM role. K8-Join will be set on the master node with the command to join the cluster. You’ll probably start screaming about why this is bad for security and such. DON’T USE THIS IN PROD. Or change it so it doesn’t do this.
Both version require a base ubuntu vm with a snapshot. The script will make a linked clone of the VM for speed and first deploy the master node. Once the master node is completed the script copy the admin.conf file to the local machine for remote access and then will call a powershell workflow to do the other nodes in parallel. Making linked clones, installing upstream k8s, install the NSX bits, configuring NSX with the appropriate configuration for the CNI to work, and then joining the cluster. For scale up later the same procedure is followed. For scale down the nodes will be removed from the cluster and the powered of and deleted. For complete deletion all nodes will be powered off and deleted.
Look for a few more posts on each section of the script and how it works later. If you go and look at the code please remember its a work in progress and needs a lot of cleanup and I haven’t checked it against later versions.
vSkeeBall 