Options to run a local kubernetes cluster
Note: This is as accurate as 2019-03-20 this stuff moves fast so there might be other options now.
Running kubernetes locally
This post is a general post about the different ways to deploy kubernetes locally, and emulate what you would do in a real cloud on your local machine. This can be helpful not only with cost savings, but the ability to leverage portions for CI/CD on your cloud native applications, or the situation where you don’t have reliable internet access and want to do “real” work.
The different technologies we’ll be walking through are:
Before we go any farther I strongly suggest a machine with at least 8 Gigs of RAM. Some of the following options are VM based and modern operating systems require at least 4 Gigs of RAM before issues can arise. Having 8 Gigs of RAM allows for VirtualBox or the like the like to get enough resources to be able to do more then the bare minimum.
Next you might as well
kubectl installed on your local machine, it’s the canonical way to
interface with any kubernetes cluster, and your local instance is no different. I will say
that you could call directly to the API, but I’d only suggest doing this if you had a real
minikube is the defacto entry point for most people learning or figuring out how to run
kubernetes on a local machine. I’ve even heard of some people using
minikube to run Dev or
QA environments on shared VMs on clouds. This feels a lot like the old devstack
options back in the OpenStack days. Honestly, it has echos of it, and I feel like it’s probably
the best analogy.
Just like any seasoned veteran of the OpenStack ecosystem would say, devstack is a designed for
a specific use case, and that’s how
minikube is targeted too.
minikube was/is the answer
to the question “What is the fastest way I can get kubernetes running on my laptop?” and it
succeeds at this.
You should note though, it’s only one worker node, one machine, relays on Virtualbox or the like for virtualization and has only 2 Gigs of RAM by default to use. This is all but a toy, you can run your typical commands against it, but you can’t run anything that isn’t stock kubernetes.
So how do you fix this, you can run
minikube with a couple other options to give it more resources
on start time.
So how do you fix this? Luckily you have some commands you can run!
minikube --cpus 2 --memory 4024 start
The above, will give your
minikube instance 2 virtual cpus, and 4 Gigs of RAM. This should allow
you to do more with it, like install istio but not much more. If you want to create/test
something closer to your production instance, you should give it more resources. That’s up to your
restrictions and specific situation.
After you’re done with the instance, I’d suggest either shutting it down, or deleting it completely:
If you plan on walking away from your local computer or not needing the pods your are running for long term, this is a good opportunity to do the following to the VM that is running your kubernetes cluster isn’t taking up unneeded resources.
I should say, if you had installed
kubectl we didn’t have
export anything or change any configuation
around, from what I know that’s by design.
kubectl defaults to the localhost that
minikube runs on,
which allows for a small, but valuble, positive UX situation.
docker in docker
Personally this in the choice I make. It’s from the kubernetes-sigs project called kubeadm-dind-cluster. It creates 3 docker containers on your local machine, one master and 2 worker nodes. It’s as close as you can get to what you’d really run in the real world. And because they are just containers they spin up extremely quickly.
All in all, it’s just a handful of commands, for instance:
wget https://github.com/kubernetes-sigs/kubeadm-dind-cluster/releases/download/v0.1.0/dind-cluster-v1.13.sh chmod +x dind-cluster-v1.13.sh
As you can see at my writing of this it’s version
1.13, but it should be the same with whatever
version is released now.
# start the cluster ./dind-cluster-v1.13.sh up # add kubectl directory to PATH export PATH="$HOME/.kubeadm-dind-cluster:$PATH" kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master Ready master 4m v1.13.0 kube-node-1 Ready <none> 2m v1.13.0 kube-node-2 Ready <none> 2m v1.13.0
And that’s it! You now have a fully working kubernetes cluster running inside your docker instance.
# k8s dashboard available at http://localhost:8080/api/v1/namespaces/kube-system/services/kubernetes-dashboard:/proxy # restart the cluster, this should happen much quicker than initial startup ./dind-cluster-v1.13.sh up # stop the cluster ./dind-cluster-v1.13.sh down # remove DIND containers and volumes ./dind-cluster-v1.13.sh clean
At the time of this writing microk8s was a relatively new comer to the space. It’s supported by
Canonical, the company behind Ubuntu. It’s a simple
snap install like the following:
snap install microk8s --classic
Obiviously if you don’t have
snap you’re out of luck, but if you can swing it, this is a way
to get a kuberenetes cluster up and running extremely quickly. One oddity with the system though
is every command is prefixed. Take a look at this example:
microk8s.kubectl get nodes
It has a huge advantage, you don’t have to mess with your
KUBECONFIG to talk to your local
cluster, but if you have been using kubernetes a long time you have to retrain your muscle
memory. Take a look at the [microk8s docs][microdocs] and play around with it.
k3s is the newest way to get kubernetes cluster on a local machine. Supported by rancher, it has huge promise. To quote the Github page:
Lightweight Kubernetes. Easy to install, half the memory, all in a binary less than 40mb.
That’s extremely impressive. But it’s designed for a specific use case not for the original statement of running a kubernetes cluster on your laptop. It’s designed for small environments like IoT or Edge computing. If you have smaller environment it’s worth checking out.
from scratch on a local machine
Ok, so you’ve gotten to this point, maybe nothing has fit your use case, or hell you like my writing style. The following is a very specific use case. You’ve probably come here to spin up kubernetes from scratch on your local machine, first of all, I salute you, second, wow. There are many packaged ways to do what you’re trying to do, and this will only cause heart ache.
I’m assuming you’ve gone through, and edited kubernetes the hardway to work on your cloud of choice. Now you’ve taken it and ran it locally on a VM, and now want bring it to your local dev environment. You might want to run the most recent releases of kubernetes, and have no trouble figuring out what’s going wrong when your application has trouble. You, yes you, are unique. This whole post was not aimed for you, and maybe it opened your eyes to something you haven’t thought of, but in general you, yes you are a pioneer on your own boat. May the wind will always be at your back, and have fun.
Hopefully you can see there are a ton of options here. The crazy part is that this is only as accurate as the Note at the top of the document. Kubernetes is moving fast and becoming the way to run Cloud Native Applications, but running it without a cloud can be tricky. Most of the options that were mentioned here hit most use cases and you’ll need to do your homework to see what fits you. Standardize on something, which will help your teams become more successful when they all use the same base wrapper around this amazing technology.