Installing Dotmesh on a generic Kubernetes cluster
- A Kubernetes cluster (version >= 1.7)
Before we can install Dotmesh, we need to set out admin password and api key:
export ADMIN_PASSWORD=applesinsecurePassword123 export ADMIN_API_KEY=applesinsecurePassword123
You may want to use stronger credentials.
Then we create the namespace before adding our credentials as secrets:
kubectl create namespace dotmesh echo -n $ADMIN_PASSWORD > dotmesh-admin-password.txt echo -n $ADMIN_API_KEY > dotmesh-api-key.txt kubectl create secret generic dotmesh \ --from-file=./dotmesh-admin-password.txt \ --from-file=./dotmesh-api-key.txt -n dotmesh rm -f dotmesh-admin-password.txt dotmesh-api-key.txt
namespace "dotmesh" created secret "dotmesh" created
Dotmesh relies on coreos etcd
operator to set
up our own dedicated etcd cluster in the
dotmesh namespace. We’ve
mirrored a copy of the etcd operator YAML (part
2), but it’s
unmodified so feel free to use your own.
kubectl apply -f https://get.dotmesh.io/yaml/etcd-operator-clusterrole.yaml kubectl apply -f https://get.dotmesh.io/yaml/etcd-operator-dep.yaml
clusterrolebinding "dotmesh-etcd-operator" configured deployment "etcd-operator" configured
It may take a few minutes for the etcd operator to activate.
kubectl get pods -n dotmesh to check for a running
Create an etcd cluster for dotmesh to use
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-etcd-cluster.yaml
etcdcluster "dotmesh-etcd-cluster" configured
Next, you must create the ConfigMap. If you want to customise it, please see the Kubernetes YAML reference guide.
Dotmesh operates in two storage modes:
- Local Storage mode
By default, Dotmesh running under Kubernetes is configured to store data on the host filesystem. It’s simple to get started with, but not ideal for production environments. For more information please refer to the configuration options for Local Storage mode
- PVCPerNode mode
In this mode, Dotmesh uses cloud-native storage for reliable persistent storage ideal for deployments in cloud environments. Please see this blog post on Dotmesh on cloud Kubernetes clusters: PV-per-Node Mode for more information about the motivation and technical implementation using Kubernetes PersistentVolumeClaims. Seriously consider using this mode for a production cluster, to enable this mode the ConfigMap with configuration options for PVCPerNode mode.
kubectl apply -f https://get.dotmesh.io/yaml/configmap.yaml
With the configmap created, installing Dotmesh is a simple matter of loading the Dotmesh YAML:
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-k8s-1.7.yaml
NOTE if you are using Kubernetes >
1.8 then use the following URL:
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-k8s-1.8.yaml
serviceaccount "dotmesh" configured serviceaccount "dotmesh-operator" configured clusterrole "dotmesh" configured clusterrolebinding "dotmesh" configured clusterrolebinding "dotmesh-operator" configured service "dotmesh" configured deployment "dotmesh-operator" configured serviceaccount "dotmesh-provisioner" configured clusterrole "dotmesh-provisioner-runner" configured clusterrolebinding "dotmesh-provisioner" configured deployment "dotmesh-dynamic-provisioner" configured storageclass "dotmesh" configured
By default, that will install Dotmesh on every node in your
cluster. Dot storage will be in a 10GiB file created in
/var/lib/dotmesh in the host filesystem; that’s fine for light
usage, but if you are likely to have more than 10GiB of data to deal
with in total, you’ll want to override that by creating a ZFS pool
pool on each of your nodes before installing Dotmesh. Dotmesh
will use that pool for Dot storage if it finds it already existing at
Checking the cluster
Let’s check to see that we have our dotmesh pods running. They might take a few moments to get going - wait for the pods to start before proceeding. Here is an example of the desired outcome on a 3 node cluster. Your results may vary depending on your cluster size.
kubectl get po -n dotmesh
NAME READY STATUS RESTARTS AGE etcd-operator-56b49b7ffd-529zn 1/1 Running 0 1h dotmesh-etcd-cluster-0000 1/1 Running 0 1h dotmesh-etcd-cluster-0001 1/1 Running 0 1h dotmesh-etcd-cluster-0002 1/1 Running 0 1h dotmesh-operator-7ff894567-mx75b 1/1 Running 0 1h server-node1 1/1 Running 0 1h server-node2 1/1 Running 0 1h server-node3 1/1 Running 0 1h dotmesh-dynamic-provisioner-7b766c4f7f-hkjkl 1/1 Running 0 1h
To get the kubelet to pick up the flexvolume driver dotmesh just installed - log into each of the nodes and restart the kubelet process:
sudo systemctl restart kubelet
NOTE In Kubernetes 1.8 restarting the kubelet will not be needed
Customising the installation
If you want a non-default installation - for instance, only running Dotmesh on those of your nodes that have capacious fast disks, as those are the only ones where stateful containers will reside - the YAML we supply is easy to customise. Check out the Kubernetes YAML reference guide for the full run-down!
dm client to control Dotmesh
In order to manage branches and commits, push and pull dots, and so
on, you’ll need to connect the
dm client to your Kubernetes-hosted
Dotmesh cluster. To do that, you’ll need the API key you chose in the
setup phase, and the hostname of a node in the cluster:
dm remote add NAME [email protected]
API key: Paste your API key here, it won't be echoed! Remote added.
NAME is just a name for this cluster that you’ll use in
subsequent dm remote
commands, so pick something
that describes it.
You can then switch to that remote, and use it:
dm remote switch NAME dm list