Install & Setup

Install on generic Kubernetes

Installing Dotmesh on a generic Kubernetes cluster

You're gonna need.
  • A Kubernetes cluster (version >= 1.7)

Before we can install Dotmesh, we need to set out admin password and api key:

export ADMIN_PASSWORD=applesinsecurePassword123
export ADMIN_API_KEY=applesinsecurePassword123

You may want to use stronger credentials.


Then we create the namespace before adding our credentials as secrets:

kubectl create namespace dotmesh
echo -n $ADMIN_PASSWORD > dotmesh-admin-password.txt
echo -n $ADMIN_API_KEY > dotmesh-api-key.txt
kubectl create secret generic dotmesh \
  --from-file=./dotmesh-admin-password.txt \
  --from-file=./dotmesh-api-key.txt -n dotmesh
rm -f dotmesh-admin-password.txt dotmesh-api-key.txt
namespace "dotmesh" created
secret "dotmesh" created


Dotmesh relies on coreos etcd operator to set up our own dedicated etcd cluster in the dotmesh namespace. We’ve mirrored a copy of the etcd operator YAML (part 1, part 2), but it’s unmodified so feel free to use your own.

kubectl apply -f
kubectl apply -f
clusterrolebinding "dotmesh-etcd-operator" configured
deployment "etcd-operator" configured

It may take a few minutes for the etcd operator to activate. Use kubectl get pods -n dotmesh to check for a running etcd-operator pod.

Create an etcd cluster for dotmesh to use

kubectl apply -f
etcdcluster "dotmesh-etcd-cluster" configured


Next, you must create the ConfigMap. If you want to customise it, please see the Kubernetes YAML reference guide.

Dotmesh operates in two storage modes:

  • Local Storage mode

By default, Dotmesh running under Kubernetes is configured to store data on the host filesystem. It’s simple to get started with, but not ideal for production environments. For more information please refer to the configuration options for Local Storage mode

  • PVCPerNode mode

In this mode, Dotmesh uses cloud-native storage for reliable persistent storage ideal for deployments in cloud environments. Please see this blog post on Dotmesh on cloud Kubernetes clusters: PV-per-Node Mode for more information about the motivation and technical implementation using Kubernetes PersistentVolumeClaims. Seriously consider using this mode for a production cluster, to enable this mode the ConfigMap with configuration options for PVCPerNode mode.

kubectl apply -f

With the configmap created, installing Dotmesh is a simple matter of loading the Dotmesh YAML:

kubectl apply -f

NOTE if you are using Kubernetes > 1.8 then use the following URL:

kubectl apply -f
serviceaccount "dotmesh" configured
serviceaccount "dotmesh-operator" configured
clusterrole "dotmesh" configured
clusterrolebinding "dotmesh" configured
clusterrolebinding "dotmesh-operator" configured
service "dotmesh" configured
deployment "dotmesh-operator" configured
serviceaccount "dotmesh-provisioner" configured
clusterrole "dotmesh-provisioner-runner" configured
clusterrolebinding "dotmesh-provisioner" configured
deployment "dotmesh-dynamic-provisioner" configured
storageclass "dotmesh" configured

By default, that will install Dotmesh on every node in your cluster. Dot storage will be in a 10GiB file created in /var/lib/dotmesh in the host filesystem; that’s fine for light usage, but if you are likely to have more than 10GiB of data to deal with in total, you’ll want to override that by creating a ZFS pool called pool on each of your nodes before installing Dotmesh. Dotmesh will use that pool for Dot storage if it finds it already existing at install time.

Checking the cluster

Let’s check to see that we have our dotmesh pods running. They might take a few moments to get going - wait for the pods to start before proceeding. Here is an example of the desired outcome on a 3 node cluster. Your results may vary depending on your cluster size.

kubectl get po -n dotmesh
NAME                                           READY     STATUS        RESTARTS   AGE
etcd-operator-56b49b7ffd-529zn                 1/1       Running       0          1h
dotmesh-etcd-cluster-0000                      1/1       Running       0          1h
dotmesh-etcd-cluster-0001                      1/1       Running       0          1h
dotmesh-etcd-cluster-0002                      1/1       Running       0          1h
dotmesh-operator-7ff894567-mx75b               1/1       Running       0          1h
server-node1                                   1/1       Running       0          1h
server-node2                                   1/1       Running       0          1h
server-node3                                   1/1       Running       0          1h
dotmesh-dynamic-provisioner-7b766c4f7f-hkjkl   1/1       Running       0          1h

Restart Kubelet

To get the kubelet to pick up the flexvolume driver dotmesh just installed - log into each of the nodes and restart the kubelet process:

sudo systemctl restart kubelet

NOTE In Kubernetes 1.8 restarting the kubelet will not be needed

Customising the installation

If you want a non-default installation - for instance, only running Dotmesh on those of your nodes that have capacious fast disks, as those are the only ones where stateful containers will reside - the YAML we supply is easy to customise. Check out the Kubernetes YAML reference guide for the full run-down!

Using the dm client to control Dotmesh

In order to manage branches and commits, push and pull dots, and so on, you’ll need to connect the dm client to your Kubernetes-hosted Dotmesh cluster. To do that, you’ll need the API key you chose in the setup phase, and the hostname of a node in the cluster:

dm remote add NAME admin@HOSTNAME
API key: Paste your API key here, it won't be echoed!

Remote added.

The NAME is just a name for this cluster that you’ll use in subsequent dm remote commands, so pick something that describes it.

You can then switch to that remote, and use it:

dm remote switch NAME
dm list

What’s next?