Install on generic Kubernetes
Installing Dotmesh on a generic Kubernetes cluster
- A Kubernetes cluster (version >= 1.7)
Before we can install Dotmesh, we need to set out admin password and api key:
export ADMIN_PASSWORD=applesinsecurePassword123
export ADMIN_API_KEY=applesinsecurePassword123You may want to use stronger credentials.
Credentials
Then we create the namespace before adding our credentials as secrets:
kubectl create namespace dotmesh
echo -n $ADMIN_PASSWORD > dotmesh-admin-password.txt
echo -n $ADMIN_API_KEY > dotmesh-api-key.txt
kubectl create secret generic dotmesh \
--from-file=./dotmesh-admin-password.txt \
--from-file=./dotmesh-api-key.txt -n dotmesh
rm -f dotmesh-admin-password.txt dotmesh-api-key.txt
namespace "dotmesh" created
secret "dotmesh" createdEtcd
Dotmesh relies on coreos etcd
operator to set
up our own dedicated etcd cluster in the dotmesh namespace. We’ve
mirrored a copy of the etcd operator YAML (part
1, part
2), but it’s
unmodified so feel free to use your own.
kubectl apply -f https://get.dotmesh.io/yaml/etcd-operator-clusterrole.yaml
kubectl apply -f https://get.dotmesh.io/yaml/etcd-operator-dep.yaml
clusterrolebinding "dotmesh-etcd-operator" configured
deployment "etcd-operator" configuredIt may take a few minutes for the etcd operator to activate.
Use kubectl get pods -n dotmesh to check for a running etcd-operator pod.
Create an etcd cluster for dotmesh to use
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-etcd-cluster.yaml
etcdcluster "dotmesh-etcd-cluster" configuredDotmesh
Next, you must create the ConfigMap. If you want to customise it, please see the Kubernetes YAML reference guide.
Dotmesh operates in two storage modes:
- Local Storage mode
By default, Dotmesh running under Kubernetes is configured to store data on the host filesystem. It’s simple to get started with, but not ideal for production environments. For more information please refer to the configuration options for Local Storage mode
- PVCPerNode mode
In this mode, Dotmesh uses cloud-native storage for reliable persistent storage ideal for deployments in cloud environments. Please see this blog post on Dotmesh on cloud Kubernetes clusters: PV-per-Node Mode for more information about the motivation and technical implementation using Kubernetes PersistentVolumeClaims. Seriously consider using this mode for a production cluster, to enable this mode the ConfigMap with configuration options for PVCPerNode mode.
kubectl apply -f https://get.dotmesh.io/yaml/configmap.yaml
With the configmap created, installing Dotmesh is a simple matter of loading the Dotmesh YAML:
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-k8s-1.7.yaml
NOTE if you are using Kubernetes > 1.8 then use the following URL:
kubectl apply -f https://get.dotmesh.io/yaml/dotmesh-k8s-1.8.yaml
serviceaccount "dotmesh" configured
serviceaccount "dotmesh-operator" configured
clusterrole "dotmesh" configured
clusterrolebinding "dotmesh" configured
clusterrolebinding "dotmesh-operator" configured
service "dotmesh" configured
deployment "dotmesh-operator" configured
serviceaccount "dotmesh-provisioner" configured
clusterrole "dotmesh-provisioner-runner" configured
clusterrolebinding "dotmesh-provisioner" configured
deployment "dotmesh-dynamic-provisioner" configured
storageclass "dotmesh" configuredBy default, that will install Dotmesh on every node in your
cluster. Dot storage will be in a 10GiB file created in
/var/lib/dotmesh in the host filesystem; that’s fine for light
usage, but if you are likely to have more than 10GiB of data to deal
with in total, you’ll want to override that by creating a ZFS pool
called pool on each of your nodes before installing Dotmesh. Dotmesh
will use that pool for Dot storage if it finds it already existing at
install time.
Checking the cluster
Let’s check to see that we have our dotmesh pods running. They might take a few moments to get going - wait for the pods to start before proceeding. Here is an example of the desired outcome on a 3 node cluster. Your results may vary depending on your cluster size.
kubectl get po -n dotmesh
NAME READY STATUS RESTARTS AGE
etcd-operator-56b49b7ffd-529zn 1/1 Running 0 1h
dotmesh-etcd-cluster-0000 1/1 Running 0 1h
dotmesh-etcd-cluster-0001 1/1 Running 0 1h
dotmesh-etcd-cluster-0002 1/1 Running 0 1h
dotmesh-operator-7ff894567-mx75b 1/1 Running 0 1h
server-node1 1/1 Running 0 1h
server-node2 1/1 Running 0 1h
server-node3 1/1 Running 0 1h
dotmesh-dynamic-provisioner-7b766c4f7f-hkjkl 1/1 Running 0 1hRestart Kubelet
To get the kubelet to pick up the flexvolume driver dotmesh just installed - log into each of the nodes and restart the kubelet process:
sudo systemctl restart kubelet
NOTE In Kubernetes 1.8 restarting the kubelet will not be needed
Customising the installation
If you want a non-default installation - for instance, only running Dotmesh on those of your nodes that have capacious fast disks, as those are the only ones where stateful containers will reside - the YAML we supply is easy to customise. Check out the Kubernetes YAML reference guide for the full run-down!
Using the dm client to control Dotmesh
In order to manage branches and commits, push and pull dots, and so
on, you’ll need to connect the dm client to your Kubernetes-hosted
Dotmesh cluster. To do that, you’ll need the API key you chose in the
setup phase, and the hostname of a node in the cluster:
dm remote add NAME [email protected]
API key: Paste your API key here, it won't be echoed!
Remote added.The NAME is just a name for this cluster that you’ll use in
subsequent dm remote
commands, so pick something
that describes it.
You can then switch to that remote, and use it:
dm remote switch NAME
dm list