Microk8s connects to microCeph

Connect MicroK8s cluster to existing MicroCeph cluster

Set up with Ansible

See: Connect existing MicroCeph cluster section in Set up Microk8s Kubernetes cluster and Rancher on NREC with Ansible.

Set up manually

1. make sure the rbd kernel module is loaded

sudo modprobe rbd

2. Enable rook-ceph

sudo microk8s enable rook-ceph

Then, we need to wait for pod rook-ceph-operator in namespace rook-ceph to be ready/active.

3. Add rook-release repo

sudo microk8s helm3 repo add rook-release https://charts.rook.io/release

4. Connect MicroK8s with any external Ceph cluster

sudo microk8s connect-external-ceph --ceph-conf /var/snap/microceph/current/conf/ceph.conf --keyring /var/snap/microceph/current/conf/ceph.keyring --rbd-pool microk8s-rbd

After running the command, it shows:

=================================================

Successfully imported external Ceph cluster. You can now use the following storageclass
to provision PersistentVolumes using Ceph CSI:

NAME       PROVISIONER                  RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
ceph-rbd   rook-ceph.rbd.csi.ceph.com   Delete          Immediate           true                   1s

5. testing

kubectl --namespace rook-ceph get pods -l "app=rook-ceph-operator"

kubectl --namespace rook-ceph-external get cephcluster

For testing:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
spec:
  storageClassName: ceph-rbd
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
    - name: test-container
      image: nginx
      volumeMounts:
        - mountPath: "/var/lib/www/html"
          name: vol
  volumes:
    - name: vol
      persistentVolumeClaim:
        claimName: test-pvc

References

1. HowTo setup MicroK8s with (Micro)Ceph storage