Your submission was sent successfully! Close

Jump to main content

Zero-ops scaling Kubernetes storage with MicroK8s and OpenEBS Mayastor

This is a guest post, originally posted on Angelos Kolaitispersonal blog, reproduced here with his permission. Angelos is a MicroK8s software engineer at Canonical.

How we brought a universal, cluster-wide storage solution to MicroK8s.

Why is storage in Kubernetes hard?

Kubernetes is well-known as an open-source system for automating deployment, scaling and management of containerised applications. As such, it uses concepts such as Pods and Deployments to abstract away details regarding the underlying compute, networking and storage infrastructure.

Storage is abstracted as PersistentVolumes (a volume that is provisioned in the underlying infrastructure) and PersistentVolumeClaims (a claim for a pod that uses a persistent volume) resources. Typically, running a Kubernetes cluster also involves running a CSI provisioner, which watches for PersistentVolumeClaims and automatically provisions the requested volumes.

There are a large number of CSI drivers available. For example, if running Kubernetes on top of AWS, one would most likely use the aws-ebs-csi-driver. If running Kubernetes on-premises or in an OpenStack cloud, cinder-csi-driver would be more appropriate. If running on-premises and a Ceph cluster is available, ceph-csi could also be used.

While this solves the storage problems for Kubernetes users, it moves the hurdle of managing everything to the administrators of the Kubernetes cluster, as well as the Kubernetes distribution.

So, the question is, how can storage be made simpler for the administrator of the Kubernetes cluster?

Introducing Mayastor

Mayastor is currently under-development as a sub project of the Open Source CNCF project OpenEBS.

Mayastor can be deployed on any Kubernetes cluster and provide Kubernetes-native storage. It can also use existing block devices on the Kubernetes cluster nodes (if available), as well as image files.

Further, the Mayastor data plane supports replication. You can replicate your volume data across multiple pools, to ensure redundancy in case of a single node failure.

In general, Mayastor is a very interesting solution for Kubernetes-native, fast, redundant storage that works universally across any Kubernetes cluster.

Sounds interesting! How do I try this out?

Deploy Mayastor

NOTE: For the latest instructions, refer to the mayastor addon page.

The mayastor addon will be available with MicroK8s 1.24. You can already try it out by installing MicroK8s from latest/edge:

  1. Enable nvme_tcp and HugePages:sudo apt-get install linux-modules-extra-$(uname -r) sudo modprobe nvme_tcp sudo sysctl vm.nr_hugepages=1024
  2. Install MicroK8s from latest/edge:sudo snap install microk8s --classic --channel latest/edge
  3. Enable Mayastor:sudo microk8s enable core/mayastor

Deploy a test workload

Now that mayastor is deployed, it is time to test it using an example pod:

# pod-with-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
  name: my-pvc
  storageClassName: mayastor
  accessModes: [ReadWriteOnce]
  resources: { requests: { storage: 5Gi } }
apiVersion: v1
kind: Pod
  name: nginx
    - name: pvc
        claimName: my-pvc
    - name: nginx
      image: nginx
        - containerPort: 80
        - name: pvc
          mountPath: /usr/share/nginx/html

Create the pod with:

sudo microk8s.kubectl create -f pod-with-pvc.yaml

Then, we can check to see that Mayastor has created our persistent volume, and the pod has come up:

sudo microk8s.kubectl get pod,pvc

The output should look like this:

pod/nginx   1/1     Running   0          4m

NAME                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/my-pvc   Bound    pvc-e280b734-3224-4af3-af0b-e7ad3c4e6d79   5Gi        RWO            mayastor       4m

Conclusion and future steps

The new mayastor addon in MicroK8s is a very simple way to provision fast, replicated and redundant (though not highly-available) persistent volumes in Kubernetes.

The mayastor addon is considered beta as of MicroK8s 1.24 release. You can already try it by installing from the latest/edge track.

Future steps include:

  • Scale and stress testing.
  • Graduate addon from beta to stable status.
  • Refine the mayastor addon. Right now it automatically provisions image-file backed pools. but we want to make the addon easier to configure for MicroK8s clusters with designated block devices for storage.
  • Compare Mayastor with Rook (Ceph) and other Kubernetes native storage solutions.
  • Document and test backup and restore of mayastor volumes.
  • Support arm64 architectures. This is pending support in mayastor

Photo by Darrin Moore on Unsplash

Newsletter signup

Select topics you're
interested in

In submitting this form, I confirm that I have read and agree to Canonical's Privacy Notice and Privacy Policy.

Related posts

Kubernetes로 5G의 잠재력 활용하기

5세대 무선 기술인 5G는 우리가 연결하고 소통하는 방식을 변화하고 있습니다. 4G보다 최대 100배 빠른 데이터 전송 속도를 제공하는 5G 기술은 더 빠른 다운로드 및 업로드 시간, 낮은 latency 그리고 더 많은 기기의 연결이 가능합니다. 추가로 5G 네트워크는 고대역 및...

MicroK8s is now on AWS marketplace

MicroK8s is now on AWS marketplace Everyone knows that MicroK8s is an extremely lightweight, extensible, reliable, CNCF-compliant distribution of Kubernetes....

Harnessing the potential of 5G with Kubernetes

Harnessing the potential of 5G with Kubernetes 5G is the fifth generation of wireless technology which is transforming the way we connect and communicate....