Microk8s with EKS-D

Deploy HyperTest on self managed k8s cluster using Microk8s and EKSD

Tech Stack Overview:

Microk8s: MicroK8s is a light weight kubernetes tool to easily get a multi-node highly available kubernetes cluster

EKS-D: EKS-D Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (Amazon EKS).

Prerequisites

1. Create Ubuntu VM's (>=22.04 LTS)

Recommended minimum resources required to run HyperTest on a VM are as follows:

CPU	RAM	Disk
4vCPU	16GB RAM	100GB Disk

You should have root user access of Ubuntu VM's

Getting Started

Cluster Setup

1. Installation of Microk8s

• SSH into ubuntu vm

• Run the below command for installation of microk8s

sudo snap install microk8s --classic --channel 1.25-eksd/stable

use `snap info microk8s|grep eksd` to check for latest releases

2. Enabling EBS CSI Driver

1. Create an IAM user with programmatic access in your AWS account

2. Note the access-key-id and secret-access-key of the user

3. Attach the below policy to that user by going to Permissions and searching for below policy Name: AmazonEBSCSIDriverPolicy Policy ARN: arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy

1. Now enable CSI driver in your microk8s cluster using the credentials for the user we just created. We have given the user the access to use Amazon EBS CSI Driver. We will just configure microk8s to use that user for CSI Driver.

sudo microk8s enable aws-ebs-csi-driver -k <access-key-id> -a <secret-access-key>

3. Enable Ingress plugin

Enable microk8s ingress plugin using the below command

sudo microk8s enable ingress

4. Add Worker Node (Optional)

If you have created two seperate node for worker and master HyperTest node. You have to join your worker node with master to make that one k8s cluster

1. SSH into master node vm

2. Execute below command to obtain joining token

sudo microk8s add-node| grep -m1 " --worker"

• From the output copy the line “microk8s join –worker” to join other node as a worker node, else remove –worker flag from end to join as a master node and create a highly available cluster

• SSH into worker node

• Run the microk8s join command copied from above (use sudo if required)

• Do this from all VM’s you want to join to microk8s cluster

5. Label Nodes

If you have created a single node for hypertest master and worker node, then you will have to label the VM with labels for both

### To get node-name 
sudo microk8s kubectl get no

### For hypertest master node
sudo microk8s kubectl label nodes <node-name> hypertest_master_node=yes
### For hypertest worker nodes
sudo microk8s kubectl label nodes <node-name> hypertest_worker_node=yes
### Taints for both hypertest node types
sudo microk8s kubectl taint nodes <node-name> hypertest_node=yes:NoExecute

## To verify labels
sudo microk8s kubectl get nodes --show-labels

6. Storage and Ingress Classes

• SSH into master node

• Create ht-ingress-storage-classes.yaml file with as shown below

ht-ingress-storage-classes.yaml

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: hypertest-ingress-class
spec:
  controller: k8s.io/ingress-nginx  #specify your controller name here. eg: k8s.io/ingress-nginx for nginx-ingress-controller https://kubernetes.github.io/ingress-nginx/
---

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
    name: hypertest-storage-class
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
provisioner: ebs.csi.aws.com #specify your provisioner name here. eg: microk8s.io/hostpath
parameters: # specify parameters for your storage class here
  type: gp3
allowVolumeExpansion: true

Create the ingress and storage class using the following command

sudo microk8s kubectl apply -f ht-ingress-storage-classes.yaml

After creating ingress and storage classes, we will now deploy HyperTest controller. Please refer the below guide for the same. You can skip the pre-reqs and directly jump to

https://docs.hypertest.co/detailed-setup-guide/detailed-setup-guide/installation/kubernetes-cluster/hypertest-installation#deploy-hypertest-controller-service

HyperTest Installation