Installing Longhorn in k3s kubernetes for storage management
May 21, 2023
kapilnakhwa
I have multiple disks loaded into my different nodes and they are all not the same, Hence I needed a reliable disk volume manager for my k3s kubernetes cluster. After few options like using NFS I decided to give longhorn a try.
Installing longhorn was pretty simple.
Add in the longhorn helm repo.
Installing longhorn was pretty simple.
Add in the longhorn helm repo.
❯ helm repo add longhorn https://charts.longhorn.io "longhorn" has been added to your repositories
Gave it a quick update
❯ helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "longhorn" chart repository ...Successfully got an update from the "qjoly" chart repository ...Successfully got an update from the "wekan" chart repository ...Successfully got an update from the "yugabytedb" chart repository ...Successfully got an update from the "traefik" chart repository ...Successfully got an update from the "k8s-at-home" chart repository ...Successfully got an update from the "truecharts" chart repository
Helm install command.
❯ helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.4.2 NAME: longhorn LAST DEPLOYED: Sun May 21 17:56:10 2023 NAMESPACE: longhorn-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Longhorn is now installed on the cluster! Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized. Visit our documentation at https://longhorn.io/docs/
Give it a minute or two and check if pods have been created.
❯ kubectl -n longhorn-system get pod NAME READY STATUS RESTARTS AGE longhorn-conversion-webhook-5c75577484-vcm4g 1/1 Running 0 5m48s longhorn-conversion-webhook-5c75577484-kdt44 1/1 Running 0 5m48s longhorn-ui-5d8569577-2qnlz 1/1 Running 0 5m48s 4 apiVersion: networking.k8s.io/v1 longhorn-ui-5d8569577-t6zzm 1/1 Running 0 5m48s longhorn-recovery-backend-c4c77d7d9-z4zz5 1/1 Running 0 5m48s longhorn-recovery-backend-c4c77d7d9-4tk8x 1/1 Running 0 5m48s longhorn-admission-webhook-65669f9957-d5dlk 1/1 Running 0 5m48s longhorn-manager-4ch7l 1/1 Running 0 5m48s longhorn-manager-2wcnq 1/1 Running 0 5m48s longhorn-driver-deployer-9d85f7675-q2jmm 1/1 Running 0 5m48s longhorn-admission-webhook-65669f9957-j2f7d 1/1 Running 0 5m48s csi-snapshotter-588bd6f85d-htwfs 1/1 Running 0 4m56s csi-resizer-8558456d59-sxs9z 1/1 Running 0 4m57s csi-attacher-7b5d5ff865-pmkwn 1/1 Running 0 4m57s csi-attacher-7b5d5ff865-kwhxj 1/1 Running 0 4m57s longhorn-csi-plugin-v9c6v 3/3 Running 0 4m56s longhorn-csi-plugin-cpqzw 3/3 Running 0 4m56s csi-resizer-8558456d59-668r2 1/1 Running 0 4m57s csi-resizer-8558456d59-qtkwc 1/1 Running 0 4m57s csi-provisioner-68d785644d-59rbj 1/1 Running 0 4m57s csi-attacher-7b5d5ff865-vsnv6 1/1 Running 0 4m57s csi-provisioner-68d785644d-5xzxv 1/1 Running 0 4m57s csi-snapshotter-588bd6f85d-kflqr 1/1 Running 0 4m56s csi-provisioner-68d785644d-pq424 1/1 Running 0 4m57s csi-snapshotter-588bd6f85d-nnvjz 1/1 Running 0 4m56s engine-image-ei-f9e7c473-cqjpg 1/1 Running 0 5m13s engine-image-ei-f9e7c473-jfmqb 1/1 Running 0 5m13s instance-manager-e-fee03a6dad8001a3ed9e950bfe5a1ace 1/1 Running 0 5m13s instance-manager-r-fee03a6dad8001a3ed9e950bfe5a1ace 1/1 Running 0 5m13s instance-manager-e-a7ab7c8a44898b0fc4d4ba6fc8937839 1/1 Running 0 5m11s instance-manager-r-a7ab7c8a44898b0fc4d4ba6fc8937839 1/1 Running 0 5m11s
One immediate problem, I noticed was that longhorn-frontend was installed in ClusterIP mode, which meant it could not be accessed from ourside world. I needed it to be accessed from other devices in my VPN, So
I downloaded the helm values into my computer to be able to change it.
I downloaded the helm values into my computer to be able to change it.
❯ helm show values longhorn/longhorn >> values.yaml
Checking on the values file , I changed the type of UI to NodePort.
service: 14 ui: 13 type: NodePort 12 nodePort: 32323
Then reapply these values using helm.
❯ helm upgrade -n longhorn-system longhorn longhorn/longhorn -f values.yaml Release "longhorn" has been upgraded. Happy Helming! NAME: longhorn LAST DEPLOYED: Sun May 21 18:18:11 2023 NAMESPACE: longhorn-system STATUS: deployed REVISION: 2 TEST SUITE: None NOTES: Longhorn is now installed on the cluster! Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized. Visit our documentation at https://longhorn.io/docs/
Check for the TYPE
❯ kubectl -n longhorn-system get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE longhorn-replica-manager ClusterIP None <none> <none> 37m longhorn-engine-manager ClusterIP None <none> <none> 37m longhorn-backend ClusterIP 10.43.2.160 <none> 9500/TCP 37m longhorn-recovery-backend ClusterIP 10.43.91.212 <none> 9600/TCP 37m longhorn-admission-webhook ClusterIP 10.43.218.102 <none> 9443/TCP 37m longhorn-conversion-webhook ClusterIP 10.43.185.183 <none> 9443/TCP 37m csi-attacher ClusterIP 10.43.72.249 <none> 12345/TCP 37m csi-provisioner ClusterIP 10.43.228.126 <none> 12345/TCP 37m csi-resizer ClusterIP 10.43.97.55 <none> 12345/TCP 37m csi-snapshotter ClusterIP 10.43.86.159 <none> 12345/TCP 37m longhorn-frontend NodePort 10.43.196.87 <none> 80:32323/TCP 37m
Cool, Notice the longhorn front end as TYPE NodePort.
Now i needed to add in a ingressRoute for it to be accessible from my VPN.
Now i needed to add in a ingressRoute for it to be accessible from my VPN.
1 apiVersion: traefik.containo.us/v1alpha1 1 kind: IngressRoute 2 metadata: 3 name: longhorn-ingress-route 4 spec: 5 entryPoints: 6 - web 7 - websecure 8 routes: 9 - kind: Rule 10 match: Host("longhorn.100.111.59.19.sslip.io") 11 services: 12 - name: longhorn-frontend 13 port: 80 ~ ~ ~ ~
Apply the newly created ingress Route
❯ kubectl -n longhorn-system apply -f longhorn-ingress-route.yaml
Test it out in a browser.
I still have a problem though. My added external volumes are not shown yet . Which seemed to be pretty easy to just edit the node and add the mounted path manually.
If you have any questions or would want to solve any issues with helm, k3s or longhorn , Feel free to reach out to me .
I still have a problem though. My added external volumes are not shown yet . Which seemed to be pretty easy to just edit the node and add the mounted path manually.
If you have any questions or would want to solve any issues with helm, k3s or longhorn , Feel free to reach out to me .