logo

Lucas Katayama/Local Kubernetes with k3s

Created Wed, 14 Feb 2024 13:23:43 -0300 Modified Wed, 14 Feb 2024 13:23:43 -0300
365 Words

Requirements

  • Docker Desktop
    • brew install --cask docker
  • Helm
    • brew install helm
  • linkerd
    • brew install linkerd

Resource Manager

Kubernetes Cluster (k3s and k3d)

k3d cluster create -p "80:80@loadbalancer" -p "443:443@loadbalancer" --k3s-arg "--disable=traefik@server:0"
title: --k3s-arg "--disable=traefik@server:0"

Disable k3s traefik to use a meshed one

Service Mesh (linkerd)

$ brew install linkerd
$ linkerd check --pre
$ linkerd install --crds | kubectl apply -f -
$ linkerd install | kubectl apply -f -
# For docker-desktop use:
# linkerd install --set proxyInit.runAsRoot=true | kubectl apply -f -
$ linkerd check

O11y (linkerd viz + jaeger + grafana)

$ linkerd jaeger install | kubectl apply -f -
$ helm install grafana -n grafana --create-namespace grafana/grafana -f https://raw.githubusercontent.com/linkerd/linkerd2/main/grafana/values.yaml
$ linkerd viz install --set jaegerUrl=jaeger.linkerd-jaeger:16686 --set grafana.url=grafana.grafana:3000 | kubectl apply -f -

Apply grafana RBAC

apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata:
  namespace: linkerd-viz
  name: grafana
spec:
  targetRef:
    group: policy.linkerd.io
    kind: Server
    name: prometheus-admin
  requiredAuthenticationRefs:
    - kind: ServiceAccount
      name: grafana
      namespace: grafana

Ingress Controller (Traefik)

Kubernetes Proxy

$ helm install traefik traefik/traefik  -n traefik --create-namespace

# inject sidecar
$ kubectl get -n traefik deploy/traefik -o yaml \
  | linkerd inject --ingress  - \
  | kubectl apply -f -

Traefik Jaeger Tracing

$ kubectl edit deploy -n traefik

Add the following params to spec.containers[0].args

- --tracing.jaeger=true
- --tracing.jaeger.collector.endpoint=http://collector.linkerd-jaeger:14268/api/traces?format=jaeger.thrift
- --tracing.jaeger.propagation=b3

Example Service - EmojiVoto

$ curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/emojivoto.yml | linkerd inject - | kubectl apply -f -

# inject sidecar 
$ kubectl get deploy -o yaml -n emojivoto | linkerd inject -  | kubectl apply -f -

# configure tracing
$ kubectl -n emojivoto set env --all deploy OC_AGENT_HOST=collector.linkerd-jaeger:55678

Apply IngressRoute

---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: l5d-header-middleware
  namespace: emojivoto
spec:
  headers:
    customRequestHeaders:
      l5d-dst-override: "web-svc.emojivoto.svc.cluster.local:80"
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: emojivoto-route
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`app.localhost`) && PathPrefix(`/`)
      # middlewares:
      #   - name: l5d-header-middleware
      services:
        - kind: Service
          name: web-svc
          port: 80
          namespace: emojivoto

[!info] Application should be accessible on http://app.localhost

Useful port-forwards

linkerd viz dashboard

$ linkerd viz dashboard

traefik dashboard

$ kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -n traefik) 9000:9000 -n traefik

Result

Deployments

$ k9s -A

O11y

linkerd dashboard

Proxy

References