BETA You’re responsible for monitoring and scaling your self-hosted 3B deployment. Use this guide at your own risk and test changes before applying them in production.
The autoscaling.enabled setting currently supports only KEDA. Install and verify KEDA before setting it to true in 3B v10.2.0 or later.
KEDA 2.21.1 provides the external metrics API for 3B autoscaling. The Grafana dashboard is optional.
On a fresh cluster, install KEDA in kube-system. Reuse an existing release in keda. With the shared stack, 3B sends metrics to the Collector and KEDA queries Prometheus, both in monitoring. You enable scaling later in the 3B workload namespace.
Check for an existing install
Use Bash, Helm, kubectl, and Python 3. For a Helm deployment, use your workstation’s cluster context. For a bootstrap install, run this on the K3s server with access to its kubeconfig.
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
Set your 3B namespace and check the external metrics provider.
app_namespace=3b kubectl config current-context helm list --all-namespaces kubectl get apiservice v1beta1.external.metrics.k8s.io --ignore-not-found \ -o custom-columns='NAMESPACE:.spec.service.namespace,SERVICE:.spec.service.name'
Select the deployed keda release’s namespace, or kube-system for a fresh install.
keda_namespace=$(helm list --deployed --all-namespaces --filter '^keda$' -o json | python3 -c ' import json, sys releases = json.load(sys.stdin) print(releases[0]["namespace"] if releases else "kube-system") ') printf 'KEDA namespace: %s\n' "$keda_namespace"
For an existing release, skip to Verify KEDA. To add the dashboard to keda-2.21.1, use the command below. For other versions, apply the dashboard settings at your current chart version. If a different provider owns the external metrics API, check with your cluster administrator first. Only one Service can provide that API.
Install KEDA and its dashboard
The dashboard needs Prometheus Operator’s ServiceMonitors and Grafana’s dashboard sidecar. Both come with the K3s and Kubernetes with Helm monitoring stacks. For a custom stack, match its ServiceMonitor labels and Grafana data source UID. If your workload namespace differs from 3b, update app_namespace and the dashboard’s replicas query.
Add the chart repository locally. This fixes repo kedacore not found even when helm list --all-namespaces shows an installed release.
helm repo add kedacore https://kedacore.github.io/charts --force-update
To use a dedicated keda namespace on a fresh install, set keda_namespace=keda first. For the controller alone, skip to Install KEDA without the dashboard.
Paste the full block into your terminal. If you maintain a KEDA values file, merge the YAML into it with vi, vim, or nano, keep existing extraObjects entries, and pass the merged file to Helm with --values instead.
helm upgrade --install keda kedacore/keda \
--version 2.21.1 --namespace "${keda_namespace:?Run the namespace check first}" \
--create-namespace --reuse-values --wait --timeout 10m --values - <<'YAML'
resources:
operator:
requests: {cpu: 100m, memory: 100Mi}
metricServer:
requests: {cpu: 100m, memory: 100Mi}
webhooks:
requests: {cpu: 100m, memory: 100Mi}
prometheus:
operator: &scrape
enabled: true
serviceMonitor:
enabled: true
additionalLabels: {release: monitoring}
metricServer: *scrape
webhooks: *scrape
extraObjects:
- apiVersion: v1
kind: ConfigMap
metadata:
name: keda-overview
labels:
grafana_dashboard: "1"
data:
keda-overview.json: |
{
"uid": "keda-lab",
"title": "KEDA overview",
"schemaVersion": 39,
"refresh": "30s",
"time": {"from": "now-1h", "to": "now"},
"panels": [
{
"id": 1, "type": "stat", "title": "Scrape health (1 = up)",
"gridPos": {"x": 0, "y": 0, "w": 24, "h": 4},
"datasource": {"type": "prometheus", "uid": "prometheus"},
"options": {"colorMode": "none", "textMode": "value_and_name"},
"targets": [{"refId": "A", "instant": true, "range": false, "expr": "min by (service) (up{namespace=\"{{ .Release.Namespace }}\",service=~\"keda-operator|keda-operator-metrics-apiserver|keda-admission-webhooks\",endpoint=\"metrics\"})"}]
},
{
"id": 2, "type": "timeseries", "title": "KEDA pod memory",
"gridPos": {"x": 0, "y": 4, "w": 12, "h": 8},
"datasource": {"type": "prometheus", "uid": "prometheus"},
"fieldConfig": {"defaults": {"unit": "bytes"}, "overrides": []},
"targets": [{"refId": "A", "expr": "sum by (pod) (container_memory_working_set_bytes{namespace=\"{{ .Release.Namespace }}\",pod=~\"keda-(operator|admission-webhooks)-.+\",job=\"kubelet\",metrics_path=\"/metrics/cadvisor\",container!=\"\",container!=\"POD\"})"}]
},
{
"id": 3, "type": "timeseries", "title": "3B replicas managed by KEDA",
"gridPos": {"x": 12, "y": 4, "w": 12, "h": 8},
"datasource": {"type": "prometheus", "uid": "prometheus"},
"targets": [{"refId": "A", "expr": "max by (horizontalpodautoscaler) (kube_horizontalpodautoscaler_status_current_replicas{namespace=\"3b\",horizontalpodautoscaler=~\"keda-hpa-.+\"})"}]
},
{
"id": 4, "type": "timeseries", "title": "Edge admission utilization (50% target)",
"gridPos": {"x": 0, "y": 12, "w": 12, "h": 8},
"datasource": {"type": "prometheus", "uid": "prometheus"},
"fieldConfig": {"defaults": {"unit": "percent", "min": 0, "max": 100, "displayName": "${__field.labels.admission}"}, "overrides": []},
"targets": [{"refId": "A", "expr": "avg by (stack, admission) (avg_over_time(edge_admission_utilization{admission=~\"execution|response\"}[1m]))"}]
},
{
"id": 5, "type": "timeseries", "title": "Worker slots wanted",
"gridPos": {"x": 12, "y": 12, "w": 12, "h": 8},
"datasource": {"type": "prometheus", "uid": "prometheus"},
"fieldConfig": {"defaults": {"unit": "suffix: slots", "min": 0, "decimals": 0, "displayName": "${__field.labels.stack}"}, "overrides": []},
"targets": [{"refId": "A", "expr": "max by (stack) (last_over_time(orchestrator_worker_slots_wanted[30s]))"}]
}
]
}
YAML
Expect STATUS: deployed. Prometheus discovers the three ServiceMonitors labeled release: monitoring. Grafana watches all namespaces for ConfigMaps labeled grafana_dashboard: "1".
Install KEDA without the dashboard
For a fresh controller with an existing metrics backend, the chart defaults work without Grafana or ServiceMonitors. If you installed the dashboard above, continue to Verify KEDA.
helm upgrade --install keda kedacore/keda \
--version 2.21.1 --namespace "${keda_namespace:?Run the namespace check first}" \
--create-namespace --wait --timeout 10m
Verify KEDA
kubectl -n "$keda_namespace" get pods -l app.kubernetes.io/instance=keda kubectl -n "$keda_namespace" top pods -l app.kubernetes.io/instance=keda kubectl wait --for=condition=Available apiservice/v1beta1.external.metrics.k8s.io --timeout=3m kubectl get --raw '/apis/external.metrics.k8s.io/v1beta1' kubectl -n "$app_namespace" get scaledobjects.keda.sh,hpa
Expect three Running pods with all containers ready, condition met, and an APIResourceList with groupVersion set to external.metrics.k8s.io/v1beta1. An empty ScaledObject list is normal before you enable 3B autoscaling.
Open the KEDA dashboard
If you installed the dashboard, check its ConfigMap and scrape results.
kubectl -n "$keda_namespace" get configmap keda-overview
kubectl -n monitoring exec statefulset/prometheus-monitoring-kube-prometheus-prometheus -c prometheus -- \
/bin/promtool query instant http://localhost:9090 \
"min by (service) (up{namespace=\"$keda_namespace\",service=~\"keda-operator|keda-operator-metrics-apiserver|keda-admission-webhooks\",endpoint=\"metrics\"})"
After the first scrapes, expect a row with value 1 for each of keda-operator, keda-operator-metrics-apiserver, and keda-admission-webhooks.
For NotFound, confirm the release namespace in helm list --all-namespaces. Only the optional dashboard values create the keda-overview ConfigMap. Apply them to your existing release.
Open Grafana on K3s or Kubernetes with Helm. Search Dashboards for KEDA overview, or open http://127.0.0.1:3000/d/keda-lab/keda-overview with the default port-forward.
KEDA scrape health and pod memory appear first. The 3B panels show No data until you enable autoscaling. Then compare Worker slots wanted with 3B replicas managed by KEDA. Zero slots wanted is normal at idle. Common questions about 3B autoscaling
Enable autoscaling for 3B
Once KEDA is ready, add these values to your 3B release. They use the Collector and Prometheus in monitoring. Adjust the URLs if your endpoints differ.
autoscaling: enabled: true otlpMetricsEndpoint: http://otel-collector.monitoring.svc:4318/v1/metrics prometheusUrl: http://monitoring-kube-prometheus-prometheus.monitoring.svc:9090 edge: maxReplicas: 2 worker: maxReplicas: 3
Your existing edge.replicas and worker.replicas set the minimums. With autoscaling, 3B uses worker.maxReplicas to set its execution limit. At one slot per worker, three ready workers can run three executions at once.
