BETA You’re responsible for monitoring and scaling your self-hosted 3B deployment. Use this guide at your own risk and test changes before applying them in production.
Install Prometheus, Grafana, and the OpenTelemetry Collector as private ClusterIP Services in monitoring. If your cluster already has monitoring, go to Monitor a 3B Helm release.
What do I need before installing?
Use Bash on a workstation with Helm, kubectl, and permission to install cluster-wide resources, including Prometheus Operator custom resource definitions.
kubectl config current-context helm list --all-namespaces kubectl get storageclass
Confirm the 3B cluster context and check with your team for monitoring managed outside Helm. The default StorageClass must support a 20 GiB ReadWriteOnce claim. To select another, set storageClassName beside accessModes below. Kubernetes storage classes
On one node, the stack requests 660m CPU and 1,664 MiB of RAM. Add 50m CPU and 32 MiB per additional node running node-exporter. Prometheus usage depends on time series and query load.
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts --force-update helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts --force-update
Install Prometheus and Grafana
Paste each Helm install block into your terminal, including the closing YAML line.
Prometheus keeps up to seven days of kubelet and workload metrics with a 15 GiB retention cap. This setup disables Alertmanager and skips control-plane scrape jobs commonly unavailable on managed clusters.
helm upgrade --install monitoring prometheus-community/kube-prometheus-stack \
--version 93.4.1 --namespace monitoring --create-namespace \
--wait --timeout 10m --values - <<'YAML'
alertmanager:
enabled: false
kubeScheduler:
enabled: false
kubeControllerManager:
enabled: false
kubeProxy:
enabled: false
kubeEtcd:
enabled: false
grafana:
enabled: true
service:
type: ClusterIP
resources:
requests: {cpu: 100m, memory: 192Mi}
sidecar:
resources:
requests: {cpu: 25m, memory: 64Mi}
prometheusOperator:
resources:
requests: {cpu: 100m, memory: 128Mi}
prometheusConfigReloader:
resources:
requests: {cpu: 10m, memory: 32Mi}
kube-state-metrics:
resources:
requests: {cpu: 50m, memory: 64Mi}
prometheus-node-exporter:
hostNetwork: false
resources:
requests: {cpu: 50m, memory: 32Mi}
prometheus:
service:
type: ClusterIP
prometheusSpec:
replicas: 1
enableOTLPReceiver: true
retention: 7d
retentionSize: 15GiB
resources:
requests: {cpu: 250m, memory: 1Gi}
storageSpec:
volumeClaimTemplate:
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 20Gi
YAML
prom_sts=prometheus-monitoring-kube-prometheus-prometheus kubectl -n monitoring wait --for=create "statefulset/$prom_sts" --timeout=10m kubectl -n monitoring rollout status "statefulset/$prom_sts" --timeout=10m kubectl -n monitoring get pvc
Expect one ready Prometheus pod and a Bound 20 GiB claim. For a Pending claim, check provisioning events with kubectl -n monitoring describe pvc.
Install the OpenTelemetry Collector
Prometheus scrapes pod metrics immediately. Once you enable autoscaling, 3B also sends scaling gauges to http://otel-collector.monitoring.svc:4318/v1/metrics for the Collector to forward to Prometheus.
helm upgrade --install otel-collector open-telemetry/opentelemetry-collector \
--version 0.180.0 --namespace monitoring \
--wait --timeout 10m --values - <<'YAML'
fullnameOverride: otel-collector
mode: deployment
replicaCount: 1
image:
repository: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s
command:
name: otelcol-k8s
service:
type: ClusterIP
ports:
otlp-http: {enabled: true}
otlp: {enabled: false}
jaeger-compact: {enabled: false}
jaeger-thrift: {enabled: false}
jaeger-grpc: {enabled: false}
zipkin: {enabled: false}
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {memory: 256Mi}
config:
receivers:
otlp:
protocols:
http:
endpoint: ${env:MY_POD_IP}:4318
grpc: null
jaeger: null
prometheus: null
zipkin: null
exporters:
debug: null
otlp_http:
endpoint: http://monitoring-kube-prometheus-prometheus.monitoring.svc:9090/api/v1/otlp
service:
pipelines:
logs: null
traces: null
metrics:
receivers: [otlp]
exporters: [otlp_http]
YAML
How do I check the install and open Grafana?
kubectl -n monitoring get deployment/otel-collector
kubectl -n monitoring get service monitoring-grafana monitoring-kube-prometheus-prometheus otel-collector
kubectl -n monitoring exec "statefulset/$prom_sts" -c prometheus -- \
/bin/promtool query instant http://localhost:9090 'count(kube_pod_info{namespace="3b"})'
Expect 1/1 ready and ClusterIP Services for Grafana (80), Prometheus (9090), and the Collector (4318). After the first kube-state-metrics scrape, expect a positive pod count. Replace 3b if your namespace differs.
For the private Grafana URL and credentials, see Monitor a 3B Helm release. To scale workers and Edge pods, see Autoscale a 3B Helm release.
