Skip to main content

Prometheus Service Installation Guide

This is an installation guide for On-Premise environments that want to use the system monitoring feature.

Overview​

This document provides a guide for installing Prometheus using the kube-prometheus-stack Helm Chart.

Starting from the OVA Virtual Appliance Baseline version 2025.09, Prometheus is installed as a standard feature.

If Prometheus is installed, you can enable system monitoring features with the following configmap settings. (The installation method differs from the namespace.)

### portal-web-config
VUE_APP_USE_SYSTEM_MONITORING: true
### cloud-spsvr-config
CUSTOM_PROMETHEUS_URL: http://rancher-monitoring-prometheus.monitoring.svc.cluster.local:9090
  • List of services included in kube-prometheus-stack
ServiceFunctionRequired status
prometheusCluster metric data collection and storage⭕️
prometheus-config-reloaderAutomatic detection and application of Prometheus settings⭕️
prometheus-operatorResource management and automation related to Prometheus⭕️
prometheus-node-exporterNode system resource data provision (CPU, memory, disk, network)⭕️
kube-state-metricsProvides resource status information operating in a Kubernetes cluster.⭕️
alertmanagerSend alert emails and webhooks generated by Prometheus.❌
grafanaVisualize the data collected from Prometheus and provide it in the form of a dashboard.❌

Installation Method​

  1. Prepare the necessary Docker Image files.

    • Docker Image Pull

      # prometheus
      docker pull quay.io/prometheus/prometheus:v3.2.1

      # prometheus-operator
      docker pull quay.io/prometheus-operator/prometheus-operator:v0.80.1

      # prometheus-node-exporter
      docker pull quay.io/prometheus/node-exporter:v1.9.0

      # prometheus-config-reloader
      docker pull rancher/mirrored-prometheus-operator-prometheus-config-reloader:v0.59.1

      # kube-state-metrics
      docker pull rancher/mirrored-kube-state-metrics-kube-state-metrics:v2.6.0

    • Change Docker Image Tag via Harbor Path in a Closed Network Environment

      # prometheus
      docker tag quay.io/prometheus/prometheus:v3.2.1 harbor.local:5050/prometheus/prometheus:v3.2.1

      # prometheus-operator
      docker tag quay.io/prometheus-operator/prometheus-operator:v0.80.1 harbor.local:5050/prometheus/prometheus-operator:v0.80.1

      # prometheus-node-exporter
      docker tag quay.io/prometheus/node-exporter:v1.9.0 harbor.local:5050/prometheus/prometheus-node-exporter:v1.9.0

      # prometheus-config-reloader
      docker tag rancher/mirrored-prometheus-operator-prometheus-config-reloader:v0.59.1 harbor.local:5050/prometheus/prometheus-config-reloader:v0.59.1

      # kube-state-metrics
      docker tag rancher/mirrored-kube-state-metrics-kube-state-metrics:v2.6.0 harbor.local:5050/prometheus/kube-state-metrics:v2.6.0

    • Save Docker Image as .tar file

      # prometheus
      docker save -o prometheus.tar harbor.local:5050/prometheus/prometheus:v3.2.1

      # prometheus-operator
      docker save -o prometheus-operator.tar harbor.local:5050/prometheus/prometheus-operator:v0.80.1

      # prometheus-node-exporter
      docker save -o node-exporter.tar harbor.local:5050/prometheus/prometheus-node-exporter:v1.9.0

      # prometheus-config-reloader
      docker save -o prometheus-config-reloader.tar harbor.local:5050/prometheus/prometheus-config-reloader:v0.59.1

      # kube-state-metrics
      docker save -o kube-state-metrics.tar harbor.local:5050/prometheus/kube-state-metrics:v2.6.0

  2. Preparing the kube-prometheus-stack Helm Chart

    # helm 저장소 추가
    helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
    helm repo update

    # prometheus-operator-crds 다운로드
    helm pull prometheus-community/prometheus-operator-crds

    # kube-prometheus-stack chart 다운로드
    helm pull prometheus-community/kube-prometheus-stack

  3. Modify the configuration values in the kube-prometheus-stack chart values.yaml file.

    • kube-prometheus-stack chart values.yaml

      # values.yaml
      fullnameOverride: "rancher-monitoring"
      alertmanager:
        enabled: false #사용 안함
      grafana:
        enabled: false #사용 안함
      kubeApiServer:
        enabled: false #사용 안함
      kubeControllerManager:
        enabled: false #사용 안함
      coreDns:
        enabled: false #사용 안함
      kubeEtcd:
        enabled: false #사용 안함
      kubeScheduler:
        enabled: false #사용 안함
      kubeProxy:
        enabled: false #사용 안함
      prometheusOperator:
        image:
          registry: harbor.local:5050 #로컬 Harbor 주소로 변경 필요
          repository: prometheus/prometheus-operator
          tag: "v0.80.1"
        prometheusConfigReloader:
          image:
            registry: harbor.local:5050 #로컬 Harbor 주소로 변경 필요
            repository: prometheus/prometheus-config-reloader
            tag: "v0.59.1"
      prometheus:
        prometheusSpec:
          scrapeInterval: "60s"
          evaluationInterval: "60s"
          retention: 10d #데이터 보관 주기기
          image:
            registry: harbor.local:5050 #로컬 Harbor 주소로 변경 필요
            repository: prometheus/prometheus
            tag: v3.2.1
          volumeClaimTemplate: #PVC 설정
            spec:
              storageClassName: longhorn
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi

    • prometheus-node-exporter subchart values.yaml

      # charts\prometheus-node-exporter\values.yaml
      image:
        registry: harbor.local:5050 #로컬 Harbor 주소로 변경 필요
        repository: prometheus/prometheus-node-exporter
        tag: "v1.9.0"

    • kube-state-metrics subchart values.yaml

      # charts\kube-state-metrics\values.yaml
      image:
        registry: harbor.local:5050 #로컬 Harbor 주소로 변경 필요
        repository: prometheus/kube-state-metrics
        tag: "v2.6.0"

    ※ If you need to limit the collection metrics​

    If you need to limit the collected metrics due to PVC capacity restrictions, you must additionally change the values below in values.yaml.

    kubelet:
      serviceMonitor:
        enabled: false #kubelet default job 제외 처리
    prometheusOperator:
      serviceMonitor:
        selfMonitor: false #prometheus-operator default job 제외 처리
    prometheus:
      serviceMonitor:
        selfMonitor: false #prometheus default job 제외 처리
      prometheusSpec:
        additionalScrapeConfigs:
          - job_name: "node-exporter-custom"
            kubernetes_sd_configs:
              - role: endpoints
                namespaces:
                  names: ["cattle-monitoring-system"]
            relabel_configs:
              - source_labels: [__meta_kubernetes_namespace]
                regex: cattle-monitoring-system
                action: keep
              - source_labels:
                  [__meta_kubernetes_service_label_app_kubernetes_io_name]
                regex: prometheus-node-exporter
                action: keep
              - source_labels: [__meta_kubernetes_endpoint_port_name]
                regex: http-metrics
                action: keep
              - source_labels: [__meta_kubernetes_endpoint_address_target_kind]
                regex: Pod
                action: keep
              - source_labels: [__meta_kubernetes_pod_node_name]
                target_label: instance
                action: replace
            metric_relabel_configs:
              - source_labels: [__name__]
                regex: node_uname_info|node_cpu_seconds_total|node_load1|node_load5|node_load15|node_memory_MemTotal_bytes|node_memory_MemAvailable_bytes|node_memory_Active_bytes|node_memory_.*|node_filesystem_(size|avail|free|used)_bytes|node_network_(receive|transmit)_bytes_total
                action: keep

  4. Packaging the modified kube-prometheus-stack Helm Chart

    helm package kube-prometheus-stack # kube-prometheus-stack-69.8.2.tgz 형식의 패키지 파일 생성

  5. Generated Docker Image file and __PH_0__prometheus-operator-crds, kube-prometheus-stack Helm ChartCopy to the build server

  6. Create a project in local Harbor using the Harbor web UI.

    • Project Name : prometheus
    • Access Level : Public

  7. Load & push Docker Image to local Harbor on the deployment server.

    # prometheus
    docker load -i prometheus.tar
    docker push harbor.local:5050/prometheus/prometheus:v3.2.1

    # prometheus-operator
    docker load -i prometheus-operator.tar
    docker push harbor.local:5050/prometheus/prometheus-operator:v0.80.1

    # prometheus-node-exporter
    docker load -i node-exporter.tar
    docker push harbor.local:5050/prometheus/prometheus-node-exporter:v1.9.0

    # prometheus-config-reloader
    docker load -i prometheus-config-reloader.tar
    docker push harbor.local:5050/prometheus/prometheus-config-reloader:v0.59.1

    # kube-state-metrics
    docker load -i kube-state-metrics.tar
    docker push harbor.local:5050/prometheus/kube-state-metrics:v2.6.0

  8. Prometheus CRDs configuration

    Remove the existing Prometheus CRD that is already configured.

    # CRD 조회
    kubectl get crd | grep monitoring.coreos.com

    # 위 명령어 실행 결과로 나온 CRD 삭제
    kubectl delete crd <CRD 명>

    After removing the CRD, install the CRD using the copied CRDs Helm chart.

    # Namespace : cattle-monitoring-system
    # ReleaseName : prometheus

    helm install prometheus-operator-crds prometheus-operator-crds-20.0.0.tgz -n cattle-monitoring-system --create-namespace

  9. Installing Prometheus

    # Namespace : cattle-monitoring-system
    # ReleaseName : prometheus

    helm install prometheus kube-prometheus-stack-69.8.2.tgz -n cattle-monitoring-system --create-namespace

    If you need to limit the collection metrics, you can run the following command to prevent basic metric collection.

    kubectl delete servicemonitor prometheus-prometheus-node-exporter -n cattle-monitoring-system
    kubectl delete servicemonitor prometheus-kube-state-metrics -n cattle-monitoring-system

Installation Uninstallation​

If you need to uninstall Prometheus, you can do so using the following command.

helm uninstall prometheus -n cattle-monitoring-system

Additional Settings​

Management Center configMap settings​

To use the system monitoring feature of the management center, the configMap values of the management center frontend/backend must be set correctly.

  • Security365 Management Center Frontend configMap

    VUE_APP_USE_SYSTEM_MONITORING: Whether to use system monitoring features

    ### portal-web-config
    VUE_APP_USE_SYSTEM_MONITORING: true

  • Security365 Management Center Backend configMap

    CUSTOM_PROMETHEUS_URL: Internal address of the Prometheus service in the cluster

    ### cloud-spsvr-config
    CUSTOM_PROMETHEUS_URL: http://rancher-monitoring-prometheus.cattle-monitoring-system.svc.cluster.local:9090