Skip to content

pgo monitoring enabled but the monitoring user not created inside postgres and the metrics fail to generate #4501

@sinawic

Description

@sinawic

Overview

Environment

  • Platform: (Kubernetes)
  • Platform Version: (v1.31.1)
  • PGO Image Tag: (ubi9-6.0.1-0)
  • Postgres Version (18)
  • Storage: (local-path)

Steps to Reproduce

git clone https://github.com/CrunchyData/postgres-operator.git
cd postgres-operator
git checkout v6.0.1
kubectl apply -k config/namespace
kubectl apply --server-side -k config/default

the config i used to create my cluster using kustomize

apiVersion: postgres-operator.crunchydata.com/v1
kind: PostgresCluster
metadata:
  name: sinawic
spec:
  service:
    type: NodePort
    nodePort: 30432
  postgresVersion: 18
  users:
    - name: postgres
      databases:
        - postgres
  instances:
    - name: instance1
      replicas: 2
      dataVolumeClaimSpec:
        accessModes:
          - "ReadWriteOnce"
        storageClassName: local-path
        resources:
          requests:
            storage: 8Gi
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/hostname
                    operator: In
                    values:
                      - node2
                      - node3

  patroni:
    dynamicConfiguration:
      postgresql:
        parameters:
          max_connections: 100
          idle_in_transaction_session_timeout: 60000
          idle_session_timeout: 60000

  monitoring:
    pgmonitor:
      exporter: {}

EXPECTED

the exporter container cant seem to be able to connect to the postgres instance, and as far as i went through, the user ccp_monitoring doesnt get created and it seems its exact user the exporter is trying to connect to postgres with.
so i inspected the exporter metrics. it responds but the metrics are just general things like related to go and stuff like that.
the password secret for the ccp_monitoring user is created and injected int the exporter container but not the user itself.

after i created and granted the required user:

CREATE USER ccp_monitoring WITH PASSWORD 'password_created_for_monitoring_user_in_the_monitoring_secret';
GRANT pg_monitor TO ccp_monitoring;

metrics got generated and apparently things are back to normal.
but i expected by doing:

  monitoring:
    pgmonitor:
      exporter: {}

it should have been automatically created, like in the previous version <16>.
i followed the official docs in https://access.crunchydata.com/documentation/postgres-operator/latest/architecture/monitoring
and it states that we should either enable the metrics or the otel.

Logs

level=INFO source=server.go:73 msg="Established new database connection" fingerprint=localhost:5432
level=ERROR source=collector.go:178 msg="Error opening connection to database" err="error querying postgresql version: pq: password authentication failed for user \"ccp_monitoring\""
level=ERROR source=postgres_exporter.go:684 msg="error scraping dsn" err="Error opening connection to database (postgresql://ccp_monitoring:PASSWORD_REMOVED@localhost:5432/postgres): pq: password authentication failed for user \"ccp_monitoring\"" dsn=postgresql://ccp_monitoring:PASSWORD_REMOVED@localhost:5432/postgres

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions