feat(metrics): add Prometheus metrics support with endpoint and configuration

.env.example

@@ -89,6 +89,14 @@ AI_TIMEOUT=300s
 # set it to override for proxies/self-hosted gateways, e.g. AI_PROVIDER=ollama
 # AI_PROVIDER=
 
+# OTLP endpoint traces and metrics are exported to.
+#OTEL_EXPORTER_ENDPOINT=http://localhost:4317
+# Prometheus scrape endpoint: the same metrics in Prometheus text format at GET /metrics on the
+# main HTTP port (on by default, no collector needed). Unauthenticated — it reveals model names,
+# token counts, and spend, so restrict it at the proxy/firewall or disable it when the bot is
+# internet-facing.
+#PROMETHEUS_METRICS_ENABLED=false
+
 # Per-model AI settings (thrillhousebot.ai.models.<model>.*, keyed by the AI_MODEL name like the
 # pricing map). Keep entries for several models and switch AI_MODEL freely — only the active
 # model's entry is read. max-input-tokens is the model's input hard cap (context window): the

CHANGELOG.md

@@ -4,6 +4,10 @@ All notable changes to ThrillhouseBot.
 
 ## [Unreleased]
 
+### Added
+
+- **Prometheus metrics endpoint**: the metrics exported over OTLP — token usage, latency, AI cost, and the new `thrillhouse.reviews.total` finished-reviews-by-outcome counter — are also served in Prometheus text format at `GET /metrics` on the main HTTP port, so self-hosters can scrape directly without an OTLP collector. On by default; the endpoint is unauthenticated, so shield it at the proxy/firewall or disable it with `PROMETHEUS_METRICS_ENABLED=false` (scrapes then return `404`; OTLP export is unaffected)
+
 ## [0.5.0] — 2026-07-26
 
 Review precision: confidence now decides where a finding lands, newly-added parsers and regexes are stress-tested for their own failure modes, and several classes of false positive are guarded at both the generator and the verifier. Operators gain configurable CI-gating and blocking strictness, structured skip reasons, and per-model generation parameters.

README.md

@@ -49,7 +49,7 @@ guide, configuration reference, architecture, comparison, and the hosted
 - A summary comment on the first run, with a risk breakdown and a changed-files walkthrough
 - Operable from the PR with comment commands — `/help`, `/review`, `/summary`, `/describe`, `/changelog`, `/add-docs`, `/resolve`, `/pause`, `/resume`
 - Live dashboard (Next.js) with a WebSocket activity feed, cost charts, and token tracking
-- OpenTelemetry traces, token histograms, cost counters, and latency metrics
+- OpenTelemetry traces, token histograms, cost counters, and latency and review-outcome metrics — exported via OTLP and scrapeable in Prometheus format at `/metrics`
 - Optional reasoning-effort dial and per-model generation/budget caps for OpenAI-compatible endpoints
 - Reads per-repo instructions from `.github/thrillhousebot.md`, falling back to Copilot/Claude/Agents files
 - Compiles ahead-of-time with GraalVM/Mandrel, so it starts fast and stays small
@@ -268,6 +268,8 @@ will change per provider:
 | `HTTP_CONNECT_TIMEOUT` | Outbound HTTP connect timeout (GitHub API, OAuth) | `10s` |
 | `HTTP_REQUEST_TIMEOUT` | Outbound HTTP request timeout (GitHub API, OAuth) | `10s` |
 | `WEBSOCKET_KEEPALIVE_MS` | Dashboard WebSocket keepalive interval in ms; `0` or negative disables it (and stale replay-buffer eviction) | `25000` |
+| `OTEL_EXPORTER_ENDPOINT` | OTLP endpoint traces and metrics are exported to | `http://localhost:4317` |
+| `PROMETHEUS_METRICS_ENABLED` | Serve the same metrics in Prometheus text format at `GET /metrics` on the main HTTP port; the endpoint is unauthenticated, so disable or shield it on internet-facing deployments | `true` |
 
 ### AI call budget
 
@@ -444,7 +446,8 @@ Labelling is best-effort — a failure here never blocks or fails the review.
 
 ## Observability
 
-All telemetry is exported via OTLP:
+All telemetry is exported via OTLP (`OTEL_EXPORTER_ENDPOINT`, default
+`http://localhost:4317`):
 
 | Signal | Metric |
 |---|---|
@@ -453,12 +456,34 @@ All telemetry is exported via OTLP:
 | `gen_ai.client.operation.duration` | Histogram: latency in seconds |
 | `thrillhouse.ai.cost.total` | Counter: USD cost by model |
 | `thrillhouse.review.skips` | Counter: automatic reviews skipped, tagged with `reason` and `repository` |
+| `thrillhouse.reviews.total` | Counter: finished reviews by `outcome` (`completed`/`failed`) |
 
 Spans and metrics are tagged with `gen_ai.provider.name`, derived from `AI_BASE_URL`
 (e.g. `deepseek`, `openai`, `groq`, `openrouter`). Loopback and unrecognized endpoints
 report `unknown`; set `AI_PROVIDER` to label them (e.g. a local `ollama` or `vllm` server,
 a proxy, or a self-hosted gateway).
 
+### Prometheus
+
+The same metrics are also served in Prometheus text format at `GET /metrics` on
+the main HTTP port — no OTLP collector needed:
+
+```yaml
+scrape_configs:
+  - job_name: thrillhousebot
+    static_configs:
+      - targets: ["your-bot-host:8080"]
+```
+
+Metric names follow the Prometheus conversion of the OTel names above (e.g.
+`thrillhouse_ai_cost_total`, `gen_ai_client_token_usage`,
+`thrillhouse_reviews_total`).
+
+The endpoint is unauthenticated and reveals model names, token counts, and
+spend. On an internet-facing deployment, restrict `/metrics` at your reverse
+proxy or firewall, or disable it with `PROMETHEUS_METRICS_ENABLED=false`
+(scrapes then return `404`; OTLP export is unaffected).
+
 ## Troubleshooting
 
 ### PR opened but no review posted

pom.xml

@@ -22,6 +22,8 @@
         <quarkus-langchain4j.version>1.12.0</quarkus-langchain4j.version>
         <opentelemetry.version>1.62.0</opentelemetry.version>
         <opentelemetry.alpha.version>1.62.0-alpha</opentelemetry.alpha.version>
+        <!-- Keep in lockstep with the prometheus-metrics version opentelemetry-exporter-prometheus ships -->
+        <prometheus-metrics.version>1.8.0</prometheus-metrics.version>
         <jackson-bom.version>2.22.1</jackson-bom.version>
         <skipITs>true</skipITs>
         <!-- JUnit tag filters for surefire; the eval profile flips them to run PromptEvalTest -->
@@ -141,6 +143,24 @@
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-opentelemetry</artifactId>
         </dependency>
+        <!-- Prometheus scrape endpoint (/metrics) backed by the same OTel metrics as OTLP.
+             The exporter declares its prometheus-metrics deps runtime-scoped, so the two
+             artifacts our code compiles against are pulled in explicitly at the exporter's
+             own version (see its pom when bumping). -->
+        <dependency>
+            <groupId>io.opentelemetry</groupId>
+            <artifactId>opentelemetry-exporter-prometheus</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>io.prometheus</groupId>
+            <artifactId>prometheus-metrics-model</artifactId>
+            <version>${prometheus-metrics.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>io.prometheus</groupId>
+            <artifactId>prometheus-metrics-exposition-textformats</artifactId>
+            <version>${prometheus-metrics.version}</version>
+        </dependency>
 
         <!-- WebSockets -->
         <dependency>

src/main/java/dev/thiagogonzaga/thrillhousebot/config/ThrillhouseConfig.java

@@ -53,6 +53,19 @@ public interface ThrillhouseConfig {
 
   AiPricingConfig ai();
 
+  MetricsConfig metrics();
+
+  interface MetricsConfig {
+    /**
+     * Serves the OpenTelemetry metrics in Prometheus text format at {@code GET /metrics} on the
+     * main HTTP port, alongside the OTLP export. The endpoint is unauthenticated; disable it (or
+     * shield it at the proxy) on internet-facing deployments.
+     */
+    @WithDefault("true")
+    @WithName("prometheus-enabled")
+    boolean prometheusEnabled();
+  }
+
   interface GitHubConfig {
     @WithName("app-id")
     String appId();

src/main/java/dev/thiagogonzaga/thrillhousebot/observability/PrometheusMetricsRegistrar.java

@@ -0,0 +1,63 @@
+/*
+ * Copyright 2026 Thiago Gonzaga
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package dev.thiagogonzaga.thrillhousebot.observability;
+
+import dev.thiagogonzaga.thrillhousebot.config.ThrillhouseConfig;
+import io.opentelemetry.exporter.prometheus.PrometheusMetricReader;
+import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder;
+import io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.inject.Inject;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * Registers a {@link PrometheusMetricReader} on the OpenTelemetry SDK so the metrics exported over
+ * OTLP can also be scraped in Prometheus text format from {@code GET /metrics} (served by {@link
+ * PrometheusMetricsResource}). Quarkus invokes every CDI bean implementing {@link
+ * AutoConfiguredOpenTelemetrySdkBuilderCustomizer} while building the SDK; the reader stays unset —
+ * and the endpoint answers 404 — when {@code thrillhousebot.metrics.prometheus-enabled=false} or
+ * the OTel SDK itself is disabled.
+ */
+@ApplicationScoped
+public class PrometheusMetricsRegistrar implements AutoConfiguredOpenTelemetrySdkBuilderCustomizer {
+
+  private final boolean enabled;
+  private final AtomicReference<PrometheusMetricReader> reader = new AtomicReference<>();
+
+  @Inject
+  public PrometheusMetricsRegistrar(ThrillhouseConfig config) {
+    this.enabled = config.metrics().prometheusEnabled();
+  }
+
+  @Override
+  public void customize(AutoConfiguredOpenTelemetrySdkBuilder builder) {
+    if (!enabled) {
+      return;
+    }
+    builder.addMeterProviderCustomizer(
+        (meterProviderBuilder, configProperties) -> {
+          var prometheusReader = PrometheusMetricReader.create();
+          reader.set(prometheusReader);
+          return meterProviderBuilder.registerMetricReader(prometheusReader);
+        });
+  }
+
+  /** The registered reader, or empty when the endpoint is disabled or the SDK never started. */
+  Optional<PrometheusMetricReader> reader() {
+    return Optional.ofNullable(reader.get());
+  }
+}

src/main/java/dev/thiagogonzaga/thrillhousebot/observability/PrometheusMetricsResource.java

@@ -0,0 +1,58 @@
+/*
+ * Copyright 2026 Thiago Gonzaga
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package dev.thiagogonzaga.thrillhousebot.observability;
+
+import io.prometheus.metrics.expositionformats.PrometheusTextFormatWriter;
+import jakarta.inject.Inject;
+import jakarta.ws.rs.GET;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.core.MediaType;
+import jakarta.ws.rs.core.Response;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+
+/**
+ * Prometheus scrape endpoint on the main HTTP port. Serves whatever the {@link
+ * PrometheusMetricsRegistrar} collected — token usage, latency, cost, and review-outcome metrics —
+ * in Prometheus text format; answers 404 while no reader is registered (endpoint disabled via
+ * {@code PROMETHEUS_METRICS_ENABLED=false} or OTel SDK disabled).
+ */
+@Path("/metrics")
+public class PrometheusMetricsResource {
+
+  private static final PrometheusTextFormatWriter WRITER = PrometheusTextFormatWriter.create();
+
+  private final PrometheusMetricsRegistrar registrar;
+
+  @Inject
+  public PrometheusMetricsResource(PrometheusMetricsRegistrar registrar) {
+    this.registrar = registrar;
+  }
+
+  @GET
+  public Response scrape() throws IOException {
+    var reader = registrar.reader();
+    if (reader.isEmpty()) {
+      return Response.status(Response.Status.NOT_FOUND)
+          .type(MediaType.TEXT_PLAIN)
+          .entity("Prometheus metrics are disabled (PROMETHEUS_METRICS_ENABLED=false).")
+          .build();
+    }
+    var buffer = new ByteArrayOutputStream();
+    WRITER.write(buffer, reader.get().collect());
+    return Response.ok(buffer.toByteArray(), WRITER.getContentType()).build();
+  }
+}

src/main/java/dev/thiagogonzaga/thrillhousebot/observability/ReviewOutcomeMetrics.java

@@ -0,0 +1,53 @@
+/*
+ * Copyright 2026 Thiago Gonzaga
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package dev.thiagogonzaga.thrillhousebot.observability;
+
+import io.opentelemetry.api.OpenTelemetry;
+import io.opentelemetry.api.common.AttributeKey;
+import io.opentelemetry.api.common.Attributes;
+import io.opentelemetry.api.metrics.LongCounter;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.inject.Inject;
+
+/**
+ * Counts finished PR reviews by outcome. Recorded once per review at its terminal transition (the
+ * orchestrator's completed/failed paths), never per AI call — retried calls inside one review must
+ * not inflate the count.
+ */
+@ApplicationScoped
+public class ReviewOutcomeMetrics {
+
+  private static final AttributeKey<String> OUTCOME = AttributeKey.stringKey("outcome");
+
+  private final LongCounter reviews;
+
+  @Inject
+  public ReviewOutcomeMetrics(OpenTelemetry otel) {
+    this.reviews =
+        otel.getMeter("thrillhousebot")
+            .counterBuilder("thrillhouse.reviews.total")
+            .setDescription("Finished PR reviews by outcome")
+            .build();
+  }
+
+  public void recordCompleted() {
+    reviews.add(1, Attributes.of(OUTCOME, "completed"));
+  }
+
+  public void recordFailed() {
+    reviews.add(1, Attributes.of(OUTCOME, "failed"));
+  }
+}

src/main/java/dev/thiagogonzaga/thrillhousebot/review/ReviewOrchestrator.java

@@ -21,6 +21,7 @@
 import dev.thiagogonzaga.thrillhousebot.dashboard.ReviewSessionPersistence;
 import dev.thiagogonzaga.thrillhousebot.dashboard.SessionEventBroadcaster;
 import dev.thiagogonzaga.thrillhousebot.github.*;
+import dev.thiagogonzaga.thrillhousebot.observability.ReviewOutcomeMetrics;
 import io.quarkus.logging.Log;
 import jakarta.enterprise.context.ApplicationScoped;
 import jakarta.enterprise.context.control.ActivateRequestContext;
@@ -60,6 +61,8 @@ public class ReviewOrchestrator {
 
   private final FindingPipeline findingPipeline;
 
+  private final ReviewOutcomeMetrics outcomeMetrics;
+
   private final FindingFeedbackCaptureService findingFeedbackCapture;
 
   private final ExecutorService reviewExecutor;
@@ -176,6 +179,7 @@ public ReviewOrchestrator(
       ReviewPublisher reviewPublisher,
       VerdictBuilder verdictBuilder,
       FindingPipeline findingPipeline,
+      ReviewOutcomeMetrics outcomeMetrics,
       FindingFeedbackCaptureService findingFeedbackCapture,
       @ReviewExecutor ExecutorService reviewExecutor) {
     this.config = config;
@@ -190,6 +194,7 @@ public ReviewOrchestrator(
     this.reviewPublisher = reviewPublisher;
     this.verdictBuilder = verdictBuilder;
     this.findingPipeline = findingPipeline;
+    this.outcomeMetrics = outcomeMetrics;
     this.findingFeedbackCapture = findingFeedbackCapture;
     this.reviewExecutor = reviewExecutor;
   }
@@ -469,6 +474,7 @@ void applyReviewResult(ReviewSession session, ReviewResult result) {
             s.setAiResponseJson(session.getAiResponseJson());
           }
         });
+    outcomeMetrics.recordCompleted();
   }
 
   /** Applies failure fields to the in-memory session and persisted entity together. */
@@ -479,6 +485,7 @@ void applyReviewFailure(ReviewSession session, String errorMessage) {
           s.setStatus(ReviewSession.STATUS_FAILED);
           s.setErrorMessage(errorMessage);
         });
+    outcomeMetrics.recordFailed();
   }
 
   private void applySessionState(ReviewSession session, Consumer<ReviewSession> mutator) {

src/main/resources/application.properties

@@ -111,6 +111,12 @@ quarkus.langchain4j.openai.chat-model.log-responses=false
 # OpenTelemetry
 quarkus.otel.exporter.otlp.traces.endpoint=${OTEL_EXPORTER_ENDPOINT:http://localhost:4317}
 quarkus.otel.service.name=thrillhousebot
+# Metrics are recorded via the OpenTelemetry API; enabling them exports over OTLP alongside traces
+# (build-time flag, baked into the artifact).
+quarkus.otel.metrics.enabled=true
+# Prometheus scrape endpoint: the same metrics, served in Prometheus text format at GET /metrics
+# on the main HTTP port. Unauthenticated — restrict or disable it on internet-facing deployments.
+thrillhousebot.metrics.prometheus-enabled=${PROMETHEUS_METRICS_ENABLED:true}
 
 # Review settings
 thrillhousebot.review.max-review-comments=50