Dynamic path templating for webhook destinations

[ambroziepaval]

Problem / use case.

When a tenant initiates an operation, say a action.initiated event, tey often want the resulting webhook delivered to a per-operation callback path chosen at initiation time, e.g. https://app.example.com/actions/<uuid>/callback

From a developer-experience standpoint this is a very natural thing to reach for: "I just want Outpost to handle the retries, queuing and delivery of this one HTTP request, so why can't I just hand it the destination URL on the fly?" It's an intuitive ask, which is part of why it keeps coming up.

Today a webhook destination's url is fully static per destination. The only way to achieve a per-operation path is to create a throwaway destination for each operation and publish with destination_id. That means unbounded destination churn plus lifecycle and cleanup burden, which does not fit a high-volume "one callback per action" pattern.
It also fragments the portal/UI experience: instead of thousands of single-use destinations, each with its own isolated logs, delivery attempts, and metrics, templating keeps everything under a single stable destination. All the usual actions (viewing delivery history, retrying, inspecting attempts, editing config) then operate on one coherent destination rather than being scattered across per-operation throwaways.

Proposal

Allow the path portion of a webhook URL to be templated from event fields, while the scheme + host remain fixed static config. This reuses the Go text/template mechanism already used for signing secrets and signature content in the webhook provider.

Example destination confog:

config.url               = "https://app.example.com"
config.url_path_template = "/actions/{{.metadata.callback_id}}/callback"

Publishing an event with metadata.callback_id = "abc-123" delivers to:

https://app.example.com/actions/abc-123/callback

Retries, queuing and signing all continue to work exactly as they do today.

Why templated path, not arbitrary URL

A fully event-controlled URL, while much easier to integrate, is deliberately not proposed, because it introduces three problems:

1. SSRF. Event data an attacker can influence could redirect delivery to internal or cloud-metadata endpoints (169.254.169.254, localhost, private ranges).
2. Rate-limiting / concurrency fragmentation. Outpost caches publishers and reasons about destinations by a stable config identity. A URL that changes every event fragments that model.
3. Fragmented telemetry. Per-destination metrics and logs assume a stable target host; a moving host makes per-destination dashboards meaningless.

Pinning scheme + host to operator-controlled config while templating only the path neutralizes all three: the host (the SSRF-relevant part) is never event-controlled, the destination identity stays stable, and telemetry stays attributed per destination.

Design choices (proposed, open to input)

1. Separate field url_path_template rather than template markers inside url, so the base URL stays a plain, validatable URL and scheme + host can never be templated.
2. Template inputs: metadata (and possibly data). The event's metadata string map is a small, safe surface; parsed data could also be exposed if there's appetite. One open point: metadata keys are already emitted as X-Outpost-* headers, so a value used to shape the path would also go out as a header; if that's undesirable, a reserved metadata namespace (e.g. keys prefixed _path.) could be usable in the template but filtered out of the headers.
3. Guardrails: percent-encode resolved path segments; reject templates/results containing ://, .., @, or a leading //; enforce the rendered result is still same-origin as the configured base URL.

Open questions

1. Template-resolution failure at delivery time — hard-fail the attempt (visible in the attempt log) vs. fall back to the base URL? Proposal: hard-fail.
2. Where template inputs should come from — metadata only, also parsed data, or is there a different source altogether that would be a better fit than either metadata or data? Open to suggestions if neither is the right home for a path-shaping value.

[alexluong]

Hi @ambroziepaval, thanks for opening the issue, this is an interesting idea. I've seen similar patterns before and you're right that Outpost doesn't support this pattern yet. That said, I'm not 100% sure if this is something we want to fully support within Outpost since it does complicate the destination schema a bit.

Is this patterns something you're using in your system already? What's the benefit of a dynamic URL instead of a static URL with the data in the payload?

cc @alexbouchardd @leggetter for input!

[alexbouchardd]

It's not something I've seen supported in many (or any) other webhook implementations. The flexibility is nice, but it is unclear if it warrants the additional complexity and edge cases like template resolution failures.

In the case where do implement something like this, I would support templating as part of url and not add a new property that's for sure

[alexluong]

Usually these are more "callback" than "webhook" in my opinion.

Sometimes I also see query params instead of dynamic URL path.

[jcagarcia]

Wanted to add a few use cases that reinforce the need for this kind of per-request URL flexibility, beyond the destination churn problem already described.

• Migrating existing clients to Outpost. We have clients who have never used a pre-configured destination at all — they pass a different callback URL on every resource they create (e.g. notifications_url on each payment). Getting them onto Outpost today means teaching them a new configuration step (set up a destination first) before they can do anything. If URLs can be expressed dynamically at the event level, the migration path is much smoother: they keep their existing pattern and we adopt Outpost incrementally.
• Self-service URL changes without support intervention. When API consumers don't have access to a dashboard to manage destinations, any URL change on their side currently requires them to contact our support team to update the destination. If the URL can come from the event payload, they can start sending a new URL immediately without any coordination — they're fully self-sufficient.
• Testing / QA scenarios. When writing integration tests or running staged smoke tests, it's very common to want to temporarily redirect a single request to a mock or stub URL without reconfiguring the destination globally. Being able to supply the target URL at publish time would make test setups much cleaner.

All three cases share a common root: the consumer knows the target URL at event time, not at destination-configuration time. The path-template approach addresses this well, though I'm curious why allowing a direct URL to be provided per-event (rather than a template) would be a problem — it seems like the simpler solution for most of these cases, with host allowlist validation as the SSRF guard if needed.

[jcagarcia]

Adding also some examples from the industry:

• DLocal allows to provide a notification_url each time a create payment request is performed. https://docs.dlocal.com/reference/create-payment
• Trustly also allows to provide a NotificationURL parameter when creating a deposit. https://docs.trustly.com/api/deposit
• In our case, Flywire also allows to provide a notifications_url each time a charge payment request is performed. https://developers.flywire.com/education/Content/resource_payments.htm#Charging

[alexbouchardd]

Interestingly, the use cases described here all refer to providing a specific URL as part of the payload rather than interpolation. I wonder if the right approach is therefore interpolation or some kind of URL override. The problem I would see is that the URL override would inherently have to apply to all destinations, which conflicts with the fan-out design of Outpost.

Do you have something to propose on that front or you believe interpolate is still the right design approach?

[jcagarcia]

Thanks for the question @alexbouchardd

Yes, I think the right design is a URL override rather than interpolation, and I have a concrete proposal that addresses the fan-out conflict you raised.

The core idea: URL override scoped to the publish call

The override would apply to the specific event being published and would replace the matched destination URL(s) for that delivery. Consider a tenant with two destinations:

• topic=* → https://app.example.com/logs
• topic=payment.* → https://app.example.com/payments

When payment.created is published, Outpost fans out to both. If the publisher provides a URL override with no topic qualifier, both deliveries would be redirected to that URL. That may not always be desirable.

Topic-scoped override

To give publishers finer control, the override could accept an optional topic parameter:

POST /publish
{
  "topic": "payment.created",
  "url": "https://app.example.com/payments/pay_123/callback",
  "url_topic": "payment.*"
}

This would only redirect the delivery matched by payment.* → the /logs destination keeps receiving the event normally. Publishers who want to override all matched destinations simply omit url_topic.

Destination-free tenants

It's also worth considering allowing a tenant to operate with no pre-configured destination at all, relying entirely on dynamic URLs at publish time. This is the migration path we need: existing clients who already pass a notifications_url per resource don't have to learn a new destination setup step to adopt Outpost — they keep their existing pattern and we adopt Outpost incrementally.

On SSRF: a host allowlist at the tenant level (configured once by the operator) would keep this safe — publishers can vary path and query string freely, but the host must be pre-approved.

Does this direction make sense? Happy to go deeper on any of the pieces.

[alexbouchardd]

Thanks for this! I do think there's a good idea there, but to manage expectations, my impression is that this is a fairly rare case and requires some major rework. For instance, right now each event/attempt needs to be associated with a destination, and supporting destination-less tenants is a big endeavour.

I'd love to propose a workaround: each of your tenants could have a "default" destination with a URL that points to a reverse proxy you manage. The event metadata could then include the notification_url and the reverse proxy would read the url from the headers and forward the HTTP request to that URL.

[jcagarcia]

Hey @alexbouchardd , sure, using a reverse proxy managed by us was our workaround intention if this issue was discarded.

However, if I'm not wrong and I correctly understand the flow, when the consumer performs the GET /events, the consumer will see the reverse proxy url and not the final one, right?

[alexbouchardd]

It's the /attempts endpoint, but yes. The destination.config.url will contain the proxy URL but could be rewritten by your API by post-processing the Outpost response. Fully acknowledging that's a workaround.

[ambroziepaval]

While working on the dynamic-URL destination approach suggested above, I hit a routing/idempotency gap that I think is worth solving alongside it.

The gap
/publish accepts either no destination_id (fan out to all matching destinations) or exactly one. There's no way to send a single event to a specific subset. Doing it today means one publish call per destination, and since the event id is the idempotency key, each call mints a distinct event, so one logical event becomes N events across logs and metrics. Reusing the same id to unify them doesn't work: the second call collides on the key and is dropped as a duplicate, so its destination never receives the event.

This bites the dedicated-destination workaround directly: a proxy destination still needs a topic subscription, so untargeted publishes fan out to it too, and the only way to exclude it is client-side enumeration, which reintroduces the "one event becomes many" problem.

Proposal
Add an optional destination_ids (plural) field to /publish so a single event routes to a caller-chosen subset: one event id, one idempotency claim, one fan-out. Suggested semantics:

• Compatibility. Keep destination_id working, add destination_ids alongside it, 400 if both are set. Longer term, treat destination_ids as canonical and deprecate the singular field through a deprecation window.
• Matching still applies per targeted destination (topic + filter), same as destination_id today — naming a destination narrows the candidate set, it doesn't bypass matching.
• Missing/disabled entries are skipped, matching how an unknown single destination_id behaves today.

It fits the existing model: idempotency already keys off event.ID and wraps the whole fan-out, so this isn't a semantic change, just a caller-chosen fan-out.

Known limitation
Because matching still applies, a proxy destination must subscribe to a matching topic and will still be caught by untargeted broadcasts. Fully closing that would need a separate destination-side opt-out (e.g. an explicit_only flag); I've left it out to keep scope tight, but the two compose well.

Happy to open a PR if this direction sounds reasonable.

[alexluong]

Hey @ambroziepaval, I think the design intention is for the publisher to defer Outpost to do the matching.

What's the context where you already know about the exact list of destinations you want to send events to? I assume that means you're relying much on topics?

[ambroziepaval]

Hey @alexluong , I completely agree that the publisher should defer to Outpost for matching. The intent was never that I know the exact list of destinations to sed to, it was the opposite: I know the one destination I don't want to send to (the system-managed "default" destination that points back to our reverse proxy for handling dynamic URLs).

That destination subscribes to topics=* so it always matches, but it should only fire when the publish call actually carries a dynamic URL. Without a way to exclude it, the only option was client-side enumeration of all other destintions, which was exactly the "one event becones N" I described above.

BUT, as I was writing this reply, I realised the solution already exists: the filter field on destinations, with the $exists operator. We configure the dynamic destination with: "filter": { "metadata": { "notification_url": { "$exist": true } } }. Then the published needs to put the dynamic URL into the event metadata.

Outpost's own matching engine handles the exclusion: single publish call, single event ID, normal idempotency, no client-side enumeration.

Thanks for pushing back on framing, it led me to a cleaner solution.

[alexluong]

Ah that's a nice use case of filter, glad to see it's working for you!