Skip to content

[FCP-4223] Use Endpoints API for flink endpoint list/use#3355

Merged
Daniel Ayaz (danielayaz) merged 1 commit into
confluentinc:mainfrom
jsnrth:FCP-4223/flink-endpoint-list-via-endpoints-api
May 21, 2026
Merged

[FCP-4223] Use Endpoints API for flink endpoint list/use#3355
Daniel Ayaz (danielayaz) merged 1 commit into
confluentinc:mainfrom
jsnrth:FCP-4223/flink-endpoint-list-via-endpoints-api

Conversation

@jsnrth

@jsnrth Jason Roth (jsnrth) commented May 14, 2026

Copy link
Copy Markdown
Contributor

Release Notes

New Features

  • confluent flink endpoint list and confluent flink endpoint use now use the Confluent Cloud Endpoints API (endpoint/v1/endpoints, service=FLINK). This aligns the CLI with the API used by Terraform and the Cloud Console, and surfaces endpoint shapes — notably multi-PLATT PrivateLink Gateway access points — that the legacy URL-template aggregation could not represent.

Checklist

  • I have successfully built and used a custom CLI binary, without linter issues from this PR.

  • I have clearly specified in the What section below whether this PR applies to Confluent Cloud, Confluent Platform, or both. (Confluent Cloud only.)

  • I have verified this PR in Confluent Cloud pre-prod or production environment, if applicable.

  • I have verified this PR in Confluent Platform on-premises environment, if applicable. (N/A — Cloud-only command.)

  • I have attached manual CLI verification results or screenshots in the Test & Review section below.

  • I have added appropriate CLI integration or unit tests for any new or updated commands and functionality.

  • I confirm that this PR introduces no breaking changes or backward compatibility issues.

  • I have indicated the potential customer impact if something goes wrong in the Blast Radius section below.

  • I have put checkmarks below confirming that the feature associated with this PR is enabled in:

    • Confluent Cloud prod
    • Confluent Cloud stag
    • Confluent Platform
    • Check this box if the feature is enabled for certain organizations only

    (Endpoints API is GA in Cloud prod + stag; no feature flag.)

What

Replace the three-source CLI-side aggregation (ListFlinkRegions + ListNetworkPrivateLinkAttachments + ListNetworks + URL templating) with a single call to the new Endpoints API:

endpoints, err := c.V2Client.ListEndpoints(envId, cloud, region, "FLINK", nil, "")

Three adjustments preserve the existing output contract while consuming the new API:

  1. Filter to endpoint_type == "REST" — the API also returns LANGUAGE_SERVICE endpoints (flinkpls.*) used by the Cloud Console SQL editor; these are not usable by CLI dataplane commands.
  2. Restore the https:// scheme on display — the API returns bare FQDNs; the legacy command produced full URLs. Scheme preservation if already present keeps the test gateway URLs (http://127.0.0.1:1026) working.
  3. Uppercase the cloud column — the API returns lowercase (aws); the legacy column convention is uppercase (AWS). The cloud / service query params are also normalized to uppercase inside ListEndpoints itself, so callers don't have to remember.

endpointUse validation surfaces real errors (API outage, missing environment) instead of returning false and triggering a misleading "endpoint invalid" message — the validator returns (bool, error) and the caller propagates.

The endpointv1 SDK is wired into the shared ccloudv2.Client as EndpointClient for future reuse.

Blast Radius

Cloud-only. Possible deltas vs. the legacy aggregation:

  • Fewer rows than before if the Endpoints API doesn't model an endpoint type the legacy code did. The API is the source of truth used by Terraform and the Cloud Console today, so any such gap would already be visible there.
  • API outage → endpoint list and endpoint use return an error (with unable to list Flink endpoints context on the list path). Same failure mode the legacy ListFlinkRegions had.
  • Net SDK call count drops (3 → 1 per invocation).

No impact on Confluent Platform. No flag, command, or output schema changes.

References

Test & Review

Local verification: make build, make lint-go, unit tests, and TestFlinkEndpointList / TestFlinkEndpointUse integration suites all clean. The two TestFlinkEndpointList failures observed locally are pre-existing on main (backtick rendering in error suggestions for untouched code paths).

Test additions:

  • New internal/flink/command_endpoint_list_test.go unit-tests flinkEndpointUrl (bare FQDN, GLB access-point FQDN, https-preserved, http-preserved).
  • New test/test-server/endpoint_handlers.go mocks /endpoint/v1/endpoints with the same data shapes the legacy aggregation produced, plus one LANGUAGE_SERVICE row to verify the REST filter, with an explicit GET-only method guard.
  • New azure/italynorth region + list-azure-access-point.golden + use-azure-access-point.golden exercise a multi-PLATT GLB access-point URL (flink-ap4jnpj9.italynorth.azure.accesspoint.glb.confluent.cloud) — a shape the legacy code physically could not produce.

Prod verification: Tested against system-test orgs in prod for AWS eu-central-2 (both public-only and multi-PLATT + CCN) and AZURE italynorth (multi-PLATT GLB access-point). Output matches expectations: REST-only rows, https:// scheme, uppercase cloud, multi-PLATT URLs correctly surfaced.

Copilot AI review requested due to automatic review settings May 14, 2026 21:13
@confluent-cla-assistant

Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Flink endpoint discovery in the CLI to use Confluent Cloud’s Endpoints API (/endpoint/v1/endpoints, service=FLINK) so multi-PLATT (PrivateLink Gateway) access-point endpoints are correctly surfaced by confluent flink endpoint list and validated by confluent flink endpoint use.

Changes:

  • Add an EndpointClient to the shared pkg/ccloudv2.Client and implement a paginated ListEndpoints wrapper.
  • Switch flink endpoint list and the flink endpoint use validator to query the Endpoints API instead of aggregating legacy region/network APIs.
  • Extend the test server router with a mock /endpoint/v1/endpoints handler and remove the now-obsolete PLATT filter-map unit test.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/flink/command_endpoint_list.go Uses V2Client.ListEndpoints(..., service=FLINK) to build the endpoint list output.
internal/flink/command_endpoint_use.go Updates endpoint validation to check membership against Endpoints API results.
internal/flink/command_endpoint_test.go Removes unit test for legacy PLATT filter-map helper (helper removed).
pkg/ccloudv2/client.go Wires a new EndpointClient into the shared CCloud v2 client.
pkg/ccloudv2/endpoint.go Adds the Endpoints API client factory + paginated ListEndpoints implementation.
test/test-server/ccloudv2_router.go Routes /endpoint/v1/endpoints to a new mock handler.
test/test-server/endpoint_handlers.go Implements mock Endpoints API responses for specific (cloud, region) pairs.
go.mod Adds github.com/confluentinc/ccloud-sdk-go-v2/endpoint v0.4.0.
go.sum Adds sums for the new endpoint SDK dependency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/flink/command_endpoint_use.go
Comment thread test/test-server/endpoint_handlers.go
@jsnrth Jason Roth (jsnrth) force-pushed the FCP-4223/flink-endpoint-list-via-endpoints-api branch 3 times, most recently from 3a1b1cb to 772c216 Compare May 14, 2026 23:20
@jsnrth Jason Roth (jsnrth) marked this pull request as ready for review May 14, 2026 23:26
@jsnrth Jason Roth (jsnrth) requested a review from a team as a code owner May 14, 2026 23:26
Replace the three-source CLI-side aggregation (ListFlinkRegions +
ListNetworkPrivateLinkAttachments + ListNetworks + URL templating) with
a single call to the dedicated Endpoints API via
ccloud-sdk-go-v2/endpoint/v1, filtered by service=FLINK. This brings
the CLI in line with the API the UI and Terraform already use, and
surfaces endpoint shapes (notably multi-PLATT access-point URLs) that
the legacy URL-template approach could not represent.

To preserve the existing output contract: filter to endpoint_type=REST
(drops LANGUAGE_SERVICE / flinkpls.* rows), restore the https:// scheme
the legacy code produced, and uppercase the cloud column.

Tests: new test-server mock for /endpoint/v1/endpoints, plus a new
azure/italynorth region exercising a multi-PLATT GLB access-point URL.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jsnrth Jason Roth (jsnrth) force-pushed the FCP-4223/flink-endpoint-list-via-endpoints-api branch from 772c216 to b18ca44 Compare May 14, 2026 23:32
@jsnrth

Copy link
Copy Markdown
Contributor Author

Addressed all three points — force-pushed to b18ca44cf:

  1. Cloud casing — pushed normalization into ListEndpoints itself (cloud and service both strings.ToUpper-ed inside the wrapper, with a comment explaining why). Dropped the explicit uppercasing from both endpointList and the validator. Callers can pass either case without surprise.
  2. Error context on list path — restored fmt.Errorf("unable to list Flink endpoints: %w", err) wrapping.
  3. PR body — replaced PLACEHOLDER under Breaking Changes / Bug Fixes with - None, matching the convention in Add confluent endpoint list command #3268.

Build / lint / unit / integration tests all clean (same two pre-existing TestFlinkEndpointList failures on main for untouched-code backtick rendering).

@sgagniere

Copy link
Copy Markdown
Member

Overall it looks good to me.

But a question on this:

Fewer rows than before if the Endpoints API doesn't model an endpoint type the legacy code did. The API is the source of truth used by Terraform and the Cloud Console today, so any such gap would already be visible there.

What would be the impact of missing rows if they happened? If the endpoints API doesn't return them but the old code did, would those endpoints have been non-viable?

@jsnrth

Copy link
Copy Markdown
Contributor Author

If the endpoints API doesn't return them but the old code did, would those endpoints have been non-viable?

The "fewer rows" line was meant to flag the change in source of truth — but you're right that it reads alarming. The realistic worst case for a customer is "I see fewer rows because some PLATT/network is in a state the gateway doesn't route to yet" — which is the correct behavior.

@danielayaz

Daniel Ayaz (danielayaz) commented May 18, 2026

Copy link
Copy Markdown
Member

Overall looks good to me. Jason Roth (@jsnrth) Did you diff endpoint list output (old binary vs new) against the actual customer orgs from INC-10870 and INC-9186, or only against system-test orgs?

@jsnrth

Jason Roth (jsnrth) commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Did you diff endpoint list output (old binary vs new) against the actual customer orgs from INC-10870 and INC-9186, or only against system-test orgs?

Only against system test orgs. Result in this slack thread. We do not have access to customer orgs to run CLI commands.

@danielayaz Daniel Ayaz (danielayaz) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on my end, but holding off on GH approval until Steven approves here

@danielayaz Daniel Ayaz (danielayaz) merged commit af7e822 into confluentinc:main May 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants