feat(pprocutils,wasm): WASM host-egress ABI + guest stub (http_request) - #162
Merged
Conversation
…e, error codes, converters) Shared host/guest types for the WASM host-mediated network-egress capability (host function http_request), per docs/design-documents/20260726-wasm-host-egress-capability.md in ConduitIO/conduit. Adds: - proto/procutils/v1/http.proto + generated http.pb.go (HTTPHeader/HTTPRequest/HTTPResponse) - pprocutils.HTTPService interface + HTTPRequest/HTTPResponse Go structs (AuthSecretRef reserved) - pprocutils ABI error-code band entries (ErrorCodeHTTP*) - fromproto/toproto converters + generated HTTPService mock Guest-side wasm stub + embedding-processor consumer are deferred to later slices. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD
…ss capability The deferred guest half of bed0225 (shared HTTPService ABI). A standalone (WASM) processor can now reach the host-mediated network-egress capability via the egress package: - wasm/imports.go: //go:wasmimport conduit http_request, mirroring create_schema/get_schema exactly (same park/resize protocol). - wasm/http.go: httpService, the guest-side pprocutils.HTTPService — marshals via toproto.HTTPRequest, calls hostCall, unmarshals via fromproto.HTTPResponse. Verbatim reuse of the schemaService pattern. - wasm/util.go: InitUtils wires egress.HTTPService = &httpService{} at startup, alongside the existing schema wiring. - egress package (new): the processor-facing API (Do, Request, Response, sentinel Err* vars). Defaults to a deny-all HTTPService outside a standalone processor, since there is no host boundary to broker the call through in that hosting mode. Godoc states the security contract plainly: deny-by-default/operator-gated, host-injected credentials only, no redirect-following, host-size-capped responses. Per docs/design-documents/20260726-wasm-host-egress-capability.md in ConduitIO/conduit (host side: feat/wasm-host-egress-impl, host_module.go httpRequest). ABI verified symmetric against that branch: import name, proto types, and the numeric ErrorCodeHTTP* band all match byte-for-byte (both repos share this exact checkout via the host's go.mod replace during bundled development). Tests: egress package (mocked HTTPService, sentinel-error decoding via errors.Is, deny-all default) and a wire-level round-trip test for toproto/fromproto HTTPRequest/HTTPResponse — a gap the ABI commit (bed0225) left uncovered. The guest stub itself (wasm/http.go) cannot be exercised without a running wasm host; that path belongs in the bundle's cross-repo e2e test, not here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD
Contributor
|
The latest Buf updates on your PR. Results from workflow buf-validate / validate (pull_request).
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds the shared host/guest ABI and the guest-side stub for a new WASM host-egress capability — bounded, allowlisted outbound HTTP for standalone (WASM) processors. This is the SDK half of the host-egress bundle (design: conduit
docs/design-documents/20260726-wasm-host-egress-capability.md, §1 of the AI-pipeline components design). The host half (the wazerohttp_requesthost function + SSRF gate) lives inconduitio/conduit; the first consumer is the embedding processor inconduitio/conduit-processor-ai.Advances v0.20 WS8 (AI-pipeline components).
Contents (all ADDITIVE — no breaking change)
proto/procutils/v1/http.proto(+ generatedhttp.pb.go):HTTPRequest/HTTPResponsemessages (withAuthSecretReffor host-injected credentials).pprocutils:HTTPServiceinterface, 7ErrorCodeHTTP*codes appended to the existingErrorCodeStartiota band (no renumbering),fromproto/toprotoconverters, generated mock.wasm/://go:wasmimport conduit http_requestdeclaration + guest stub, mirroring the existing schema host-function stubs exactly (same buffer/park/resize + retry protocol, samepprocutils.Errornumeric-band decoding).egresspackage: the processor-facing guest API —egress.Do(ctx, egress.Request) (egress.Response, error)+ typed sentinel errors, with a deny-all default outside a WASM guest.egress/doc.gostates the four author-facing expectations (operator-gated/deny-by-default, host-injected credentials only, no redirect-following, size-capped responses).Risk tier: Tier-1-adjacent (public SDK ABI for a security boundary)
Additive only — no existing proto field/error code renumbered, so N-1 guests keep working. The ABI is verified byte-for-byte symmetric with the conduit host function (same generated proto package). The guest
httpService.Doonly executes under a real wazero host, so it is exercised in the cross-repo bundle e2e (in conduit), not unit-tested here; in-repo tests cover theegress.Doconversion, sentinel-error decoding, the deny-all default, and proto round-trips.Tests / checks
go build ./...,GOOS=wasip1 GOARCH=wasm go build -tags wasm ./...,go test ./...(567 pass), gofumpt, golangci-lint — all clean (verified locally in two hands). Lint had 2 pre-existing findings ininternal/reference/reference.go, unrelated.Adversarial self-review
Re-read for ABI drift vs the host (import name
conduit.http_request, marshalling, error band — symmetric), additive-only error-code numbering (confirmed, appended to iota), and the guest API's security-expectation docs matching the host gate's actual behavior. No open uncertainty.Merge ordering (bundle)
This SDK PR merges first so
conduitio/conduitandconduitio/conduit-processor-aican repoint theirgo.modoff a temporary localreplaceto a real SDK pseudo-version before their own PRs.