feat(stream): support set_real_ip_from to trust an inbound PROXY protocol header#13700
Draft
nic-6443 wants to merge 3 commits into
Draft
feat(stream): support set_real_ip_from to trust an inbound PROXY protocol header#13700nic-6443 wants to merge 3 commits into
nic-6443 wants to merge 3 commits into
Conversation
…ocol header `proxy_protocol_to_upstream` rebuilds the outbound PROXY protocol header from `c->sockaddr`, which is the directly connected peer. When APISIX sits behind a load balancer that speaks the PROXY protocol, that peer is the load balancer, so the client address parsed out of the inbound header is dropped on the way to the upstream. The stream `$remote_addr`, the access log and address-based matching are limited to the load balancer's address for the same reason. Add `nginx_config.stream.real_ip_from`, mirroring `nginx_config.http.real_ip_from`, which renders `set_real_ip_from` into the stream block. On a connection from a trusted address carrying an inbound PROXY protocol header, ngx_stream_realip_module writes the header's address back into `c->sockaddr`, so the rebuilt outbound header and every stream-side address consumer see the client. The option is empty by default, so the client address remains the directly connected peer unless a trusted network is configured. Requires a runtime built with `--with-stream_realip_module`.
The http block renders set_real_ip_from by default, so an unscoped grep over nginx.conf can never assert the stream side's default.
The tests overwrite conf/config.yaml wholesale, so the `yq` lookup depends on whatever admin key the deployment ends up with. Pin it in the test's own config instead, which keeps the tests hermetic and independent of the shipped default. Also document the combination that silently loses the client address: accepting and re-emitting the PROXY protocol without trusting any address.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A stream port can accept the PROXY protocol (
proxy_protocol) and send it on to the upstream (proxy_protocol_to_upstream), but the two don't compose into "pass the client address through".ngx_proxy_protocol_write()builds the outbound header fromc->sockaddr— the directly connected peer — and never looks atc->proxy_protocol. So with a load balancer in front, the upstream is told the load balancer's address and the client's is lost. The stream$remote_addr, the access log and address matching such asip-restrictionsee the load balancer for the same reason.nginx's answer is
ngx_stream_realip_module, but there was no way to configure it:set_real_ip_fromis only rendered into thehttpblock.This adds
nginx_config.stream.real_ip_from, mirroring the existingnginx_config.http.real_ip_from:On a connection from a trusted address that carries an inbound PROXY protocol header, the module writes the header's address into
c->sockaddr, so the rebuilt outbound header — and every stream-side consumer of the client address — carries the client. The directly connected address stays available as$realip_remote_addr.The option is empty by default, so nothing changes for existing configurations. It only takes effect on ports that accept the PROXY protocol, and only for peers it matches.
Depends on #13697
ngx_stream_realip_moduleis not built by default, soset_real_ip_fromis an unknown directive on runtimes before 1.3.11. #13697 bumpsAPISIX_RUNTIMEto 1.3.11, which adds--with-stream_realip_module. This PR stays in draft until that lands; CI here cannot pass before it.Tests
t/cli/test_stream_proxy_protocol.shgrows four cases. Two cover rendering: nothing is trusted by default, and eachreal_ip_fromentry renders oneset_real_ip_from. The assertions are scoped to the stream block, since the http block renders the same directive by default.Two are end to end against a PROXY-protocol-aware stream upstream that echoes the address it was handed, so they assert on the header APISIX actually rebuilds rather than on
nginx.conf:198.51.100.7reaches the upstream as198.51.100.7;Both were run locally against apisix-runtime 1.3.11.
Checklist