Skip to content

test(kubernetes): stop TEST 3 failing on the expected apiserver connect error#13701

Open
nic-6443 wants to merge 1 commit into
apache:masterfrom
nic-6443:fix/k8s-discovery-test3-flake
Open

test(kubernetes): stop TEST 3 failing on the expected apiserver connect error#13701
nic-6443 wants to merge 1 commit into
apache:masterfrom
nic-6443:fix/k8s-discovery-test3-flake

Conversation

@nic-6443

Copy link
Copy Markdown
Member

Description

t/kubernetes/discovery/kubernetes.t TEST 3 fails intermittently on master, for example in this run:

Failed test 't/kubernetes/discovery/kubernetes.t TEST 3: mixing set custom and default values -
pattern "[error]" should not match any line in error.log but matches line
"[error] ... [lua] informer_factory.lua:302: connect apiserver failed,
apiserver.host: sample.com, apiserver.port: 6443, message : connection refused,
context: ngx.timer"

TEST 3 is the only block in the file whose service.host is not the apiserver the job port-forwards to 127.0.0.1, so its informer can never connect. Configuring discovery.kubernetes starts the informer regardless, it fails to connect in an ngx.timer, and it logs an [error]. The block sets neither error_log nor no_error_log, so t/APISIX.pm gives it the default no_error_log: [error] and the block fails — but only when the connect fails early enough to land inside the block's own window. Since each block here has its own yaml_config, nginx restarts between blocks and a slow-enough resolve means the timer is killed before it ever logs. That is what makes it intermittent rather than a hard failure.

The block only compares the resolved configuration through /compare and never intends to reach an apiserver, so the connect error is expected noise rather than a defect. This overrides no_error_log for that block alone, using the same idiom already used elsewhere in t/.

The neighbouring blocks are deliberately left alone: they point at 127.0.0.1, which the job actually serves.

How this was verified

Running the file locally with nothing listening on 127.0.0.1:6443 inverts which host is reachable and reproduces the identical signature on the other blocks, while TEST 3 passes because the custom host happened to be connectable there:

TEST 1 ... matches line "[error] ... informer_factory.lua:302: connect apiserver failed,
            apiserver.host: 127.0.0.1, apiserver.port: 6443, message : connection refused,
            context: ngx.timer"
TEST 2 ... (same)
TEST 4 ... (same)
TEST 6 ... (same)
TEST 8 ... (same)

That confirms the mechanism: whichever block points at an apiserver the environment does not serve is the one the default [error] check trips on, and in CI that block is TEST 3. With an [error] present in TEST 3's window, the block fails before this change and passes after it.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

…ct error

TEST 3 is the only block in the file whose `service.host` is not the
apiserver the CI port-forwards to 127.0.0.1, so its informer can never
connect. The connect failure logs an `[error]`, and whether it lands inside
the block's window depends on how long resolving the host takes, so the
default `no_error_log: [error]` fails the block intermittently:

    TEST 3: mixing set custom and default values - pattern "[error]" should
    not match any line in error.log but matches line "... informer_factory.lua:302:
    connect apiserver failed, apiserver.host: sample.com, apiserver.port: 6443,
    message : connection refused, context: ngx.timer"

The block only compares the resolved configuration and never intends to
reach an apiserver, so the connect error is expected noise. Override
`no_error_log` for this block alone.

The neighbouring blocks are unaffected: they point at 127.0.0.1, which the
job actually serves. (Run the file without an apiserver on 127.0.0.1:6443
and the failure moves to them, with the same signature, while TEST 3 passes.)
@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant