Wait for minio to initialize before running integration tests

[CharlieTLe]

The e2e readiness probe for minio polls /minio/health/live. MinIO answers that with 200 as soon as its HTTP server is up, before it can serve S3 requests. A test that uploads right after the probe passes can fail with Server not initialized yet, please try again.

This hit TestLoadRuntimeConfigFromCloudStorage on the amd64 requires_docker job for #7863 (run 36196173168); a re-run passed.

Failure output

=== RUN   TestLoadRuntimeConfigFromCloudStorage
22:33:21 Starting minio-9000
22:33:21 Ports for container: e2e-cortex-test-minio-9000 Mapping: map[9000:33276]
    runtime_config_test.go:141:
        	Error Trace:	/__w/cortex/cortex/integration/runtime_config_test.go:141
        	Error:      	Received unexpected error:
        	            	Server not initialized yet, please try again.
        	            	upload s3 object
        	            	github.com/thanos-io/objstore/providers/s3.(*Bucket).Upload
        	            		/__w/cortex/cortex/vendor/github.com/thanos-io/objstore/providers/s3/s3.go:581
        	            	github.com/cortexproject/cortex/integration.TestLoadRuntimeConfigFromCloudStorage
        	            		/__w/cortex/cortex/integration/runtime_config_test.go:141
        	Test:       	TestLoadRuntimeConfigFromCloudStorage
22:33:21 Killing minio-9000
--- FAIL: TestLoadRuntimeConfigFromCloudStorage (1.28s)

Root cause

In MinIO's cmd/healthcheck-handler.go (identical in RELEASE.2024-05-28T17-19-04Z and RELEASE.2024-07-04T14-25-45Z, so this predates #7864):

• LivenessCheckHandler (/minio/health/live) and ReadinessCheckHandler (/minio/health/ready) only set an x-minio-server-status: offline header while the object layer is nil, and still return 200.
• ClusterCheckHandler (/minio/health/cluster) goes through checkHealth, which returns 503 until the object layer, bucket metadata and IAM are all initialized.

Reproduced locally by polling both endpoints from container start: in 6 of 6 trials, /live returned 200 for 3 to 8 consecutive polls while /cluster did not.

Change

Probe /minio/health/cluster.

Testing

Against the v1.22.0-rc.2 image:

• TestLoadRuntimeConfigFromCloudStorage, 20 of 20 passed (-count=20).
• TestGettingStartedSingleProcessConfigWithBlocksStorage, TestGettingStartedWithGossipedRing, TestLoadRuntimeConfigFromStorageBackend pass.