[SPARK-58202][K8S] Skip driver containerPort declaration when the por…

[zhengchenyu]

What changes were proposed in this pull request?

BasicDriverFeatureStep skips containerPort entries whose port is 0.

Why are the changes needed?

When we use host network, we want to resolve port conflicts by binding port 0. But setting spark.driver.port / spark.driver.blockManager.port / spark.ui.port / spark.connect.grpc.binding.port to 0 makes driver Pod creation fail. The error log is as follows:

Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://XX.XX.XX.XX:443/api/v1/namespaces/default/pods. Message: Pod "xxx-driver" is invalid: [spec.containers[0].ports[0].containerPort: Required value, spec.containers[0].ports[1].containerPort: Required value ...

BasicExecutorFeatureStep already skips the entry when the port is 0. The driver side should follow the same pattern.

Does this PR introduce any user-facing change?

No

How was this patch tested?

• Unit tests in BasicDriverFeatureStepSuite.
• Test in kubernetes cluster.

[zhengchenyu]

@sunchao @dongjoon-hyun Can you please review this PR?