[FLINK-40190][python] Add PyFlink DataFrame creation and conversion APIs

[auroflow]

What is the purpose of the change

This pull request completes the data creation and conversion APIs for the PyFlink DataFrame API described in FLINK-40190.

It adds creation from pandas DataFrames, PyArrow tables, PyFlink Tables, dictionaries, records, and integer ranges. It also supports conversion back to Table and pandas, explicit column names, and event-time watermarks where applicable.

Brief change log

• Add from_pandas(), from_arrow(), from_table(), and range(), and complete the existing from_dict() and from_records() creators.
• Add DataFrame.to_table() and DataFrame.to_pandas().

Verifying this change

This change added tests and can be verified as follows:

• Added plain unit tests for schema and watermark validation, record normalization, range behavior, Arrow batch splitting, and empty Arrow streams.
• Added planner-backed tests for pandas and Arrow schema inference, positional renaming, typed empty inputs, watermark metadata, and Table identity.
• Added wrapper tests for to_table() and to_pandas().
• Added an integration smoke test covering from_arrow() → DataFrame transformation → to_pandas().

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes
• The serializers: yes — Python in-memory and Arrow source serialization only; no state or persisted serialization format changes
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? PyDocs and public API docstrings

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Codex (GPT-5)

[flinkbot]

CI report:

• 66e5281 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build