Skip to content

docs: explain S3 object-store configuration for SQL - #1716

Open
emecii wants to merge 1 commit into
apache:mainfrom
emecii:docs/s3-sql-object-store
Open

docs: explain S3 object-store configuration for SQL#1716
emecii wants to merge 1 commit into
apache:mainfrom
emecii:docs/s3-sql-object-store

Conversation

@emecii

@emecii emecii commented Sep 7, 2026

Copy link
Copy Markdown

Which issue does this PR close?

Closes #970.

Rationale for this change

CREATE EXTERNAL TABLE ... OPTIONS ('aws.…' ...) fails because aws is not a
SQL configuration namespace. The Python API already supports this workflow by
configuring an AmazonS3 object and registering it on the SessionContext, but
the object-store guide only showed register_parquet.

What changes are included in this PR?

  • Make the existing S3 example self-contained by adding its imports and
    SessionContext construction.
  • Document the supported S3-plus-SQL-DDL path after the object store is
    registered.
  • State that S3 credentials belong on AmazonS3, rather than in SQL OPTIONS.

Are there any user-facing changes?

Yes. The user guide now explains how to use an already registered S3 object
store with CREATE EXTERNAL TABLE.

Validation: checked the documentation diff, parsed all eight Python snippets in
the page with ast.parse, and verified the SQL DDL reaches the configured S3
object store with the published DataFusion wheel. The example is deliberately
documented as requiring valid S3 credentials.

I understand and reviewed this AI-assisted documentation change end-to-end.

Signed-off-by: Yifan Chen <emecii23@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aws.XXX options are not recognized in SQL API for CREATE EXTERNAL TABLE with AWS S3 location

1 participant