Skip to content

v3.2: $self field (Alternative Approach) #4556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: v3.2-dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
261 changes: 258 additions & 3 deletions src/oas.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,22 +308,53 @@ Note that some URI fields are named `url` for historical reasons, but the descri

Unless specified otherwise, all fields that are URIs MAY be relative references as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-4.2).

Relative references in [Schema Objects](#schema-object), including any that appear as `$id` values, use the nearest parent `$id` as a Base URI, as described by [JSON Schema Specification Draft 2020-12](https://www.ietf.org/archive/id/draft-bhutton-json-schema-01.html#section-8.2).
#### Establishing the Base URI

Relative URI references in other Objects, and in Schema Objects where no parent schema contains an `$id`, MUST be resolved using the referring document's base URI, which is determined in accordance with [[RFC3986]] [Section 5.1.2 – 5.1.4](https://tools.ietf.org/html/rfc3986#section-5.1.2).
In practice, this is usually the retrieval URI of the document, which MAY be determined based on either its current actual location or a user-supplied expected location.
Relative URI references are resolved using the appropriate base URI, which MUST be determined in accordance with [[RFC3986]] [Section 5.1.1 – 5.1.4](https://tools.ietf.org/html/rfc3986#section-5.1.1) and, for Schema objects, [JSON Schema draft 2020-12 Section 8.2](https://www.ietf.org/archive/id/draft-bhutton-json-schema-01.html#section-8.2), as illustrated by the examles in [Appendix G: Examples of Base URI Determination and Reference Resolution](#appendix-g-examples-of-base-uri-determination-and-reference-resolution).

If `$self` is a relative URI-reference, it is resolved agains the next possible base URI source ([[RFC3986]] [Section 5.1.2 – 5.1.4](https://tools.ietf.org/html/rfc3986#section-5.1.2)) before being used for the resolution of other relative URI-references.

The most common base URI source that is used in the event of a missing or relative `$self` (in the [OpenAPI Object](#openapi-object)) and (for [Schema Object](#schema-object)) `$id` is the retrieval URI.
Implementations MAY support document retrieval, although see the [Security Considerations](#security-considerations) sections for additional guidance.
Even if retrieval is supported, it may be impossible due to network configuration or server unavailability (including the server hosting an older version while a new version is in development), or undesirable due to performance impacts.
Therefore, all implementations SHOULD allow users to provide the intended retrieval URI for each document so that references can be resolved as if retrievals were performed.

#### Resolving URI fragments

If a URI contains a fragment identifier, then the fragment should be resolved per the fragment resolution mechanism of the referenced document. If the representation of the referenced document is JSON or YAML, then the fragment identifier SHOULD be interpreted as a JSON-Pointer as per [RFC6901](https://tools.ietf.org/html/rfc6901).

#### Relative URI References in CommonMark Fields

Relative references in CommonMark hyperlinks are resolved in their rendered context, which might differ from the context of the API description.

### Relative References in API URLs

API endpoints are by definition accessed as locations, and are described by this specification as **_URLs_**.

Unless specified otherwise, all fields that are URLs MAY be relative references as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-4.2).

Because the API Is a distinct entity from the OpenAPI Document, RFC3986's base URI rules for the OpenAPI Document do not apply.
Unless specified otherwise, relative references are resolved using the URLs defined in the [Server Object](#server-object) as a Base URL. Note that these themselves MAY be relative to the referring document.

#### Examples of API Base URL Determination

Assume a retrieval URI of `https://device1.example.com` for the following OpenAPI Document:

```YAML
openapi: 3.2.0
$self: https://apidescriptions.example.com/foo
info:
title: Example API
version: 1.0
servers:
- url: .
description: The production API on this device
- url: ./test
description: The test API on this device
```

For API URLs, the `$self` field, which identifies the OpenAPI Document, is ignored, and the retrieval URI is used instead. This produces a normalized production URL of `https://device1.example.com`, and a normalized test URL of `https://device1.example.com/test`.

### Schema

This section describes the structure of the OpenAPI Description format.
Expand All @@ -342,6 +373,7 @@ This is the root object of the [OpenAPI Description](#openapi-description).
| Field Name | Type | Description |
| ---- | :----: | ---- |
| <a name="oas-version"></a>openapi | `string` | **REQUIRED**. This string MUST be the [version number](#versions) of the OpenAPI Specification that the OpenAPI Document uses. The `openapi` field SHOULD be used by tooling to interpret the OpenAPI Document. This is _not_ related to the API [`info.version`](#info-version) string. |
| <a name="oas-self"></a>$self | `string` | This string MUST be in the form of a URI-reference as defined by [[RFC3986]] [Section 4.1](https://www.rfc-editor.org/rfc/rfc3986#section-4.1). The `$self` field provides the self-assigned URI of this document, which also serves as its base URI in accordance with [[RFC3986]] [Section 5.1.1](https://www.rfc-editor.org/rfc/rfc3986#section-5.1.1). Implementations MUST support identifying the targets of [API description URIs](#relative-references-in-api-description-uris) using the URI defined by this field when it is present. See [Establishing the Base URI](#establishing-the-base-uri) for the base URI behavior when `$self` is absent or relative, and see [Appendix G]((#appendix-g-examples-of-base-uri-determination-and-reference-resolution)) for examples of using `$self` to resolve references. |
| <a name="oas-info"></a>info | [Info Object](#info-object) | **REQUIRED**. Provides metadata about the API. The metadata MAY be used by tooling as required. |
| <a name="oas-json-schema-dialect"></a> jsonSchemaDialect | `string` | The default value for the `$schema` keyword within [Schema Objects](#schema-object) contained within this OAS document. This MUST be in the form of a URI. |
| <a name="oas-servers"></a>servers | [[Server Object](#server-object)] | An array of Server Objects, which provide connectivity information to a target server. If the `servers` field is not provided, or is an empty array, the default value would be a [Server Object](#server-object) with a [url](#server-url) value of `/`. |
Expand All @@ -354,6 +386,9 @@ This is the root object of the [OpenAPI Description](#openapi-description).

This object MAY be extended with [Specification Extensions](#specification-extensions).

Implementations MAY choose to support referencing OpenAPI Documents that contain `$self` by another URI such as the retrieval URI, however this behavior is not interoperable and relying on it is NOT RECOMMENDED.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this sentence.

If I want my OpenAPI Documents to be interoperable, what SHOULD I do when referencing (other) OpenAPI Documents that contain $self?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ralfhandl As stated in RFC2119, "SHOULD" and "RECOMMENDED" are equivalent, as are their negations. So anything that is NOT RECOMMENDED is something that you SHOULD NOT do, and therefore you SHOULD do the other thing (in this case, use the $self URI as your reference target).

I can add a sentence making that explicit if you think it is needed.


This came from a discussion with @karenetheridge about other ambiguities with $id. Let's say that you have enabled auto-retrieval of references, treating them as URLs (because you have some reason to believe that will work, like the OAD author said so, out-of-band). If you successfully retrieve a referenced document from X because one $ref used X, and it has a $self URI of Y... you've already retrieved it from X. Can you keep referencing it as X or do you have to reference it as Y, even though you know X works because you just got it there?

This is a weird case. With JSON Schema, we made a mistake and didn't talk about it, as @karenetheridge pointed out. But intuitively, most people seem to think that if you can use X successfully, and particularly if you already used X as the retrieval URL (actually retrieved, not just supplied as the "yeah if you retrieved this you woudl have gotten it from X" simulated retrieval URI) once and it worked, then X should keep working.

You can come up with a similar situation where the "other" URI is from an encapsulated entity (multipart/related example) or even an application-supplied default. These get harder and harder to contrive, but it's possible.

I don't want to require (MUST) that implementations support these messy situations. Implementations really SHOULD NOT perform unrestricted retrieval automatically as it's a security nightmare. To say nothing of the weird contortions needed to justify the other possible sources.

But if you are supporting retrieval, you can easily get into this situation where someone uses a retrieval URL that does not match $self, and it works, and you have to decide whether to keep letting it work or not.

This sentence allows implementations to keep letting that work if they want to, without requiring it. Given the history of $ref, particularly in 2.0 (and arguably 3.0) where it is strictly a URL, many tools almost certainly behave this way already. This sentence gives them the flexibility to leave that in place (and keep compatibility for their existing 3.1 customers) while adding the new 3.2 functionality.

Does that help? Again, I can tack on "Therefore, for interoperability, OAD authors SHOULD (or even MUST? Techncially it's not interoperable otherwise?) ensure their reference target URIs align with $self." or something to that effect if you think it is needed.

OAD authors MUST write references using the target document's `$self` URI in order to have interoperable behavior.

#### Info Object

The object provides metadata about the API.
Expand Down Expand Up @@ -482,6 +517,8 @@ An object representing a Server.

This object MAY be extended with [Specification Extensions](#specification-extensions).

See [Examples of API Base URL Determination](#examples-of-api-base-url-determination) for examples of resolving relative server URLs.

##### Server Object Example

A single server would be described as:
Expand Down Expand Up @@ -4895,3 +4932,221 @@ components:
```

In the `other` document, the referenced path item has a Security Requirement for a Security Scheme, `MySecurity`. The same Security Scheme exists in the original entry document. As outlined in [Resolving Implicit Connections](#resolving-implicit-connections), `MySecurity` is resolved with an [implementation-defined behavior](#undefined-and-implementation-defined-behavior). However, documented in that section, it is RECOMMENDED that tools resolve component names from the [entry document](#openapi-description-structure). As with all implementation-defined behavior, it is important to check tool documentation to determine which behavior is supported.

## Appendix G: Examples of Base URI Determination and Reference Resolution

This section shows each of the four possible sources of base URIs, followed by an example with a relative `$self` and `$id`.

### Base URI Within Content

A base URI within the resource's content (RFC3986 Section 5.1.1) is the highest-precedence source of a base URI.
For OpenAPI Documents, this source is the OpenAPI Object's `$self` field, while for Schema Objects that contain a `$id`, or are a subschema of a Schema Object containing a `$id`, the source is the `$id` field:

Assume the retrieval URI of the following document is `file://home/someone/src/api/openapi.yaml`:

```YAML
openapi: 3.2.0
$self: https://example.com/api/openapi
info:
title: Example API
version: 1.0
paths:
/foo:
get:
requestBody:
$ref: "shared/foo#/components/requestBodies/Foo"
```

Assume the retrieval URI for the following document is `https://git.example.com/shared/blob/main/shared/foo.yaml`:

```YAML
openapi: 3.2.0
$self: https://example.com/api/shared/foo
info:
title: Shared components for all APIs
version: 1.0
components:
requestBodies:
Foo:
content:
application/json:
schema:
$ref: ../schemas/foo
schemas:
Foo:
$id: https://example.com/api/schemas/foo
properties:
bar:
$ref: bar
Bar:
$id: https://example.com/api/schemas/bar
type: string
```

In this example, the retrieval URIs are irrelevant because both documents define `$self`.

For the relative `$ref` in the first document, it is resolved against `$self` to produce `https://example.com/api/shared/foo#/components/requestBodies/Foo`.
The portion of that URI before the '#' matches the `$self` of the second document, so the reference target is resolved to `#/components/requestBodies/Foo` in that second document.

In that document, the `$ref` in the Request Body Object is resolved using that document's `$self` as the base URI, producing `https://example.com/api/schemas/foo`.
This matches the `$id` at `#/components/schemas/Foo/$id` so it points to that Schema Object.
That Schema Object has a subschema with `$ref: bar`, which is resolved against the `$id` to produce `https://example.com/api/schemas/bar`, which matches the `$id` at `#/components/schemas/Bar/$id`.

Note that referring to a schema with a JSON Pointer that crosses a Schema Object with a `$id` [is not interoperable](https://www.ietf.org/archive/id/draft-bhutton-json-schema-01.html#name-json-pointer-fragments-and-).
The JSON Schema specification does not address the case of using a pointer _to_ a Schema Object containing an `$id` without crossing into that Schema Object.
Therefore it is RECOMMENDED that OAD authors use `$id` values to reference such schemas rather than JSON Pointers.

Note also that it is impossible for the reference at `#/components/schemas/Foo/properties/bar/$ref` to reference the schema at `#/components/schemas/Bar` using a JSON Pointer, as the JSON Pointer would be resolved relative to `https://example.com/api/schemas/foo`, not to the OpenAPI Document's base URI from `$self`.

### Base URI From Encapsulating Entity

If no base URI can be determined within the content, the next location to search is any encapsulating entity (RFC3986 Section 5.1.2).

This is common for Schema Objects encapsulated within an OpenAPI Document.
An example of an OpenAPI Document itself being encapsulated in another entity would be a `multipart/related` archive ([[?RFC2557]]), such as the following `multipart/related; boundary="boundary-example"; type="application/openapi+yaml"` document.
Note that this is purely an example, and support for such multipart documents or any other format that could encapsulate an OpenAPI Document is not a requirement of this specification.

RFC2557 was written to allow sending hyperlinked sets of documents as email attachments, in which case there would not be a retrieval URI for the multipart attachment (although the format could also be used in HTTP as well).

```MULTIPART
--boundary-example
Content-Type: application/openapi+yaml
Content-Location: https://inaccessible-domain.com/api/openapi.yaml

openapi: 3.2.0
info:
title: Example API
version: 1.0
externalDocs:
url: docs.html
components:
requestBodies:
Foo:
content:
application/json:
schema:
$ref: "#/components/api/schemas/Foo"
schemas:
Foo:
properties:
bar:
$ref: schemas/bar
--boundary-example
Content-Type: application/schema+json; schema=https://spec.openapis.org/oas/3.2/schema-base/YYYY-MM-DD
Content-Location: https://example.com/api/schemas/bar

{
"type": "string"
}
--boundary-example
Content-Type: text/html
Content-Location: https://example.com/api/docs.html

<html>
<head>
<title>API Documentation</title>
</head>
<body>
<p>Awesome documentation goes here</p>
</body>
</html>
--boundary-example
```

In this example, the URI for each part, which also serves as its base URI, comes from the part's `Content-Location` header as specified by RFC2557.
Since the Schema Object at `#/components/schemas/Foo` does not contain an `$id`, the reference in its subschema uses the OpenAPI Document's base URI, which is taken from the `Content-Location` header of its part within the `multipart/related` format.
The resulting reference to `https://example.com/schemas/bar` matches the `Content-Location` header of the second part, which according to RFC2557 allows the reference target to be located within the multipart archive.

Similarly, the `url` field of the [External Documentation Object](#external-documentation-object) is resolved against the base URI from `Content-Location`, producing `https://example.com/api/docs.html` which matches the `Content-Location` of the third part.

### Base URI From the Retrieval URI

If no base URI is provided from either of the previous sources, the next source is the retrieval URI (RFC 3986 Section 5.1.3).

For this example, assume that the YAML OpenAPI Document was retrieved from `https://example.com/api/openapis.yaml` and the JSON Schema document from `https://example.com/api/schemas/foo`

Assume this document was retrieved from `https://example.com/api/openapis.yaml`:

```YAML
openapi: 3.2.0
info:
title: Example API
version: 1.0
components:
requestBodies:
Foo:
content:
application/json:
schema:
$ref: schemas/foo
```

Assume this document was retrieved from `https://example.com/api/schemas/foo`:

```JSON
{
"type": "object",
"properties": {
"bar": {
"type": "string"
}
}
}
```

Resolving the `$ref: schemas/foo` against the retrieval URI of the OpenAPI Document produces `https://example.com/api/schemas/foo`, the retrieval URI of the JSON Schema document.

### Application-Specific Default Base URI

When constructing an OpenAPI Document in memory that does not have a `$self`, or an encapsulating entity, or a retrieval URI, applications can resolve internal (fragment-only) references by assuming a default base URI (RFC3986 Section 5.1.4).
While this sort of internal resolution an be performed in practice without choosing a base URI, choosing one, such as a URN with a randomly generated UUID (e.g. `urn:uuid:f26cdaad-3193-4398-a838-4ecb7326c4c5`) avoids the need to implement it as a special case.

### Resolving Relative `$self` and `$id`

Let's re-consider the first example in this appendix, but with relative URI-references for `$self` and `$id`, and retrieval URLs that support that relative usage:


Assume that the following is retrieved from `https://staging.example.com/api/openapi`:

```YAML
openapi: 3.2.0
$self: /api/openapi
info:
title: Example API
version: 1.0
paths:
/foo:
get:
requestBody:
$ref: "shared/foo#/components/requestBodies/Foo"
```

Assume the retrieval URI for the following document is `https://staging.example.com/api/shared/foo`:

```YAML
openapi: 3.2.0
$self: /api/shared/foo
info:
title: Shared components for all APIs
version: 1.0
components:
requestBodies:
Foo:
content:
application/json:
schema:
$ref: ../schemas/foo
schemas:
Foo:
$id: /api/schemas/foo
properties:
bar:
$ref: bar
Bar:
$id: /api/schemas/bar
type: string
```

In this example, All of the `$self` and `$id` values are relative URI-references consisting of an absolute path.
This allows the retrieval URL to set the host (and scheme), in this case `https://staging.example.com`, resulting in the first document's `$self` being `https://staging.example.com/openapi`, and the second document's `$self` being `https://staging.example.com/api/shared/foo`, with `$id` values of `https://staging.example.com/api/schemas/foo` and `https://staging.example.com/api/schemas/bar`.
Relative `$self` and `$id` values of this sort allow the same set of documents to work when deployed to other hosts, e.g. `https://example.com` (production) or `https://localhost:8080` (local development).
5 changes: 5 additions & 0 deletions src/schemas/validation/schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ properties:
openapi:
type: string
pattern: '^3\.2\.\d+(-.+)?$'
$self:
type: string
format: uri-reference
$comment: MUST NOT contain a fragment
pattern: '^[^#]*$'
info:
$ref: '#/$defs/info'
jsonSchemaDialect:
Expand Down