Skip to content

feat(config): Add clouds add command - #1888

Open
dmbuil wants to merge 1 commit into
gtema:mainfrom
dmbuil:feature-config-clouds-add
Open

feat(config): Add clouds add command#1888
dmbuil wants to merge 1 commit into
gtema:mainfrom
dmbuil:feature-config-clouds-add

Conversation

@dmbuil

@dmbuil dmbuil commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Add osc config clouds add, a hand-written command that reads the JSON of an application credential from stdin and merges a ready-to-use cloud entry into clouds.yaml. Connection settings (auth_url, region, TLS options) are inherited from the cloud selected with --os-cloud; no authentication is performed.

Supports --split to keep the credential in a separate secure.yaml, --overwrite to replace an existing entry, and --file to target a specific path. Files carrying the secret are written with mode 0600 and the credential is zeroized in memory after use.

The command is dispatched in entry_point before a session is established, as it edits local files only.

Closes #1323

Add `osc config clouds add`, a hand-written command that reads the
JSON of an application credential from stdin and merges a
ready-to-use cloud entry into clouds.yaml. Connection settings
(auth_url, region, TLS options) are inherited from the cloud selected
with --os-cloud; no authentication is performed.

Supports --split to keep the credential in a separate secure.yaml,
--overwrite to replace an existing entry, and --file to target a
specific path. Files carrying the secret are written with mode 0600
and the credential is zeroized in memory after use.

The command is dispatched in entry_point before a session is
established, as it edits local files only.

Closes gtema#1323

Signed-off-by: dmbuil <dariomartinbuil@outlook.com>

@gtema gtema left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem that I see with this approach is that it overwrites the concrete yaml file (loosing comments and everything else in it). It would immediately break lot of things in the configs at least for me. What if instead of serde_yaml we use rust-yaml which is a "port" of python's ruamel that allows to preserve all comments of the yaml in the de/ser roundtrip to be less disruptive

}

/// One entry under the `clouds:` key of a clouds.yaml/secure.yaml file.
#[derive(Debug, Default, Serialize)]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it would be more logical to reuse structures from the sdk/core crate. This would enable adding not only app_cred connections, but also something else (who knows what) and at the same time prevent code deviation.

@dmbuil

dmbuil commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

One problem that I see with this approach is that it overwrites the concrete yaml file (loosing comments and everything else in it). It would immediately break lot of things in the configs at least for me. What if instead of serde_yaml we use rust-yaml which is a "port" of python's ruamel that allows to preserve all comments of the yaml in the de/ser roundtrip to be less disruptive

You're right about the file rewrite, and it can get worse if one uses advanced YAML features. Round-tripping through serde_yaml also expands anchors and destroys merge keys:

 # input                        # output
 # my clouds                    clouds:
 clouds:                          base:
   base: &base                      region_name: RegionOne
     region_name: RegionOne       devstack:
   devstack:                        <<:
     <<: *base                        region_name: RegionOne
     auth:                          auth:
       auth_url: 'https://…'          auth_url: https://…

(I already knew this issue from other projects such as lazy-ssh, but thought of it as a necessary evil)

So for anyone using &base / <<: *base across regions the file comes back structurally different, not just stripped of comments.

I did try rust-yaml before replying, and I don't think it gets us there. On this input:

 # Top of file comment
 # second line
 clouds:
   # this is my devstack
   devstack:
     auth_type: password       # inline comment

load_str_with_comments -> dump_str_with_comments (with preserve_comments: true, LoaderType::RoundTrip) reorders the two header comments, drops the inline comment entirely, moves # this is my devstack to the end of the file, and emits the header comments three more times after the document. It expands anchors the same way serde_yaml does.
To be sure it wasn't my poor reading skills, I ran the crate's own examples/comment_preservation_demo.rs unmodified: on its 4-line sample it duplicates every comment 3-4 times and appends them all after the document body (o_0). The README's "✅ Full Round-trip Comment Preservation" doesn't survive a real use case; it's a young crate (1.0 in May) and that feature may need more time in the oven.

Giving it a second thought, I'd suggest instead is to stop re-serializing the file at all: parse only to validate and to detect a name collision, then splice the rendered entry into the existing text and write the original bytes back untouched everywhere else: not as elegant, but at least should work.
Appending under clouds becomes byte-exact: anchors, quoting, blanklines... all should survive because we never round-trip them. --overwrite needs locating the existing entry's line range and replace it. Anything a line-oriented edit can't do safely (flow-style clouds: {…}, multi-document files) will error out with a clear message instead of silently rewriting.

On the other hand, serde_yaml is archived, and maybe worth revisiting... but I'd keep that separate from this, for now.

Happy to go implement the splice approach unless you'd rather take this somewhere else :)

@gtema

gtema commented Jul 29, 2026

Copy link
Copy Markdown
Owner

what about yamlpatch or yaml-edit?

@dmbuil

dmbuil commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

I thought of the harcoded serde_yaml not to include many other "foreign crates". But now this door has been opened, I've gone through a couple of them (using a little bit of AI for the comparison).

Crate Parses fixture Preserves Correct nesting MSRV ≤1.89
noyalib ✅ byte-exact ✅ 1.86
yamlpatch ✅ byte-exact ✅ (with workaround) ❌ 1.97
granit-parser n/a — parser only n/a ✅ 1.81
yaml-edit ❌ drops comments ❌ invalid YAML ✅ 1.70
rust-yaml ❌ scrambles/dupes ✅ 1.85
yaml_lib ❌ drops all ❌ corrupts anchors ✅ 1.88

Biggest downside of yamlpatch (the most popular) is that it imposes Rust min vers 1.97, which is way higher than OSC's 1.89. Next candidate would be noyalib, but it is a single-maintainer crate.
granit-parser would fit as well, but requires more careful implementation.

The other projects will scramble, drop or duplicate comments / advanced YAML configs.

Given that config.yaml is pretty important as to use any crate around, I wanted your guidance here, @gtema .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] output application credentials as clouds.yaml

2 participants