Implement $fy: rule-based shell to mjs translation (JS + Rust)

.gitattributes

@@ -0,0 +1,3 @@
+# The `$fy` fixtures are compared byte for byte by both test suites, so they
+# must check out with LF endings on every platform, Windows included.
+fixtures/fy/* text eol=lf

fixtures/fy/README.md

@@ -0,0 +1,21 @@
+# `$fy` translation fixtures
+
+`sample.sh` is a shell script exercising every construct the `$fy` translator
+supports; `sample.mjs` is the module both implementations must produce from it.
+
+Both test suites assert against these files, which is what keeps the JavaScript
+and Rust translators byte-for-byte equivalent:
+
+- `js/tests/fy-tool.test.mjs`
+- `rust/tests/fy.rs`
+
+Regenerate `sample.mjs` after an intentional change to the rules — either
+implementation produces the same bytes, which is the point of the fixture:
+
+```sh
+cd rust && cargo run --quiet --example fy_translate -- ../fixtures/fy/sample.sh > ../fixtures/fy/sample.mjs
+```
+
+```sh
+cd js && bun -e "import {translateShellToMjs} from './src/fy/index.mjs'; import {readFileSync} from 'node:fs'; process.stdout.write(translateShellToMjs(readFileSync('../fixtures/fy/sample.sh','utf8')).code)" > ../fixtures/fy/sample.mjs
+```

fixtures/fy/sample.mjs

@@ -0,0 +1,41 @@
+#!/usr/bin/env node
+import { $, shell } from 'command-stream';
+
+const args = process.argv.slice(2);
+let TARGET;
+
+// deploy helper
+shell.set("e");
+
+TARGET = `${args[0] ?? `staging`}`;
+process.env.REGION = `eu`;
+
+async function deploy(...args) {
+  let tag = ``;
+  tag = `${(await $`git rev-parse --short HEAD`).stdout.trim()}`;
+  await $`echo "deploying ${tag} to ${TARGET}" | tee deploy.log`;
+}
+
+if ((await $`[ -d dist ]`).code === 0) {
+  await $`deploy`;
+} else {
+  await $`echo "nothing to deploy" >&2`;
+}
+
+for (const env of [`staging`, `prod`]) {
+  await $`echo "${env}"`;
+}
+
+switch (`${TARGET}`) {
+  case `staging`:
+    await $`deploy`;
+    break;
+  default:
+    await $`echo unknown`;
+    break;
+}
+
+if ((await $`ls && echo ok`).code !== 0) {
+  await $`echo failed`;
+}
+process.exit(0);

fixtures/fy/sample.sh

@@ -0,0 +1,30 @@
+#!/bin/sh
+# deploy helper
+set -e
+
+TARGET=${1:-staging}
+export REGION=eu
+
+deploy() {
+  local tag
+  tag="$(git rev-parse --short HEAD)"
+  echo "deploying $tag to $TARGET" | tee deploy.log
+}
+
+if [ -d dist ]; then
+  deploy
+else
+  echo "nothing to deploy" >&2
+fi
+
+for env in staging prod; do
+  echo "$env"
+done
+
+case "$TARGET" in
+  staging) deploy ;;
+  *) echo unknown ;;
+esac
+
+ls && echo ok || echo failed
+exit 0

js/.changeset/fy-rule-based-translation.md

@@ -0,0 +1,10 @@
+---
+'command-stream': minor
+---
+
+Add the `$fy` virtual command, a shell-to-JavaScript translator built as a
+rule-based translation on top of link-foundation/meta-language: the script is
+first formalized as a links network, then rewritten into a command-stream module
+by a `TranslationRuleSet`. Pipelines, `&&`/`||`, `if`/`while`/`until`/`for`/`case`,
+functions, redirects, assignments and `${...}` expansions translate structurally
+rather than textually, and untranslated constructs are reported as diagnostics.

js/docs/meta-language-gaps.md

@@ -0,0 +1,162 @@
+# Missing meta-language features found while building `$fy`
+
+`$fy` translates shell scripts into command-stream modules using the model that
+[link-foundation/meta-language](https://github.com/link-foundation/meta-language)
+defines: a source text is _formalized_ into a links network, and the network is
+then _substituted_ into a target language through a `TranslationRuleSet`.
+
+The data model carried the whole design — `LinkNetwork`, `LinkType`,
+`LinkQuery`, `TranslationRule`, `TranslationTemplate` are exactly the right
+primitives, and `js/src/fy/shell-formalizer.mjs` and
+`js/src/fy/translation-rules.mjs` use them directly. What is missing is the
+_evaluation strategy_ that turns a rule set into output. That gap is filled
+locally by `js/src/fy/rule-engine.mjs` (~215 lines), which is written so that it
+collapses into a single upstream call once these features land.
+
+Version under test: **meta-language 0.46.0** (the current npm `latest`). Note
+that the repository's `main` is already at 0.54.0, so npm publishing lags the
+repository — that alone is worth fixing, since consumers cannot try newer work.
+
+Every gap below is reproduced by `js/experiments/meta-language-gaps.mjs`; run
+`node js/experiments/meta-language-gaps.mjs` to see the observed output.
+
+## 1. `TranslationRuleSet.render` does not substitute placeholders
+
+The most important gap. `render()` returns the matched template's text
+verbatim:
+
+```js
+ruleSet.render(network, linkId, 'JavaScript');
+// template: 'await $`{body}`;'
+// returns:  'await $`{body}`;'   <- `{body}` is never replaced
+```
+
+Without substitution a rule set can only emit constants, so it cannot express
+any translation whose output depends on the source.
+
+**Needed:** a placeholder syntax bound to the matched link's references, e.g.
+`{name}` resolving through a rule-declared capture map.
+
+## 2. No recursion into nested nodes
+
+Related to 1: even given substitution, a template's placeholders must be filled
+by _recursively rendering_ the captured child links. Translating `cd /tmp && ls`
+requires rendering the `and` node's two operand nodes with their own rules.
+Today only one rule fires, at one level.
+
+**Needed:** bottom-up rendering, where a placeholder recursively applies the
+rule set to the captured link.
+
+## 3. Only the first matching rule is ever applied
+
+`render()` picks the first rule whose `LinkQuery` matches and stops. A real
+rule set has one rule per construct and needs all of them applied across the
+network — rule _selection per link_, not per network.
+
+**Needed:** per-link rule resolution (each link claimed by its best-matching
+rule), which is what `RuleEngine`'s `rulesByLink` map does.
+
+## 4. No reference-capture API on `TranslationRule`
+
+A rule needs to name its children so templates can refer to them
+(`{condition}`, `{body}`). `TranslationRule` exposes no such API, so `$fy`
+attaches a plain `referenceCaptures` property and its own engine reads it:
+
+```js
+const created = new TranslationRule(term, syntax(term));
+created.referenceCaptures = { condition: 0, body: 1 }; // no upstream equivalent
+```
+
+**Needed:** `rule.withCapture(name, index)` (or capture names on the query
+itself, which would be nicer — it lets a query bind names structurally rather
+than positionally).
+
+This one is **version-specific**: the Rust crate at 0.54.0 already has
+`TranslationRule::with_reference_capture`, so it only needs to be ported to
+JavaScript and published to npm.
+
+## 5. No variadic placeholder for a node's children
+
+A shell script, a `block`, a pipeline and a word list all have an unbounded
+number of children joined by a separator (`"\n"`, `" | "`, `" "`). There is no
+way to express "render every reference and join them".
+
+**Needed:** something like `{*children|separator}`. `$fy` implements
+`{*name:mode|sep}` with `\n`/`\t`/`\s` escapes in the separator.
+
+## 6. No conditional/optional template segments
+
+A command may or may not have an assignment prefix; an absent capture currently
+has no defined behaviour and the placeholder text is emitted literally.
+
+**Needed:** `{?name}…{/name}` segments, plus the rule that an unresolved
+placeholder renders as empty rather than as its own source text.
+
+## 7. No notion of a target _sub_-language / rendering context
+
+The same shell node must render differently by context. `ls -la` is a statement
+(``await $`ls -la`;``) at the top level but a fragment (`ls -la`) inside a
+pipeline, and a variable is `NAME` as an expression but `${NAME}` inside a
+template literal. `$fy` models this with four target languages —
+`JavaScript`, `JavaScript:command`, `JavaScript:value`, `JavaScript:expression`
+— plus a fallback chain, because meta-language has no first-class concept of a
+rendering context.
+
+**Needed:** either sub-languages with declared fallbacks, or per-placeholder
+context selection, so context-dependence stays declarative instead of leaking
+into imperative code.
+
+## 8. No automatic indentation of multi-line substitutions
+
+When a rendered child spans several lines and is substituted after leading
+whitespace, its continuation lines must be indented to match, or every nested
+block comes out ragged. `$fy` does this in `indentContinuation`.
+
+**Needed:** indentation-aware substitution (the placeholder's column becomes
+the continuation indent).
+
+## 9. No builtin `Shell` language profile, and `parse()` produces no syntax nodes
+
+`LanguageProfile.builtin()` resolves only `javascript`/`js`, and `parse()` on
+Shell source tokenises per character without producing shell syntax nodes. This
+is expected for a young project rather than a defect, but it means `$fy` has to
+supply its own parser (`js/src/fy/shell-script-parser.mjs`) and formalizer.
+
+**Needed (lower priority):** a `Shell` profile, or documentation stating that
+front-ends are expected to be supplied by the consumer.
+
+## 10. No public dynamic-arity link insertion (Rust only)
+
+Found while porting `$fy` to Rust against the crates.io crate at **0.54.0**.
+`LinkNetwork::insert_link` and `insert_syntax_node` are const-generic over the
+reference count:
+
+```rust
+pub fn insert_syntax_node<const N: usize>(&mut self, language: &str, kind: &str, refs: [LinkId; N]) -> LinkId
+```
+
+so the arity must be a compile-time constant, while a parser produces nodes of
+arity known only at run time (a script has as many statements as it has). The
+one dynamic entry point, `insert_dynamic_link`, is `pub(crate)`.
+
+`rust/src/fy/formalizer.rs` works around this by matching on the child count up
+to a fixed maximum and grouping any wider node into nested `chunk` nodes that
+`rust/src/fy/engine.rs` flattens again — output-neutral, but it means the
+network shape no longer matches the source tree.
+
+**Needed:** a public `insert_link(&mut self, language, kind, refs: &[LinkId])`
+(or simply making `insert_dynamic_link` public). The JavaScript package is
+unaffected: `insertSyntaxNode` already takes an array.
+
+## Summary
+
+Gaps 1–3 block rule-based translation entirely; 4–8 are what a usable rule
+language needs in practice; 9 is scope; 10 is Rust-only ergonomics that a
+run-time parser cannot avoid. `js/src/fy/rule-engine.mjs` is a working
+reference implementation of 1–8 over meta-language's own types and is offered
+upstream.
+
+Gaps 1–3 were re-confirmed against the Rust crate at 0.54.0 by reading
+`translation_rules.rs`: `TranslationRuleSet::render` is `pub(crate)`, applies
+only the first matching rule, does not recurse, and does not substitute — so
+`rust/src/fy/engine.rs` is the same reference implementation in Rust.