merge: merge main to feat/migtate-e2e

Author: hopelynd

AGENTS.md

@@ -104,6 +104,17 @@ CLI 只为「自己能权威解释的错误」发出语义化信号,服务端的
 
 如果命令调用 Console Gateway,`defineCommand` 必须设置 `auth: "console"`。runtime 会基于 `CONSOLE_AUTH_FLAGS` 自动在 help 中展示 `--console-region`、`--console-site`、`--console-switch-agent`、`--workspace-id`,并由 `authStage` 解析/注入 console credential。命令不要重复声明这些凭证域 flag,也不要手动从 env/config 解析 token。
 
+### 5. 禁止单字母变量命名
+
+所有变量、参数、回调形参必须使用有语义的命名,不允许单字母(如 `i`、`m`、`p`、`t`、`e`、`s`)。具体表现:
+
+- 回调参数: `.map((m) => ...)` → `.map((model) => ...)`, `.find((t) => ...)` → `.find((template) => ...)`
+- catch 变量: `catch (e)` → `catch (error)`
+- for-of 循环: `for (const i of items)` → `for (const item of items)`
+- 临时变量: `const s = ...` → `const strategy = ...`
+
+例外: 仅当作用域极小(≤3 行)且语义从上下文完全明确时,可使用 `k`/`v`(Object.entries 的 key/value)。
+
 ## 完成改动后的快速验证
 
 ```sh

README.md

@@ -38,7 +38,7 @@ Equip your AI Agent out-of-the-box with these capabilities, composable across co
 - **MCP integration** — Orchestrate Bailian MCP servers: list services, inspect tools, and invoke any tool directly from the terminal
 - **Web search** — Real-time internet retrieval for up-to-date, accurate answers
 - **Model recommendation** — Describe your scenario and get best-fit model suggestions; supports scoped search, model comparison, and alternative discovery
-- **Fine-tuning & deployment** — Upload datasets, create SFT/LoRA/DPO/CPT jobs (`finetune create`), probe job status non-blockingly (`finetune watch`), query per-model training capability (`finetune capability`), and deploy trained models as endpoints (`deploy create`)
+- **Fine-tuning & deployment** — Upload datasets, create text/audio/image fine-tune jobs (`finetune text|audio|image create`; text covers SFT/LoRA/DPO/CPT), probe job status non-blockingly (`finetune watch`), query per-model training capability (`finetune capability`), and deploy trained models as endpoints (`deploy text|audio|image create`)
 - **Console capabilities** — Browse Bailian apps (`app list`), check free-tier quota (`usage free`), view model usage statistics (`usage stats`), manage workspaces (`workspace list`), and manage rate limits (`quota list/request/check/history`)
 - **Local file auto-upload** — Every URL parameter accepts a local path; uploaded to free temp storage with 48-hour validity
 
@@ -114,10 +114,10 @@ bl auth login --console
 
 # Fine-tune & deploy — a one-shot train-to-serve workflow
 bl dataset upload --file ./train.jsonl                 # Upload a .jsonl dataset (validated first)
-bl finetune create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # Local paths auto-upload
+bl finetune text create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # Local paths auto-upload
 bl finetune watch --job-id ft-xxx --output json       # Non-blocking status probe (exit 0/1/3 = done/failed/running)
 bl finetune capability --model qwen3-8b               # Which training types a model supports
-bl deploy create --model qwen3-8b --name my-svc --plan mu  # Deploy the trained model as an endpoint
+bl deploy text create --model qwen3-8b --name my-svc --plan mu  # Deploy the trained model as an endpoint
 
 # Browse apps / free-tier quota / usage statistics / workspaces
 bl app list

README.zh.md

@@ -38,7 +38,7 @@ _专为 AI Agent 打造，每个命令均可作为结构化工具调用。_
 - **MCP 集成** — 统一调度百炼 MCP 服务：列出服务、查看工具、直接在终端调用任意工具
 - **联网搜索** — 实时互联网信息检索，提升回答准确性及时效性
 - **模型推荐** — 描述你的场景，智能推荐最适合的模型；支持限定范围搜索、模型对比和替代发现
-- **微调与部署** — 上传数据集、创建 SFT/LoRA/DPO/CPT 调优任务（`finetune create`）、非阻塞探测任务状态（`finetune watch`）、按模型查训练能力（`finetune capability`），并把训练好的模型部署为推理服务（`deploy create`）
+- **微调与部署** — 上传数据集、创建文本/音频/图像调优任务（`finetune text|audio|image create`；文本涵盖 SFT/LoRA/DPO/CPT）、非阻塞探测任务状态（`finetune watch`）、按模型查训练能力（`finetune capability`），并把训练好的模型部署为推理服务（`deploy text|audio|image create`）
 - **控制台能力** — 浏览百炼应用（`app list`），查询模型免费额度（`usage free`），查看模型用量统计（`usage stats`），管理业务空间（`workspace list`），管理限流与提额（`quota list/request/check/history`）
 - **本地文件自动上传** — 所有 URL 参数同时支持本地路径，免费临时存储 48 小时
 
@@ -112,10 +112,10 @@ bl auth login --console
 
 # 微调与部署 — 从训练到服务的一站式流程
 bl dataset upload --file ./train.jsonl                 # 上传 .jsonl 数据集（先校验）
-bl finetune create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # 本地路径自动上传
+bl finetune text create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # 本地路径自动上传
 bl finetune watch --job-id ft-xxx --output json       # 非阻塞状态探测（退出码 0/1/3 = 成功/失败/进行中）
 bl finetune capability --model qwen3-8b               # 查询模型支持哪些训练方式
-bl deploy create --model qwen3-8b --name my-svc --plan mu  # 把训练好的模型部署为推理服务
+bl deploy text create --model qwen3-8b --name my-svc --plan mu  # 把训练好的模型部署为推理服务
 
 # 浏览应用 / 免费额度 / 用量统计 / 业务空间
 bl app list

packages/cli/README.md

@@ -38,7 +38,7 @@ Equip your AI Agent out-of-the-box with these capabilities, composable across co
 - **MCP integration** — Orchestrate Bailian MCP servers: list services, inspect tools, and invoke any tool directly from the terminal
 - **Web search** — Real-time internet retrieval for up-to-date, accurate answers
 - **Model recommendation** — Describe your scenario and get best-fit model suggestions; supports scoped search, model comparison, and alternative discovery
-- **Fine-tuning & deployment** — Upload datasets, create SFT/LoRA/DPO/CPT jobs (`finetune create`), probe job status non-blockingly (`finetune watch`), query per-model training capability (`finetune capability`), and deploy trained models as endpoints (`deploy create`)
+- **Fine-tuning & deployment** — Upload datasets, create text/audio/image fine-tune jobs (`finetune text|audio|image create`; text covers SFT/LoRA/DPO/CPT), probe job status non-blockingly (`finetune watch`), query per-model training capability (`finetune capability`), and deploy trained models as endpoints (`deploy text|audio|image create`)
 - **Console capabilities** — Browse Bailian apps (`app list`), check free-tier quota (`usage free`), view model usage statistics (`usage stats`), manage workspaces (`workspace list`), and manage rate limits (`quota list/request/check/history`)
 - **Local file auto-upload** — Every URL parameter accepts a local path; uploaded to free temp storage with 48-hour validity
 
@@ -114,10 +114,10 @@ bl auth login --console
 
 # Fine-tune & deploy — a one-shot train-to-serve workflow
 bl dataset upload --file ./train.jsonl                 # Upload a .jsonl dataset (validated first)
-bl finetune create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # Local paths auto-upload
+bl finetune text create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # Local paths auto-upload
 bl finetune watch --job-id ft-xxx --output json       # Non-blocking status probe (exit 0/1/3 = done/failed/running)
 bl finetune capability --model qwen3-8b               # Which training types a model supports
-bl deploy create --model qwen3-8b --name my-svc --plan mu  # Deploy the trained model as an endpoint
+bl deploy text create --model qwen3-8b --name my-svc --plan mu  # Deploy the trained model as an endpoint
 
 # Browse apps / free-tier quota / usage statistics / workspaces
 bl app list

packages/cli/README.zh.md

@@ -38,7 +38,7 @@ _专为 AI Agent 打造，每个命令均可作为结构化工具调用。_
 - **MCP 集成** — 统一调度百炼 MCP 服务：列出服务、查看工具、直接在终端调用任意工具
 - **联网搜索** — 实时互联网信息检索，提升回答准确性及时效性
 - **模型推荐** — 描述你的场景，智能推荐最适合的模型；支持限定范围搜索、模型对比和替代发现
-- **微调与部署** — 上传数据集、创建 SFT/LoRA/DPO/CPT 调优任务（`finetune create`）、非阻塞探测任务状态（`finetune watch`）、按模型查训练能力（`finetune capability`），并把训练好的模型部署为推理服务（`deploy create`）
+- **微调与部署** — 上传数据集、创建文本/音频/图像调优任务（`finetune text|audio|image create`；文本涵盖 SFT/LoRA/DPO/CPT）、非阻塞探测任务状态（`finetune watch`）、按模型查训练能力（`finetune capability`），并把训练好的模型部署为推理服务（`deploy text|audio|image create`）
 - **控制台能力** — 浏览百炼应用（`app list`），查询模型免费额度（`usage free`），查看模型用量统计（`usage stats`），管理业务空间（`workspace list`），管理限流与提额（`quota list/request/check/history`）
 - **本地文件自动上传** — 所有 URL 参数同时支持本地路径，免费临时存储 48 小时
 
@@ -112,10 +112,10 @@ bl auth login --console
 
 # 微调与部署 — 从训练到服务的一站式流程
 bl dataset upload --file ./train.jsonl                 # 上传 .jsonl 数据集（先校验）
-bl finetune create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # 本地路径自动上传
+bl finetune text create --model qwen3-8b --datasets ./train.jsonl --training-type sft-lora  # 本地路径自动上传
 bl finetune watch --job-id ft-xxx --output json       # 非阻塞状态探测（退出码 0/1/3 = 成功/失败/进行中）
 bl finetune capability --model qwen3-8b               # 查询模型支持哪些训练方式
-bl deploy create --model qwen3-8b --name my-svc --plan mu  # 把训练好的模型部署为推理服务
+bl deploy text create --model qwen3-8b --name my-svc --plan mu  # 把训练好的模型部署为推理服务
 
 # 浏览应用 / 免费额度 / 用量统计 / 业务空间
 bl app list

packages/cli/src/commands.ts

@@ -52,7 +52,9 @@ import {
   datasetGet,
   datasetDelete,
   datasetValidate,
-  finetuneCreate,
+  finetuneTextCreate,
+  finetuneAudioCreate,
+  finetuneImageCreate,
   finetuneList,
   finetuneGet,
   finetuneCancel,
@@ -62,7 +64,9 @@ import {
   finetuneExport,
   finetuneWatch,
   finetuneCapability,
-  deployCreate,
+  deployTextCreate,
+  deployAudioCreate,
+  deployImageCreate,
   deployList,
   deployGet,
   deployModels,
@@ -133,7 +137,9 @@ export const commands: Record<string, AnyCommand> = {
   "dataset get": datasetGet,
   "dataset delete": datasetDelete,
   "dataset validate": datasetValidate,
-  "finetune create": finetuneCreate,
+  "finetune text create": finetuneTextCreate,
+  "finetune audio create": finetuneAudioCreate,
+  "finetune image create": finetuneImageCreate,
   "finetune list": finetuneList,
   "finetune get": finetuneGet,
   "finetune cancel": finetuneCancel,
@@ -143,7 +149,9 @@ export const commands: Record<string, AnyCommand> = {
   "finetune export": finetuneExport,
   "finetune watch": finetuneWatch,
   "finetune capability": finetuneCapability,
-  "deploy create": deployCreate,
+  "deploy text create": deployTextCreate,
+  "deploy audio create": deployAudioCreate,
+  "deploy image create": deployImageCreate,
   "deploy list": deployList,
   "deploy get": deployGet,
   "deploy models": deployModels,

packages/commands/src/commands/dataset/upload.ts

@@ -6,6 +6,7 @@ import {
   parseDatasetSchemaFlag,
   formatIssue,
   MAX_DATASET_BYTES,
+  MAX_MEDIA_ZIP_BYTES,
   BailianError,
   ExitCode,
   type DatasetFile,
@@ -17,7 +18,7 @@ const UPLOAD_FLAGS = {
   file: {
     type: "string",
     valueHint: "<path>",
-    description: "Local .jsonl dataset file (≤300MB)",
+    description: "Local dataset file (.jsonl or .zip; ≤300MB text, ≤1GB image)",
     required: true,
   },
   purpose: {
@@ -29,7 +30,7 @@ const UPLOAD_FLAGS = {
     type: "string",
     valueHint: "<s>",
     description:
-      'Record schema: "chatml" (SFT), "dpo" (chosen/rejected), or "cpt" (raw text). Default auto-detects per record.',
+      'Record schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), or "image" (image generation). Default auto-detects per record.',
   },
   noValidate: {
     type: "switch",
@@ -42,42 +43,55 @@ const UPLOAD_FLAGS = {
 } satisfies FlagsDef;
 
 export default defineCommand({
-  description: "Upload a dataset file (.jsonl) to Bailian",
+  description: "Upload a dataset file (.jsonl or .zip) to Bailian",
   auth: "apiKey",
   usageArgs:
-    "--file <path> [--purpose <name>] [--schema <chatml|dpo|cpt>] [--no-validate] [--full-validate]",
+    "--file <path> [--purpose <name>] [--schema <chatml|dpo|cpt|tts|image>] [--no-validate] [--full-validate]",
   flags: UPLOAD_FLAGS,
   exampleArgs: [
     "--file train.jsonl",
     "--file dpo.jsonl --schema dpo",
     "--file cpt.jsonl --schema cpt",
+    "--file audio.zip --schema tts",
     "--file eval.jsonl --purpose evaluation",
     "--file train.jsonl --full-validate",
     "--file train.jsonl --no-validate",
   ],
   notes: [
-    "Only .jsonl is supported in this release. Three record schemas are",
-    "recognized: chatml = {messages:[...]} (SFT); dpo = {messages:[...],",
-    "chosen, rejected} where chosen/rejected are single assistant messages;",
-    'cpt = {text:"..."} (continual pre-training, raw text). With no --schema,',
-    "a record carrying chosen/rejected is validated as DPO, one with text (and",
-    "no messages) as CPT, otherwise as ChatML. Pass --schema dpo / cpt to",
-    "require that shape on every record, or --schema chatml to ignore the",
-    "preference / text fields. Other purposes may carry a different schema in",
-    "the future and would be served by a purpose-specific validator.",
-    "The dataset upload cap is 300MB per file.",
-    "Upload uses the OpenAI-compatible /compatible-mode/v1/files endpoint so",
-    "the purpose tag is persisted (the DashScope-native /api/v1/files drops it).",
+    "Supports .jsonl (text) and .zip (audio/image archives with a data.jsonl",
+    "manifest). Five record schemas are recognized: chatml = {messages:[...]}",
+    '(SFT); dpo = {messages:[...], chosen, rejected}; cpt = {text:"..."}',
+    '(continual pre-training, raw text); tts = {wav_fn:"train/xxx.wav",',
+    'text:"..."} (audio fine-tuning); image = {img_path:"..."} (image',
+    "generation). With no --schema, a record carrying wav_fn is validated as",
+    "TTS, img_path as image, chosen/rejected as DPO, text (no messages) as CPT,",
+    "otherwise ChatML. Upload cap: 300MB text, 1GB image. Upload uses the",
+    "OpenAI-compatible /compatible-mode/v1/files endpoint so the purpose tag is",
+    "persisted (the DashScope-native /api/v1/files drops it).",
   ],
   async run(ctx) {
     const { identity, settings, flags } = ctx;
     const filePath = flags.file;
     const purpose = flags.purpose || "fine-tune";
     const schema = parseDatasetSchemaFlag(flags.schema);
+    if (schema === "video") {
+      throw new BailianError(
+        `--schema video is not supported.`,
+        ExitCode.USAGE,
+        `Supported schemas: chatml, dpo, cpt, tts, image.`,
+      );
+    }
     const format = detectOutputFormat(settings.output);
+    // Image schema allows larger ZIPs (1 GB vs 300 MB for text).
+    const isMediaSchema = schema === "image";
 
     if (!flags.noValidate) {
-      const result = await validateDataset(filePath, { fullValidate: flags.fullValidate, schema });
+      const maxBytes = isMediaSchema ? MAX_MEDIA_ZIP_BYTES : MAX_DATASET_BYTES;
+      const result = await validateDataset(filePath, {
+        fullValidate: flags.fullValidate,
+        schema,
+        maxBytes,
+      });
       if (!result.valid) {
         const lines = [
           `Dataset validation failed for ${filePath}`,
@@ -112,7 +126,7 @@ export default defineCommand({
           action: "dataset.upload",
           file: filePath,
           purpose,
-          max_bytes: MAX_DATASET_BYTES,
+          max_bytes: isMediaSchema ? MAX_MEDIA_ZIP_BYTES : MAX_DATASET_BYTES,
           validate: !flags.noValidate,
           schema: schema ?? "auto",
         },

packages/commands/src/commands/dataset/validate.ts

@@ -25,7 +25,7 @@ const VALIDATE_FLAGS = {
   file: {
     type: "string",
     valueHint: "<path>",
-    description: "Local .jsonl dataset file",
+    description: "Local dataset file (.jsonl or .zip)",
     required: true,
   },
   fullValidate: {
@@ -36,38 +36,49 @@ const VALIDATE_FLAGS = {
     type: "string",
     valueHint: "<s>",
     description:
-      'Record schema: "chatml" (SFT), "dpo" (chosen/rejected), or "cpt" (raw text). Default auto-detects per record.',
+      'Record schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), or "image" (image generation). Default auto-detects per record.',
   },
 } satisfies FlagsDef;
 
 export default defineCommand({
-  description: "Locally validate a dataset file (.jsonl) without uploading",
+  description: "Locally validate a dataset file (.jsonl or .zip) without uploading",
   // 纯本地校验，不触网、不需 API key（与 `pipeline validate` 一致）。
   auth: "none",
-  usageArgs: "--file <path> [--full-validate] [--schema <chatml|dpo|cpt>]",
+  usageArgs: "--file <path> [--full-validate] [--schema <chatml|dpo|cpt|tts|image>]",
   flags: VALIDATE_FLAGS,
   exampleArgs: [
     "--file train.jsonl",
     "--file dpo.jsonl --schema dpo",
     "--file cpt.jsonl --schema cpt",
+    "--file audio.zip --schema tts",
     "--file eval.jsonl --full-validate",
     "--file train.jsonl --output json",
   ],
   notes: [
     "Default scan: every line gets a structural check, then ~160 lines (front 50,",
     "evenly spaced 100, last 10) are JSON.parsed against the active schema.",
     "Schemas: chatml = {messages:[...]} (SFT); dpo = {messages:[...], chosen,",
-    "rejected} where chosen/rejected are single assistant messages; cpt =",
-    '{text:"..."} (continual pre-training, raw text). With no --schema, a',
-    "record carrying chosen/rejected is validated as DPO, one with text (and no",
-    "messages) as CPT, otherwise as ChatML. Pass --schema dpo / cpt to require",
-    "that shape on every record (strict), or --schema chatml to ignore the",
-    "preference / text fields. Use --full-validate to JSON.parse every line.",
+    'rejected}; cpt = {text:"..."} (continual pre-training, raw text);',
+    'tts = {wav_fn:"train/xxx.wav", text:"..."} (audio fine-tuning);',
+    'image = {img_path:"..."} (image generation). With no --schema, a record',
+    "carrying wav_fn is validated as TTS, img_path as image, chosen/rejected",
+    "as DPO, text (no messages) as CPT, otherwise ChatML. Pass --schema to",
+    "require a specific shape on every record. ZIP archives (.zip) are",
+    "validated structurally (data.jsonl present, media references resolve) in",
+    "addition to per-record content checks. Use --full-validate to JSON.parse",
+    "every line.",
   ],
   async run(ctx) {
     const { settings, flags } = ctx;
     const filePath = flags.file;
     const schema = parseDatasetSchemaFlag(flags.schema);
+    if (schema === "video") {
+      throw new BailianError(
+        `--schema video is not supported.`,
+        ExitCode.USAGE,
+        `Supported schemas: chatml, dpo, cpt, tts, image.`,
+      );
+    }
     const format = detectOutputFormat(settings.output);
 
     if (settings.dryRun) {