Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .changeset/ai-agent-authoring-and-tools-removal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
"@objectstack/spec": major
"@objectstack/lint": minor
---

feat(spec,lint)!: remove `agent.tools[]`, lint agent authoring, and resolve `action_<name>` only when it actually materialises (#3820, ADR-0109 accepted)

**Breaking — `agent.tools[]` is removed.** ADR-0064's central invariant is
"an agent's tool set is the union of its surface-compatible skills' tools;
nothing falls through to the global registry", and this legacy inline slot
was the one seam that broke it: the runtime resolved `agent.tools[].name`
against the **full** tool registry with no surface check, so an `ask`-surface
agent could name an authoring tool and get it. Removing the field makes the
invariant structural — there is no second slot to disagree with the skills —
rather than a rule every reader has to remember (ADR-0049 "design+enforce or
remove"). `AIToolSchema` / the `AITool` type go with it.

*Migration:* attach capability through `skills`. An agent authoring `tools` is
not a parse error — Zod strips the unknown key — so existing stacks keep
parsing, but the slot no longer does anything.

**`validate-ai-tool-references` now models AI exposure.** The rule previously
resolved `action_<name>` against every declared action. The runtime is far
stricter (ADR-0011): it materialises a tool only when the action opts in with
`ai.exposed: true` + `ai.description` **and** has a headless path (type
`script`/`api`/`flow` with a target or body — `url`/`modal`/`form` are
UI-only). Resolving against all actions therefore blessed references the agent
could never call — the exact failure the rule exists to catch. Unresolved
`action_*` references now get their own message and fix, since "the action
isn't exposed" and "the name is fictional" need different answers.

**New rule `validate-ai-agent-authoring`** (`agent-authoring-withdrawn`,
warning): flags a stack that declares `stack.agents`. Tenant/app-package
agents were withdrawn in ADR-0063 §2 — the runtime filters them from the
catalog and refuses to load them — but `defineStack` still accepted the array,
so an app could ship agents that parse, validate, and never run. This is the
authoring-time signal that was missing (ADR-0078: loud at the producer,
tolerant at the consumer). Joins `REFERENCE_INTEGRITY_RULES`.

ADR-0109 is now **Accepted — implemented (Phase 1)**, and the AI docs teach
the zero-tool-record default path, including the three conditions that decide
whether `action_<name>` exists and why a `modal` action staying human-driven
is a design answer rather than a gap.
106 changes: 82 additions & 24 deletions content/docs/ai/agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ description: The two platform agents (ask and build), how skills extend them, an
# AI Agents

Part of the [AI module](/docs/ai). In the **open edition**, agents, tools, and
skills are **typed metadata**: you author them as source with `defineAgent` /
`defineSkill` / `defineTool` from the open `@objectstack/spec/ai` package, and an
skills are **typed metadata**. You author **skills** (`defineSkill`) — agents are
platform-owned and tool records are optional (see below) — from the open
`@objectstack/spec/ai` package, and an
external AI client (Claude, Cursor, a local model — any MCP client) reaches your
objects, queries, and business **Actions** through `@objectstack/mcp` (BYO-AI),
all governed by RLS. This page describes that metadata and the `AgentSchema` it
Expand Down Expand Up @@ -112,12 +113,74 @@ assistant to *write* your metadata and never run. Same word, two layers.
`*.agent.ts` is **closed to third parties** — the `agent` metadata type is
`allowRuntimeCreate:false, allowOrgOverride:false`, reserved for the two platform
agents and platform-owned subagents (ADR-0063 §2). To give the `ask` agent a new
capability you author an agent **skill** (`*.skill.ts`) whose `tools` reference
your Actions / Flows / queries; it then attaches to `ask`. Every skill declares
capability you author an agent **skill** (`*.skill.ts`). Every skill declares
`surface: 'ask' | 'build' | 'both'`, and an agent's tool set is the **union of its
surface-compatible skills' tools** — there is no global fall-through, so a skill
reaches an agent only when their surfaces match
([ADR-0064](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0064-tool-scoping-to-agent.md)).
`os validate` warns if a stack declares an agent (`agent-authoring-withdrawn`).

### A skill needs **no tool records** — name the Action

Per [ADR-0109](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0109-ai-tool-authoring-model.md)
the default path declares **zero** `defineTool` records. The runtime already
materialises one `action_<name>` tool per AI-exposed Action, so a skill names
that tool directly. The Action is the same one your UI button runs, so
permissions, validation and audit are identical whether a human clicks or the
assistant calls:

```typescript
import { defineAction } from '@objectstack/spec/ui';
import { defineSkill } from '@objectstack/spec/ai';

// 1. The Action you already have — opt it in to AI.
export const ConvertLeadAction = defineAction({
name: 'convert_lead',
label: 'Convert Lead',
objectName: 'crm_lead',
type: 'flow', // headless: script | api | flow
target: 'lead_conversion_flow',
ai: {
exposed: true, // ADR-0011 — opt-in, default off
description: 'Converts a qualified lead into an account, contact and opportunity.',
},
});

// 2. The skill names its materialised tool. No defineTool anywhere.
export const LeadQualificationSkill = defineSkill({
name: 'lead_qualification',
label: 'Lead Qualification',
surface: 'ask',
instructions: `Score the lead with BANT from what you read, then convert it
when the user agrees. Scoring is your judgement — there is no scoring tool.`,
tools: ['get_record', 'query_records', 'action_convert_lead'],
});
```

Three things decide whether `action_<name>` exists, and `os validate` checks
all three (`ai-skill-tool-unresolved`):

| Requirement | Why |
|---|---|
| `ai.exposed: true` + `ai.description` (≥40 chars) | ADR-0011 governance gate — AI reach is opt-in, and the description is the LLM's contract |
| type is `script` / `api` / `flow` | `url` / `modal` / `form` are UI-only: they have no headless path, so they stay human-driven **by design** |
| a `target` (or a `body`, for `script`) | something has to dispatch |

A `modal` Action is not a gap to fix. When the operation should collect input
from a person — an escalation reason, a refund amount — leave it modal and have
the skill's instructions *recommend* the button instead of calling it.

The other tools a skill may name are the **platform** ones the runtime
registers (`describe_object`, `query_records`, `aggregate_data`,
`search_knowledge`, `visualize_data`, …). Between those and your Actions, most
skills need nothing else: "analyse the pipeline", "draft this email" and "score
this lead" are *reasoning* the model does with data — writing them as tool
names is the most common authoring mistake, and produces an assistant that
claims abilities it does not have.

A `defineTool` record is an **optional refinement layer**, not a required step
— reach for one only when the AI-facing surface must differ from the Action
itself (a different LLM-facing description, fewer exposed parameters).
Both `surface:'ask'` and `surface:'build'` skills run only where the in-UI AI
runtime exists — **ObjectOS**. On the open framework
there is no in-product agent to attach them to; author capability
Expand Down Expand Up @@ -193,12 +256,9 @@ Always be professional and data-driven.`,
maxTokens: 2000,
},

// References to Actions/Flows exposed as tools (see "Actions as Tools").
tools: [
{ type: 'flow', name: 'analyze_lead', description: 'Analyze a lead and provide a qualification score' },
{ type: 'flow', name: 'suggest_next_action', description: 'Suggest the next best action for an opportunity' },
{ type: 'action', name: 'generate_email', description: 'Generate a personalized email template' },
],
// Capability comes from skills — the only tool-bearing slot (ADR-0064).
// Each skill names the platform tools and `action_<name>` tools it needs.
skills: ['lead_qualification', 'opportunity_coaching', 'email_drafting'],

// RAG access: sources to recruit knowledge from + vector store indexes.
knowledge: {
Expand Down Expand Up @@ -233,11 +293,10 @@ Always be empathetic and solution-focused.`,
maxTokens: 1500,
},

// `search_knowledge` is provided by the Knowledge Protocol tool, not declared inline.
tools: [
{ type: 'flow', name: 'triage_case', description: 'Analyze a case and assign priority' },
{ type: 'action', name: 'generate_response', description: 'Generate a customer response' },
],
// Skills carry the tools. `case_triage` names the platform data tools it
// reads with; `knowledge_search` names `search_knowledge` (Knowledge
// Protocol). Neither needs a `defineTool` record.
skills: ['case_triage', 'knowledge_search', 'response_drafting'],

knowledge: {
sources: ['support-kb', 'cases'],
Expand Down Expand Up @@ -271,10 +330,10 @@ Use reputable data sources.`,
maxTokens: 1000,
},

tools: [
{ type: 'flow', name: 'lookup_company', description: 'Look up company information' },
{ type: 'flow', name: 'enrich_contact', description: 'Enrich contact information' },
],
// The enrichment Flows are exposed with `ai.exposed`, so the runtime
// materialises `action_lookup_company` / `action_enrich_contact` for the
// skill to name.
skills: ['contact_enrichment'],
});
```

Expand Down Expand Up @@ -307,11 +366,10 @@ Use the data tools to query records and aggregate metrics.`,
maxTokens: 3000,
},

// Built-in data tools (query_records / get_record / aggregate_data) are
// available to agents automatically once registered — see "Natural Language Queries".
tools: [
{ type: 'flow', name: 'analyze_pipeline', description: 'Analyze sales pipeline health' },
],
// Pipeline analysis is reasoning over the platform's own data tools
// (query_records / aggregate_data / visualize_data), which the skill
// names directly — see "Natural Language Queries".
skills: ['revenue_forecasting'],

knowledge: {
sources: ['opportunities', 'pipeline'],
Expand Down
21 changes: 4 additions & 17 deletions content/docs/getting-started/common-patterns.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -397,23 +397,10 @@ export const supportAgent = defineAgent({
instructions: `You are a helpful customer support agent.
You can search for customer records, look up order status,
and create support tickets. Always be polite and professional.`,
tools: [
{
type: 'query',
name: 'contact',
description: 'Search customer records'
},
{
type: 'query',
name: 'order',
description: 'Look up order status'
},
{
type: 'action',
name: 'create_support_ticket',
description: 'Create a support ticket'
}
]
// Capability lives in skills, never inline on the agent (ADR-0064): a
// skill names the platform data tools plus the `action_<name>` tools
// materialised from your own AI-exposed Actions.
skills: ['customer_lookup', 'order_status', 'ticket_intake']
});
```

Expand Down
18 changes: 2 additions & 16 deletions content/docs/references/ai/agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ AI Model Configuration
## TypeScript Usage

```typescript
import { AIKnowledge, AIModelConfig, AITool, Agent, StructuredOutputConfig, StructuredOutputFormat, TransformPipelineStep } from '@objectstack/spec/ai';
import type { AIKnowledge, AIModelConfig, AITool, Agent, StructuredOutputConfig, StructuredOutputFormat, TransformPipelineStep } from '@objectstack/spec/ai';
import { AIKnowledge, AIModelConfig, Agent, StructuredOutputConfig, StructuredOutputFormat, TransformPipelineStep } from '@objectstack/spec/ai';
import type { AIKnowledge, AIModelConfig, Agent, StructuredOutputConfig, StructuredOutputFormat, TransformPipelineStep } from '@objectstack/spec/ai';

// Validate data
const result = AIKnowledge.parse(data);
Expand Down Expand Up @@ -49,19 +49,6 @@ const result = AIKnowledge.parse(data);
| **topP** | `number` | optional | |


---

## AITool

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **type** | `Enum<'action' \| 'flow' \| 'query' \| 'vector_search'>` | ✅ | |
| **name** | `string` | ✅ | Reference name (Action Name, Flow Name) |
| **description** | `string` | optional | Override description for the LLM |


---

## Agent
Expand All @@ -79,7 +66,6 @@ const result = AIKnowledge.parse(data);
| **lifecycle** | `{ id: string; description?: string; contextSchema?: Record<string, any>; initial: string; … }` | optional | [EXPERIMENTAL — not enforced] State machine defining the agent conversation flow and constraints. Parsed but no runtime consumer yet (liveness #1878/#1893). |
| **surface** | `Enum<'ask' \| 'build'>` | ✅ | Product surface this agent binds ('ask' \| 'build') — ADR-0063 §1 |
| **skills** | `string[]` | optional | Skill names to attach (Agent→Skill→Tool architecture) |
| **tools** | `{ type: Enum<'action' \| 'flow' \| 'query' \| 'vector_search'>; name: string; description?: string }[]` | optional | Direct tool references (legacy fallback) |
| **knowledge** | `{ sources?: string[]; topics?: any; indexes: string[] }` | optional | RAG access |
| **active** | `boolean` | ✅ | |
| **access** | `string[]` | optional | Who can chat with this agent |
Expand Down
7 changes: 5 additions & 2 deletions docs/adr/0109-ai-tool-authoring-model.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# ADR-0109: The AI tool authoring model — the default third-party path needs no tool records

**Status**: Proposed (2026-07-28; revised same day — see Revision note). Phase 1 (platform tool-name registry + advisory reference lint) is implemented alongside this revision; Phase 2 (the optional refinement layer) awaits acceptance.
**Status**: **Accepted — implemented (Phase 1)** (2026-07-28; revised same day before acceptance — see Revision note).
Evidence: `packages/spec/src/system/constants/platform-tool-names.ts` (registry + `platform-tool-names.test.ts`); `packages/lint/src/validate-ai-tool-references.ts` (+ `.test.ts`, wired into `REFERENCE_INTEGRITY_RULES`); conformance in `../cloud` `service-ai`/`service-ai-studio` (`platform-tool-names-conformance.test.ts`, cloud#908). First app on the model: `../hotcrm` skills — 22 tool references, 0 findings, down from 16 references with 10 dead (hotcrm#512). Phase 2 (the optional refinement layer) remains open and is gated on a real refinement need.
**Deciders**: ObjectStack Protocol Architects
**Builds on**: [ADR-0063](./0063-two-kernel-agents-skills-are-the-extension-primitive.md) (skills + tools are the third-party extension primitive), [ADR-0064](./0064-tool-scoping-to-agent.md) (an agent's tools are its skills' tools), [ADR-0078](./0078-no-silently-inert-metadata.md) (no silently inert metadata), [ADR-0049](./0049-no-unenforced-security-properties.md) (enforce-or-remove)
**Consumers**: `@objectstack/spec` (`system/constants/platform-tool-names.ts`, `ai/tool.zod.ts`, `stack.zod.ts`), `@objectstack/lint` (`validate-ai-tool-references`), `../cloud/service-ai` + `service-ai-studio` (registry conformance)
Expand Down Expand Up @@ -130,5 +131,7 @@ lint could not see: no reference rule could be correct in either direction
- [x] `PLATFORM_PROVIDED_TOOL_NAMES` + `PLATFORM_TOOL_FAMILY_PREFIXES` + invariant tests (spec — this change).
- [x] `validate-ai-tool-references`, advisory, in `REFERENCE_INTEGRITY_RULES` (lint — this change).
- [x] `'tools'` into `composeStacks`' concat list (spec — this change).
- [ ] Registry conformance tests in `../cloud` `service-ai` / `service-ai-studio` (cloud PR; activates fully when the cloud `.objectstack-sha` pin advances past this change).
- [x] Registry conformance tests in `../cloud` `service-ai` / `service-ai-studio` (cloud#908; feature-detected, activates when the `.objectstack-sha` pin advances past Phase 1).
- [x] The lint rule resolves `action_<name>` only for actions that ACTUALLY materialise — `ai.exposed` + `ai.description` + a headless type (ADR-0011). Resolving against every declared action blessed references the agent could never call; caught while porting HotCRM, where 3 of 5 referenced actions are `type:'modal'` (UI-only).
- [x] First app ported: `../hotcrm` (hotcrm#512) — 10 fictional tools removed, the two withdrawn agents deleted, state changes routed through `action_<name>`.
- [ ] **Phase 2 (on acceptance + a real refinement need):** `binding` on `ToolSchema`; flow exposure; boot-mirror provenance guard; revisit `tool` metadata-type flags; ratchet the lint rule to error per ADR-0078 once the registry has soaked a release.
9 changes: 9 additions & 0 deletions packages/lint/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,15 @@ export type {
AiToolRefSeverity,
} from './validate-ai-tool-references.js';

export {
validateAiAgentAuthoring,
AGENT_AUTHORING_WITHDRAWN,
} from './validate-ai-agent-authoring.js';
export type {
AiAgentAuthoringFinding,
AiAgentAuthoringSeverity,
} from './validate-ai-agent-authoring.js';

// One entry point for the reference-resolution rules above (#3583 §5 D5).
// Adding a rule to `REFERENCE_INTEGRITY_RULES` runs it on `validate`, `lint`
// and `compile` at once — the CLI call sites do not change.
Expand Down
8 changes: 6 additions & 2 deletions packages/lint/src/reference-integrity-suite.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ describe('reference-integrity suite — membership', () => {
'validateFlowTemplatePaths',
'validateAiSurfaceAffinity',
'validateAiToolReferences',
'validateAiAgentAuthoring',
]);
});

Expand Down Expand Up @@ -111,6 +112,8 @@ describe('reference-integrity suite — every member actually runs', () => {
],
// validateAiSurfaceAffinity: an 'ask' agent binding a 'build' skill — the
// runtime throws on this at chat time (ADR-0064 §3).
// validateAiAgentAuthoring: declaring an agent at all is withdrawn
// (ADR-0063 §2) — one fixture, two rules.
agents: [{ name: 'helper', surface: 'ask', skills: ['metadata_authoring'] }],
// validateAiToolReferences: a tool name nothing declares, registers, or
// materialises (the HotCRM fictional-tool class).
Expand Down Expand Up @@ -149,6 +152,7 @@ describe('reference-integrity suite — every member actually runs', () => {
expect(rules).toContain('flow-template-unknown-field');
expect(rules).toContain('ai-skill-surface-mismatch');
expect(rules).toContain('ai-skill-tool-unresolved');
expect(rules).toContain('agent-authoring-withdrawn');
});

it('carries a gating flow-template finding through the suite (#3810)', () => {
Expand All @@ -170,9 +174,9 @@ describe('reference-integrity suite — every member actually runs', () => {
expect(typeof f.message).toBe('string');
expect(typeof f.hint).toBe('string');
}
// Object references run first, AI tool references last.
// Object references run first, agent-authoring last.
expect(findings[0].rule).toBe('object-reference-unknown');
expect(findings[findings.length - 1].rule).toBe('ai-skill-tool-unresolved');
expect(findings[findings.length - 1].rule).toBe('agent-authoring-withdrawn');
});

it('returns nothing for an empty stack', () => {
Expand Down
2 changes: 2 additions & 0 deletions packages/lint/src/reference-integrity-suite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import { validateTranslationReferences } from './validate-translation-references
import { validateFlowTemplatePaths } from './validate-flow-template-paths.js';
import { validateAiSurfaceAffinity } from './validate-ai-surface-affinity.js';
import { validateAiToolReferences } from './validate-ai-tool-references.js';
import { validateAiAgentAuthoring } from './validate-ai-agent-authoring.js';

export type ReferenceIntegritySeverity = 'error' | 'warning';

Expand Down Expand Up @@ -98,6 +99,7 @@ export const REFERENCE_INTEGRITY_RULES: readonly ReferenceIntegrityRule[] = [
{ name: 'validateFlowTemplatePaths', run: validateFlowTemplatePaths },
{ name: 'validateAiSurfaceAffinity', run: validateAiSurfaceAffinity },
{ name: 'validateAiToolReferences', run: validateAiToolReferences },
{ name: 'validateAiAgentAuthoring', run: validateAiAgentAuthoring },
];

/**
Expand Down
Loading
Loading