Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
workflow_dispatch:
inputs:
release_type:
description: Release type for the root, native, and compatibility packages
description: Release type for the root and native packages
required: true
type: choice
default: patch
Expand Down Expand Up @@ -213,7 +213,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NODE_AUTH_TOKEN: ${{ secrets.PACKAGE_PUBLISH_TOKEN || secrets.GITHUB_TOKEN }}
run: npm run publish:${{ inputs.release_type }} -- --package root --package native --package js-fallback
run: npm run publish:${{ inputs.release_type }} -- --package root --package native

- name: Create or update GitHub Release
env:
Expand All @@ -225,16 +225,13 @@ jobs:
root_package_tag="@lzehrung/codegraph@$root_version"
native_version="${{ needs.plan-release.outputs.native_version }}"
native_tag="@lzehrung/codegraph-native@$native_version"
fallback_version=$(node -p "JSON.parse(require('node:fs').readFileSync('packages/codegraph-js-fallback/package.json', 'utf8')).version")
fallback_tag="@lzehrung/codegraph-js-fallback@$fallback_version"
tarball=$(npm pack --ignore-scripts --json | jq -r '.[0].filename')
release_notes=$(mktemp)
{
echo "Published packages:"
echo ""
echo "- \`$root_package_tag\`"
echo "- \`$native_tag\`"
echo "- \`$fallback_tag\`"
} > "$release_notes"
if gh release view "$root_tag" >/dev/null 2>&1; then
gh release edit "$root_tag" --title "$root_tag" --notes-file "$release_notes"
Expand Down
23 changes: 7 additions & 16 deletions PUBLISHING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
# Publishing Guide

Codegraph publishes as three standalone packages:
Codegraph publishes as two standalone packages:

- `@lzehrung/codegraph`: the main JS package and CLI
- `@lzehrung/codegraph-native`: the optional native Tree-sitter meta package plus per-platform binary packages
- `@lzehrung/codegraph-js-fallback`: a compatibility shim package with no grammar bundle

The main package depends on the native package optionally, so installs still succeed when no matching native binary exists. The compatibility shim remains standalone and versioned separately for package continuity.
The main package depends on the native package optionally, so installs still succeed when no matching native binary exists. When native is unavailable, Codegraph degrades to reduced graph-only and regex recovery mode; there is no separate JavaScript parser fallback package.

## Fast Path

Expand All @@ -29,15 +28,14 @@ npm run publish:resume
`publish:*` does the same work and also publishes:

- staged native target packages plus the `@lzehrung/codegraph-native` meta package when `@lzehrung/codegraph-native` is selected
- the `@lzehrung/codegraph-js-fallback` shim package when it changed
- the root `@lzehrung/codegraph` package when it changed

When a publish includes `@lzehrung/codegraph-native`, the script verifies the complete supported native target set before running the full test suite. A local WSL/macOS/Windows shell can only build its own target, so use the `release` GitHub Actions workflow or manually collect every target artifact before running a native publish locally.

You can force a package-scoped release with `--package`:

```powershell
npm run publish:patch -- --package js-fallback
npm run publish:patch -- --package root
npm run release:minor -- --package @lzehrung/codegraph-native
```

Expand All @@ -47,8 +45,8 @@ Use the `release` GitHub Actions workflow when you want GitHub to cut a complete

- Trigger it manually with `release_type=patch|minor|major`.
- The workflow builds every target declared in `packages/codegraph-native/package.json` `napi.targets`.
- It collects the per-target npm package artifacts, then runs `npm run publish:<release_type> -- --package root --package native --package js-fallback`.
- The publish step verifies that every supported native target artifact is present before publishing target packages, the native meta package, the compatibility js-fallback shim package, and the root package.
- It collects the per-target npm package artifacts, then runs `npm run publish:<release_type> -- --package root --package native`.
- The publish step verifies that every supported native target artifact is present before publishing target packages, the native meta package, and the root package.
- On success it pushes all package tags, creates or updates the overall `vX.Y.Z` GitHub Release for the root package version, and uploads the root `.tgz` asset.
- The workflow refuses reruns from a commit that is already tagged for the current root version. A fresh Actions runner cannot reconstruct the dirty local resume state that `publish:resume` expects.
- By default the workflow uses `GITHUB_TOKEN` for GitHub Packages. If an existing package is not linked to this repository or does not grant this repository write access, create a `PACKAGE_PUBLISH_TOKEN` Actions secret from a classic PAT that can write the `@lzehrung` packages; the workflow will use it for npm publishing.
Expand All @@ -62,9 +60,6 @@ Use the `release` GitHub Actions workflow when you want GitHub to cut a complete
- `@lzehrung/codegraph-native`
- Publishes the meta package.
- Resolves and loads the correct per-platform binary package.
- `@lzehrung/codegraph-js-fallback`
- Publishes the compatibility shim package.
- Does not ship grammar dependencies or depend on the root package through a local `file:` dependency.

## Manual Native Staging

Expand Down Expand Up @@ -124,14 +119,10 @@ npm run publish:native:meta
npm publish
```

9. Publish the fallback package when it changed:

```powershell
npm publish --workspace=@lzehrung/codegraph-js-fallback
```
The root package should be published last so its optional native dependency points at the final native meta version.

## Release Notes

- `@lzehrung/codegraph`, `@lzehrung/codegraph-native`, and `@lzehrung/codegraph-js-fallback` version independently.
- `@lzehrung/codegraph` and `@lzehrung/codegraph-native` version independently.
- `src/native/treeSitterNative.ts` prefers the installed `@lzehrung/codegraph-native` package and falls back to the local workspace package for development.
- If a native binary or query is unavailable at runtime, Codegraph degrades to reduced graph-only and regex recovery mode.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ npm i -g @lzehrung/codegraph
npm i -g https://github.com/lzehrung/codegraph/releases/download/vVERSION/lzehrung-codegraph-VERSION.tgz
```

Replace `VERSION` with the release you want. The root tarball does not bundle the native addon; source-language parsing still needs the scoped native package path via the `@lzehrung` GitHub Packages registry. Without it, Codegraph runs in reduced graph-only mode. The `@lzehrung/codegraph-js-fallback` package is a compatibility shim, not an alternate parser bundle.
Replace `VERSION` with the release you want. The root tarball does not bundle the native addon; source-language parsing still needs the scoped native package path via the `@lzehrung` GitHub Packages registry. Without it, Codegraph runs in reduced graph-only and regex recovery mode.

## FAQ

Expand Down Expand Up @@ -347,8 +347,8 @@ npm run publish:major
npm run publish:resume
```

Use `--package root`, `--package native`, or `--package js-fallback`, or a full package name when you need to force a specific package.
Use `--package root`, `--package native`, or a full package name when you need to force a specific package.

For GitHub-driven releases, use the manual `release` Actions workflow with `patch`, `minor`, or `major`. It publishes the root, native, and compatibility js-fallback shim packages together, pushes their tags, then creates or updates the overall `vX.Y.Z` GitHub Release with the packed root tarball asset. The workflow refuses reruns on an already-tagged release commit because fresh Actions runners cannot reconstruct the local `publish:resume` state.
For GitHub-driven releases, use the manual `release` Actions workflow with `patch`, `minor`, or `major`. It publishes the root and native packages together, pushes their tags, then creates or updates the overall `vX.Y.Z` GitHub Release with the packed root tarball asset. The workflow refuses reruns on an already-tagged release commit because fresh Actions runners cannot reconstruct the local `publish:resume` state.

For the detailed release flow, native artifact staging, and tag behavior, see [PUBLISHING.md](./PUBLISHING.md).
3 changes: 0 additions & 3 deletions codegraph-skill/codegraph/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,6 @@ Use the scoped packages only:

- package: `@lzehrung/codegraph`
- native backend: `@lzehrung/codegraph-native`
- compatibility shim: `@lzehrung/codegraph-js-fallback`

The compatibility shim is not a grammar fallback. Do not suggest installing it to restore source-language parsing when the native backend is unavailable.

Registry:

Expand Down
2 changes: 0 additions & 2 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ Requirements and install paths for local source checkouts, published packages, r
- Published installs do not require Rust or a manual native setup step on supported targets.
- Local source checkouts do not require Rust just to build `dist/`, but the native workspace addon only builds when Cargo is available.
- If no compatible native artifact is available, Codegraph drops to reduced graph-only and regex recovery mode instead of loading JS grammars.
- The root package does not install a JavaScript grammar fallback. `@lzehrung/codegraph-js-fallback` is only a compatibility shim package and does not provide parser recovery unless a future release documents that capability explicitly.

## Option 1: Local source checkout

Expand Down Expand Up @@ -84,7 +83,6 @@ Explicit CLI, library, and tool `native` options take precedence over `CODEGRAPH

- `@lzehrung/codegraph`: main library and CLI
- `@lzehrung/codegraph-native`: optional native runtime package that resolves the matching binary artifact
- `@lzehrung/codegraph-js-fallback`: compatibility shim package with no grammar bundle; installing it separately does not restore non-native Tree-sitter parsing

Reduced mode preserves graph-only and regex-backed recovery where available; it does not provide a non-native Tree-sitter parser.

Expand Down
14 changes: 6 additions & 8 deletions docs/plans/2026-05-12-graph-first-language-expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

## Context

The repo has an established language-support workflow: define support surface, add a language definition, wire JS fallback and native grammars, add fixtures/tests, then update parity/scenario docs and the agent skill surface. Follow [docs/adding-language-support.md](../../adding-language-support.md) exactly.
The repo has an established language-support workflow: define support surface, add a language definition, wire native grammars and reduced-mode behavior, add fixtures/tests, then update parity/scenario docs and the agent skill surface. Follow [docs/adding-language-support.md](../../adding-language-support.md) exactly.

This plan intentionally targets graph-first support. Each language should provide parsing, chunking, top-level symbol extraction, and static dependency edges. It should not claim full cross-file semantic navigation until shared `goto`, `references`, and native semantic parity tests prove that behavior.

Expand Down Expand Up @@ -41,8 +41,8 @@ The product boundary is important: graph-first languages may participate in depe
For each language, v1 support should provide:

- File discovery for listed extensions.
- Tree-sitter parsing through JS fallback.
- Native parser support when the Rust grammar exposes a compatible API.
- Tree-sitter parsing through the native runtime when the Rust grammar exposes a compatible API.
- Reduced-mode graph and regex recovery that stays safe when native is unavailable.
- Chunking for top-level declarations and important nested declarations.
- Definition extraction for top-level names.
- Static import/include/dependency extraction.
Expand Down Expand Up @@ -76,7 +76,6 @@ Register them in:
- `src/languages/all.ts`
- `src/languages.ts`
- `src/util/projectFiles.ts` if discovery is not fully driven by definitions
- `packages/codegraph-js-fallback/package.json`
- `packages/codegraph-native/Cargo.toml`
- `packages/codegraph-native/src/languages.rs`

Expand Down Expand Up @@ -255,16 +254,15 @@ git commit -m "docs: define graph-first language tier"

Files:

- `packages/codegraph-js-fallback/package.json`
- `packages/codegraph-native/Cargo.toml`
- `packages/codegraph-native/src/languages.rs`
- `tests/native-tree-sitter.test.ts`
- `tests/native-parser-ownership.test.ts`

Changes:

- Add JS grammar dependencies for all seven languages.
- Add Rust grammar dependencies for all seven languages.
- Keep reduced-mode behavior safe without adding a JavaScript grammar fallback.
- Register native IDs using stable lowercase language ids:
- `scala`
- `lua`
Expand All @@ -287,7 +285,7 @@ npx vitest run tests/native-tree-sitter.test.ts tests/native-parser-ownership.te
Commit:

```bash
git add packages/codegraph-js-fallback/package.json packages/codegraph-native/Cargo.toml packages/codegraph-native/src/languages.rs package-lock.json tests/native-tree-sitter.test.ts tests/native-parser-ownership.test.ts
git add packages/codegraph-native/Cargo.toml packages/codegraph-native/src/languages.rs package-lock.json tests/native-tree-sitter.test.ts tests/native-parser-ownership.test.ts
git commit -m "feat: add graph-first language parsers"
```

Expand Down Expand Up @@ -563,8 +561,8 @@ git commit -m "docs: finalize graph-first language support"

## Acceptance Criteria

- All seven languages parse through JS fallback.
- All seven languages parse through native runtime, unless a documented grammar compatibility blocker is found and reflected in parity docs.
- All seven languages retain safe reduced-mode behavior when native is unavailable.
- File discovery includes the planned extensions.
- Each language has a focused `tests/languages/*.test.ts` suite and fixtures.
- Each language extracts top-level declarations and static dependency edges listed in this plan.
Expand Down
6 changes: 3 additions & 3 deletions docs/plans/2026-06-06-performance-and-cache-opportunities.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,8 @@ Implementation checklist:
- [x] Define a native API that accepts source text and returns compact duplicate-token preprocessing results.
- [x] Start with native tokenization and normalization while keeping shingle/winnow fingerprints and candidate-pair generation in TypeScript.
- [x] Keep final reporting, grouping, and CLI JSON schema in TypeScript initially.
- [x] Preserve JS fallback behavior when native is unavailable or disabled.
- [x] Add parity fixtures comparing native and JS fallback duplicate results for exact, renamed, near, and low-confidence cases.
- [x] Preserve reduced-mode behavior when native is unavailable or disabled.
- [x] Add parity fixtures comparing native and reduced-mode duplicate results for exact, renamed, near, and low-confidence cases.
- [ ] Add a benchmark fixture with many repeated units and a performance threshold or regression report.

Likely files:
Expand Down Expand Up @@ -429,7 +429,7 @@ Checklist:
- [x] Anonymous JS/TS default exports should resolve through default imports.
- [x] JS/TS shorthand binding nodes should navigate to the actual binding/import.
- [x] Breaking-change suggestions should handle multiline exported signatures.
- [x] Publishing/install docs should clarify that the JS fallback package is only a compatibility shim and does not provide parser recovery.
- [x] Publishing/install docs should clarify that reduced mode has graph-only and regex recovery, not parser recovery.
- [x] Consider stable subpath exports before narrowing the broad root API.

## Suggested Execution Order
Expand Down
Loading