Refactor ai s3 thoughts - #1
Open
XD06 wants to merge 62 commits into
Open
Conversation
Reorder exact top-level Markdown blocks for image and file moves, skip the delayed Vditor serialization callback, and restore drag decoration after asynchronous rerenders. This fixes upward moves reverting, repeated drag failures, and source mutation around rendered time markers.
…lification and editor round-trips - sync: three-way merge before conflict toast, stable per-browser device id, drop dead broadcasts, fix version drift and missing base snapshot - thoughts: saveAndExit commits a local copy while offline (#13b); routes use per-thought storage primitives instead of full-collection rewrites (P0 write amplification); incremental search index updates - editor: stop escape accumulation (#9), persist upload placeholders (#7), serialize WYSIWYG via VditorDOM2Md so soft line breaks survive source-mode round-trips (#36) - code blocks: gutter pads blank rows for soft-wrapped lines and realigns on resize/font load; CSS clips numbers to the block (#37)
- PIN and legacy-PIN fields use type=password with an eye toggle on every secret input - layered stage shadows, radial brand glow background, SVG grain, entrance and step-swap animations honoring prefers-reduced-motion - input hover/focus depth, gradient primary button
Opt-in editor performance diagnostics behind a query flag plus heading index module with tests; test_editor_performance.js still failing (switch_stable_after_retry_ms not yet reported).
…ap WS payload Remove world-known SHARE_SECRET fallback (random per-boot when unset; completes audit P0-3), fixpoint share-page sanitizer vs obfuscated XSS, 50 MiB WS maxPayload, fix pre-existing editor-perf test. New tests wired into test:safety. Full suite 74/74; npm run check 193 files.
- settings panel lists assets with select-all/per-item checkboxes and batch delete - style multi-select checkboxes as themed gold box with white SVG check/indeterminate - asset storage/routes support listing and batch delete - add hybrid editor caret stability regression test
Show a plain boot textarea while the ~554KB rich editor loads, then hand off seamlessly while preserving keystrokes and caret. Preload vditor and modulepreload hybrid-editor; load thoughts.css non-blocking. Keep the 1.9MB reading webfont out of first paint via a system-only stack, bust the font cache with a v=2 token after re-compression, and drop the unused Source Han Serif subset.
Move the right control rail from 304px to 290px so it sits in the gutter beside the 270px right sidebar with a small gap, instead of overlapping the editor text column.
api.md and openapi.json now cover GET /api/assets, POST /api/assets/bulk-delete and DELETE /api/assets/:id, plus notepad PUT/DELETE and new asset schemas. SKILL.md documents Auth V2 scoped API tokens alongside the legacy PIN. Bump package.json and the openapi version to 1.0.9.
…reflow Source mode: entering source mode re-serialized the WYSIWYG DOM via Lute (readWysiwygMarkdownValue -> VditorDOM2Md), which reformats untouched content: padding GFM table cells, expanding the | --- | separators, and dropping the blank line after a YAML front-matter fence. The damage compounded on every toggle. Show the canonical _lastValue instead, which already tracks every real edit. Adds test_source_mode_roundtrip.js. Boot handoff: keep the opaque boot cover until the rich editor async after hook has rendered its content (new whenReady/readyPromise), then reveal in one layout pass so the empty-to-content reflow and white-card region swelling never paint. Preload the reading webfont and style the boot textarea as the same centred white card as the vditor surface.
The WYSIWYG editor persisted the value it re-derived from the DOM, not the user's original Markdown. Lute's DOM->Md pass is lossy and non-idempotent: it re-escapes emphasis markers (foo_bar -> foo\_bar -> foo\\_bar ...), pads GFM table cells, rewrites '| --- |' separators and drops the blank line after a YAML front-matter fence. A spurious input (decoration pass, caret move, mark re-render) let that reformatting reach storage, and the drift compounded on every refresh -- the '越刷新越多' backslash growth and layout churn. Add isMeaningfulMarkdownChange: a rendered-HTML equality oracle. Two Markdown strings that render to identical HTML differ only in serialization noise, so the re-serialization is dropped and _lastValue stays pristine. Gate both persistence entry points (emitChange adopt, getValue read). Unlike the per-symptom strip* helpers this is config-independent and future-proof -- a brand-new Lute reformatting is neutralised automatically, while genuine edits (including front-matter content) always render differently and are saved. Verified against the live editor's real Lute config: every real-doc round-trip reformats bytes yet renders identically, so the guard reports it non-meaningful; no reformatting is ever wrongly flagged. Add test_editor_noop_save_guard.js.
Move 78 root test_*.js into test/ via git mv (history preserved) and fix file-relative paths: require(./)->require(../), root __dirname assignments and spawn cwd -> path.resolve(__dirname, ..), and path.join(__dirname, X) -> path.join(__dirname, .., X). Self-referential __filename/dataDir usages left intact. Add committed full-suite runner scripts/run-all-tests.js wired to npm test / npm run test:all, and repoint all package.json test:* scripts at test/. Move audit/FIX-SUMMARY governance docs into docs/archive/ and update README.md, AGENT_CONTEXT.md, and docs/README.md to document the test/ location and how to run tests. Verified: npm run check (200 files + server smoke) and npm test (77/77) pass.
Conflicted outbox patches had no exit: retry() skips conflict items, markConflict never cleared, and baseVersion never advanced, so the status button was stuck on 同步冲突 forever with no way to resync. Add ThoughtOutbox.rebaseConflict (keep local: rebase onto remote version + clear conflict so the next retry resends and wins) and discardConflict (use remote: drop the local patch). Repoint the conflict status click to resolveOutboxConflicts, which offers a keep-local / discard-local choice per conflicted Thought. Extend test_thought_outbox.js to cover both recovery paths.
- /api guard re-issues the auth cookie on cookie-authenticated activity so an in-use session never expires mid-session; bearer/API clients unchanged - default COOKIE_MAX_AGE raised 24h -> 720h (30 days) as the inactivity window - document COOKIE_MAX_AGE in .env.example - add legacy sliding-renewal regression test
selectNotepad only removed the sidebar 'visible' classes and left body.mobile-sidebar-open set, so the CSS that hides the floating actions while the sidebar is open kept the FAB hidden for the rest of the session. Centralize teardown in closeMobileSidebar() (also restores the thoughts sidebar host) and call it from selectNotepad.
Uploading the same image/file again (from the UI or API) stored a fresh copy every time because ids were random UUIDs. Now each asset records a sha256 of its bytes; image/file uploads look up an existing asset with the same hash first and reuse it (HTTP 200) instead of creating a duplicate, also skipping the image re-encode. Pre-hash assets simply don't match yet.
…load Issue #3 (read side): assets already stored on the server show up in the settings asset manager, but the only way to get one into an article was to re-upload it, creating a duplicate copy. Add an inline caret-insert primitive (insertArticleMarkdownAtCaret) that both the upload flow and a new insertArticleAssetReference() go through, and wire an '插入' action in the asset manager that drops the right image/file markdown at the caret and closes settings. Guards the editor being ready before inserting.
Issue #5: while the user was typing, a background sync or 409 auto-merge would replace the whole document with a plain setValue(), collapsing the WYSIWYG caret to the top of the note - the caret appeared to vanish. Add setValuePreservingCaret()/editorHasFocus() to the editor and route every remote/merge apply through a new editor.applyRemoteValue() that keeps the caret where the user is typing, but only when the editor is focused so notepad switches and first loads still reset it as before.
Give agents a heading-level view of an article so they can rewrite one section without re-parsing the whole document: - GET /api/notes/:id/outline returns the ATX heading tree (slug, text, level, 0-based line), computed with the same parser as the editor's table of contents (a test pins the two together) - PATCH actions replace_section / append_to_section target a section by slug (or exact heading text), editing only the body up to the next heading; ambiguous heading text is rejected in favor of a slug Outline + section logic stays in scripts/note-edits.js as pure functions.
Stage 3 of the fine-grained note editing work: - POST /api/notes/:id/edits applies an ordered edit list under one write lock and a single version bump; each edit sees the previous result and any failure rejects the whole batch with the failing index, so a note is never left half-written. - Refresh SKILL.md, api.md, and openapi.json to document the guarded actions (expectedCount), section-aware editing, GET /outline, and the new batch endpoint.
Issue #6 follow-up: the earlier fix added padding-bottom:160px INSIDE the white article card (pre.vditor-reset), so short notes showed a tall empty area inside the card. Move the breathing room onto the scroll container (.vditor-wysiwyg) instead: the card now hugs its content (24px balanced bottom padding) and the gap sits BELOW the card, between it and the bottom of the editor viewport. Short notes leave a clean gap; long notes still keep their last line off the bottom edge. Regression test updated to encode the new invariant (breathing room on the container, small padding on the card).
…om gap - Source mode used top:0 + 102px internal top padding, so text floated far below the card top and looked inconsistent with WYSIWYG. Mirror the WYSIWYG card: top:64px + 24px top padding so both modes start in the same place. - Halve the desktop scroll-container bottom breathing room (160px -> 80px) per feedback that the gap was too large. - Regression test updated: pins source/WYSIWYG top alignment and the relaxed (>=64px) container bottom padding.
The boot textarea reused desktop card measurements (top:64px / padding 24px 35px 64px / line-height 1.72) on every viewport, but the mobile vditor card is a different geometry: no top offset (margin 0), an inline 10px 10px padding from vditor's setPadding narrow-viewport branch, the bottom gap moved OUTSIDE the card onto .vditor-wysiwyg (env safe-area + 64px), line-height 1.68, and vditor's scoped --border-color (#d1d5da light / #141414 dark). The boot card therefore sat 64px below the vditor card on phones with the text starting 78px lower -- the mobile vertical "region swelling" at refresh handoff. Add a @media (max-width:980px) .boot-editor override mirroring the measured mobile geometry (top:0 / bottom:calc(safe-area+64px) / padding:10px 10px 64px / line-height:1.68 / theme-scoped border color). Verified in a real browser: boot and vditor cards now match pixel for pixel at 375px (light + dark), desktop 1440px geometry unchanged.
When the caret sat in the viewport's bottom ~20% and the user committed Chinese pinyin (compositionend) inside a list block, Vditor re-rendered the edited list by first removing the adjacent previous list then re-inserting the merged version. Removing a large list above the viewport collapsed the content above the caret, the browser dropped scrollTop to compensate, and the re-insert never restored it — so each keystroke/IME commit lurched the view up and back down while the caret itself stayed put. Add bindScrollStabilization: event-based guards (compositionstart/ beforeinput/input/keydown/compositionend) re-anchor the caret at its pre-event on-screen offset in a microtask, compensating only the upward collapse so the browser's own caret-revealing scroll on the downward side is never fought. A MutationObserver on the content root's childList acts as defence in depth — it catches Vditor's structural surgery regardless of which event sequence a real IME driver dispatches, which synthetic events can't reproduce.
Desktop lays .vditor-wysiwyg out as a fixed-height internally-scrolling container, but mobile (ios-theme.css < 981px) lets the content expand the page and scrolls at the document root instead — .vditor-wysiwyg ends up with clientHeight === scrollHeight and is not scrollable. The stabilizer wrote scrollTop to .vditor-wysiwyg unconditionally, so on mobile the collapse compensation was silently dropped on a non-scrolling element and the page stayed stuck at the collapsed position (caret thrown out of view). Add getScrollContainer(): prefer .vditor-wysiwyg only when it actually carries scroll (scrollHeight > clientHeight), otherwise fall back to document.scrollingElement. readScrollStabilizeState now uses it, so both the event guards and the MutationObserver correct the right element on each layout.
The previous approach captured the caret's on-screen offset and corrected scrollTop after the collapse, which still produced a visible jump-then- snap. The user needs the viewport to stay completely still for the duration of an edit: where it was when typing started, it stays. Switch to freeze mode: on compositionstart/beforeinput/Enter/Backspace/ Delete, capture scrollTop. A capture-phase scroll listener clamps it back on the same frame any drift occurs — from Vditor removing a list above the caret or the browser re-anchoring — so the collapse never reaches paint. The freeze holds through compositionend and one extra rAF to cover Vditor's async re-render, then releases. The MutationObserver now starts a freeze from the current position if one isn't active, so paths not wrapped in an event listener are still covered. getScrollContainer picks .vditor-wysiwyg on desktop and document.scrollingElement on mobile, so the freeze targets the element that actually carries the scroll on each layout.
The pure freeze locked the viewport wherever it was when typing started, even if the caret sat outside the visible area — leaving the user typing blind. The viewport should move to reveal the caret first, then freeze. freeze() now calls bringCaretIntoView() unless a freeze is already active: it does a minimal scroll (never recenter) to bring the caret into the visible band, then freezes at that position. isProgrammaticScroll guards the reveal so the scroll listener doesn't clamp its own move. Also fix the visible-band calc for mobile: document.scrollingElement's rect spans the whole document, not the viewport, so use window dimensions (0..innerHeight) there instead of the element rect.
- Auth cookies switch SameSite=Strict -> Lax: installed-PWA cold starts can arrive without a same-site initiator on some mobile browsers, which made Strict drop the cookie and re-prompt for the PIN/password on every full exit. Lax still withholds the cookie from cross-site POSTs. - V2 browser sessions now slide-renew like the legacy PIN cookie: the session cookie is re-issued with its remaining (absolute) lifetime on authenticated API activity, at most once per hour. - The instant boot textarea applies the saved per-notepad scroll offset on startup, and restorePersistentCaret defers until Vditor is ready instead of silently dropping the position when called during boot.
- Outbox flushes are chained: a flush requested while a sync request is in flight now re-runs when it finishes, instead of being dropped until the next unrelated trigger. - Network failures schedule a 3s backoff retry; previously nothing retried until a reconnect/tab switch/next keystroke. - Leaving the Today view flushes pending outbox items instead of cancelling the timer and stranding the edit. - ws_connected refetches the day list to catch updates pushed by other devices while this client was disconnected. - The manager is created during idle startup in editor/thoughts boots too, so pushes are received in the background. - docs/sync-boundaries.md records the new outbox/reconnect semantics.
Navigation timeout 900ms -> 600ms and static timeout 650ms -> 450ms. The fallback is the same versioned cache the network response would have refreshed, so a timeout only trades one fresh copy for the previous identical one, while the old windows delayed first paint on every refresh over a slow link. Also cache .wasm assets like other static types. test_pwa_cache_regression.js now codifies the new windows.
Long articles janked on backspace/typing because every edit ran several
full-document passes:
- Change serialization (DOM clone + Lute full-doc conversion + two
Md2HTML renders) now debounces at 300ms instead of 120ms, aligned with
the app's 300ms save debounce.
- The marker MutationObserver skips its full-document TreeWalker when a
characterData edit carries no marker trigger characters ('='/'['/'<');
block rebuilds (childList) still trigger the walk as before.
- restoreListAnnotationsFromSource bails out early when the source has
no data-note= annotations instead of re-scanning line by line on every
decoration pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.