Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions core/typing_logic.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
from pygments.lexers import get_lexer_for_filename, TextLexer
from pygments.token import Token
from pygments.util import ClassNotFound
from typing import List, Dict, Any

class TypingMapBuilder:
# 关闭 Pygments 的换行归一化,保证映射长度与编辑器原文严格一致。
_LEXER_OPTIONS = {"stripnl": False, "ensurenl": False}

@staticmethod
def _get_token_category(token_type) -> str:
Expand All @@ -20,9 +23,12 @@ def _get_token_category(token_type) -> str:
@staticmethod
def build_map(code: str, filename: str) -> List[Dict[str, Any]]:
try:
lexer = get_lexer_for_filename(filename)
except Exception:
lexer = TextLexer()
lexer = get_lexer_for_filename(
filename,
**TypingMapBuilder._LEXER_OPTIONS,
)
except ClassNotFound:
lexer = TextLexer(**TypingMapBuilder._LEXER_OPTIONS)

tokens = lexer.get_tokens(code)
typing_map = []
Expand All @@ -40,4 +46,4 @@ def build_map(code: str, filename: str) -> List[Dict[str, Any]]:
"category": category # 【新增】将语法基因刻入每一个字符
})

return typing_map
return typing_map
107 changes: 107 additions & 0 deletions docs/superpowers/plans/2026-07-16-preserve-typing-map-newlines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Preserve Typing Map Newlines Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** 确保键入映射逐字符保留原始代码,包括缺失的末尾换行和多个尾随空行。

**Architecture:** 保持现有 `TypingMapBuilder` 接口不变,只调整 Pygments 词法分析器的文本归一化选项。使用 Python 标准库 `unittest` 从公开接口验证映射字符与输入原文完全一致,无需新增运行时或测试依赖。

**Tech Stack:** Python 3.8+、Pygments、unittest

## Global Constraints

- 所有文件继续使用 UTF-8 编码。
- 代码注释使用中文。
- 改动仅限键入映射的换行保真和对应测试,不重构 GUI 或 AI 模块。

---

### Task 1: 保留原始代码的换行结构

**Files:**
- Modify: `core/typing_logic.py`
- Create: `tests/test_typing_logic.py`

**Interfaces:**
- Consumes: `TypingMapBuilder.build_map(code: str, filename: str) -> List[Dict[str, Any]]`
- Produces: 保持相同签名,并保证返回节点中的 `expected_char` 拼接结果严格等于 `code`

- [x] **Step 1: 写入失败测试**

```python
import unittest

from core.typing_logic import TypingMapBuilder


class TypingMapBuilderTests(unittest.TestCase):
@staticmethod
def _mapped_text(code: str, filename: str) -> str:
return "".join(
node["expected_char"]
for node in TypingMapBuilder.build_map(code, filename)
)

def test_does_not_add_trailing_newline(self):
code = "value = 42"
self.assertEqual(self._mapped_text(code, "example.py"), code)

def test_preserves_multiple_trailing_newlines(self):
code = "value = 42\n\n"
self.assertEqual(self._mapped_text(code, "example.py"), code)

def test_plain_text_fallback_preserves_input(self):
code = "第一行\n\n"
self.assertEqual(self._mapped_text(code, "example.unknown"), code)


if __name__ == "__main__":
unittest.main()
```

- [x] **Step 2: 运行测试并确认当前行为失败**

Run: `python -m unittest discover -s tests -v`

Expected: 3 个测试因 Pygments 添加或折叠末尾换行而失败。

- [x] **Step 3: 禁用词法分析器的换行归一化**

```python
from pygments.lexers import get_lexer_for_filename, TextLexer
from pygments.token import Token
from pygments.util import ClassNotFound


class TypingMapBuilder:
_LEXER_OPTIONS = {"stripnl": False, "ensurenl": False}

@staticmethod
def build_map(code: str, filename: str) -> List[Dict[str, Any]]:
try:
lexer = get_lexer_for_filename(
filename,
**TypingMapBuilder._LEXER_OPTIONS,
)
except ClassNotFound:
lexer = TextLexer(**TypingMapBuilder._LEXER_OPTIONS)
```

- [x] **Step 4: 运行单元测试并确认通过**

Run: `python -m unittest discover -s tests -v`

Expected: `Ran 3 tests` 和 `OK`。

- [x] **Step 5: 运行语法编译检查**

Run: `python -m compileall -q core gui main.py tests`

Expected: 命令退出码为 0,且没有错误输出。

- [x] **Step 6: 提交改动**

```bash
git add core/typing_logic.py tests/test_typing_logic.py docs/superpowers/plans/2026-07-16-preserve-typing-map-newlines.md
git commit -m "fix typing map newline preservation"
```
28 changes: 28 additions & 0 deletions tests/test_typing_logic.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import unittest

from core.typing_logic import TypingMapBuilder


class TypingMapBuilderTests(unittest.TestCase):
@staticmethod
def _mapped_text(code: str, filename: str) -> str:
return "".join(
node["expected_char"]
for node in TypingMapBuilder.build_map(code, filename)
)

def test_does_not_add_trailing_newline(self):
code = "value = 42"
self.assertEqual(self._mapped_text(code, "example.py"), code)

def test_preserves_multiple_trailing_newlines(self):
code = "value = 42\n\n"
self.assertEqual(self._mapped_text(code, "example.py"), code)

def test_plain_text_fallback_preserves_input(self):
code = "第一行\n\n"
self.assertEqual(self._mapped_text(code, "example.unknown"), code)


if __name__ == "__main__":
unittest.main()