Skip to content

fix(kb): 当 markitdown 不可用时,回退到纯文本解析器处理 txt/md 文件 | fall back to plain-text parser for txt/md when markitdown is unavailable - #9676

Open
SweetenedSuzuka wants to merge 3 commits into
AstrBotDevs:masterfrom
SweetenedSuzuka:fix/kb-txt-md-parser-fallback

Conversation

@SweetenedSuzuka

@SweetenedSuzuka SweetenedSuzuka commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

摘要 / Summary

当可选依赖 markitdown-no-magika 未安装时,向知识库上传 .txt/.md/.markdown 文件会报通用的"文档解析失败"错误,掩盖了真正的原因是缺少依赖。select_parser 将纯文本格式也路由到了 MarkitdownParser,该解析器在模块顶层 from markitdown_no_magika import ...,依赖缺失时直接抛出 ModuleNotFoundError,随后被 kb_helperexcept Exception 统一转成"无法读取或解析上传文件"。

Uploading .txt/.md/.markdown files to a knowledge base failed with a generic Document parsing failed error when the optional markitdown-no-magika dependency was missing, hiding the real cause. select_parser routed plain-text formats to MarkitdownParser, whose top-level from markitdown_no_magika import ... raised ModuleNotFoundError that kb_helper's broad except Exception converted into the generic "cannot read or parse the file" message.

改动 / Changes*

  • select_parser().txt/.md/.markdown 路由到仓库已有的标准库 TextParser(支持 utf-8/gbk 等多编码解码),纯文本上传不再依赖 markitdown。.rst/.adoc/.xlsx/.docx/.xls 仍使用 MarkitdownParser,行为不变。

  • kb_helper.upload_document() 的解析兜底新增对 ModuleNotFoundError 的处理:当缺失模块正是 markitdown_no_magika 时,给出明确的"缺少 markitdown-no-magika 依赖"提示;其余缺失模块错误原样透传,避免误标为 markitdown 问题。

  • select_parser() now routes .txt/.md/.markdown to the existing stdlib TextParser (multi-encoding decode: utf-8/gbk etc.), so plain-text uploads no longer depend on markitdown. .rst/.adoc/.xlsx/.docx/.xls still use MarkitdownParser, unchanged.

  • kb_helper.upload_document()'s parse fallback handles ModuleNotFoundError: when the missing module is exactly markitdown_no_magika, it surfaces a clear "missing markitdown-no-magika dependency" message; unrelated missing-module errors pass through instead of being mislabeled as a markitdown problem.

验证 / Verification

  • 新增 tests/unit/test_kb_select_parser.py,覆盖纯文本路由到 TextParser.rst/.adoc 仍走 MarkitdownParser、缺依赖的明确提示、无关缺失模块透传。

  • 知识库相关测试套件全部通过。

  • Added tests/unit/test_kb_select_parser.py covering plain-text routing to TextParser, .rst/.adoc still resolving to MarkitdownParser, the clear missing-dependency message, and pass-through of unrelated missing modules.

  • The knowledge-base related test suite passes.

Fixes #9598

Summary by Sourcery

Route plain-text knowledge base uploads away from the markitdown-based parser and provide clearer error reporting when markitdown-no-magika is missing.

Bug Fixes:

  • Ensure .txt/.md/.markdown uploads are parsed via the TextParser so they no longer fail when markitdown-no-magika is not installed.
  • Surface a specific "missing markitdown-no-magika dependency" error during document parsing instead of a generic parse failure, while allowing unrelated missing-module errors to propagate correctly.

Tests:

  • Add unit tests covering parser selection for plain-text vs markup formats and the new missing-dependency error handling in upload_document.

…AstrBotDevs#9598)

Fixes AstrBotDevs#9598.

Uploading .txt/.md/.markdown to a knowledge base failed with a generic
"文档解析失败" error when the optional markitdown-no-magika dependency
was missing, because select_parser routed plain text to MarkitdownParser
and its top-level import raised ModuleNotFoundError that kb_helper masked.

Modifications:
- select_parser() routes .txt/.md/.markdown to the existing stdlib
  TextParser; .rst/.adoc/.xlsx/.docx/.xls still use MarkitdownParser
- kb_helper upload_document reports a clear "缺少 markitdown-no-magika
  依赖" message instead of the generic parse failure when that module is
  absent, and lets unrelated missing-module errors pass through
- Add tests covering plain-text routing, markitdown-dependent formats,
  and the missing-dependency message
@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 14, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In select_parser, the sets of extensions routed to each parser are now hard-coded in multiple places; consider centralizing these mappings (e.g. a shared dict or constants) to keep behavior consistent and easier to update when adding new formats.
  • The ModuleNotFoundError handling in upload_document relies on exc.name == "markitdown_no_magika"; to be more robust across different raise sites, you might also guard on the module name appearing in str(exc) or centralize this dependency check into a helper function.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `select_parser`, the sets of extensions routed to each parser are now hard-coded in multiple places; consider centralizing these mappings (e.g. a shared dict or constants) to keep behavior consistent and easier to update when adding new formats.
- The `ModuleNotFoundError` handling in `upload_document` relies on `exc.name == "markitdown_no_magika"`; to be more robust across different raise sites, you might also guard on the module name appearing in `str(exc)` or centralize this dependency check into a helper function.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@SweetenedSuzuka SweetenedSuzuka changed the title fix(kb): fall back to plain-text parser for txt/md when markitdown is unavailable | 当 markitdown 不可用时,回退到纯文本解析器处理 txt/md 文件 fix(kb): 当 markitdown 不可用时,回退到纯文本解析器处理 txt/md 文件 | fall back to plain-text parser for txt/md when markitdown is unavailable Aug 14, 2026
@Soulter

Soulter commented Aug 15, 2026

Copy link
Copy Markdown
Member

markitdown-no-magika 不是可选依赖吧,是你的环境安装这个库的时候遇到了什么问题吗

@SweetenedSuzuka

Copy link
Copy Markdown
Contributor Author

markitdown-no-magika 不是可选依赖吧,是你的环境安装这个库的时候遇到了什么问题吗

额……对不起,我可能做了一件欠考虑的事情。
是因为我看到 #9598 提到了他在没有 markitdown-no-magika 的情况下上传文件遇到了问题,然后我就直接配置了一个没有 markitdown-no-magika 的环境测试了一下,发现确实有问题,就直接做了一个回退来修复。
我没有去核实为什么 #9598 会没有 markitdown-no-magika,是我的问题。

@SweetenedSuzuka

SweetenedSuzuka commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

markitdown-no-magika 不是可选依赖吧,是你的环境安装这个库的时候遇到了什么问题吗

但是我觉得,在用户使用的过程中,确实有可能因为某种原因未能安装依赖,或者该依赖被其它程序破坏。
既然 #9598 遇到了这个问题,我们无法核实它到底是如何失去这个依赖的,但我认为,如果我们能做一下回退,可以增加程序的稳定性,使得程序在这种情况下也能正常运行。而与此同时,我们在控制台中抛出这个错误,可以告诉用户目前程序并不是处在健壮的运行状态,让用户知道是缺少了什么依赖。
我觉得此功能还是可以考虑加入的。
就算回退机制没有必要,但对缺少依赖引起的错误进行抛出应该是有必要的?你怎么看?如果有必要的话,我可以直接在这个PR里改,或者另起一个PR对这项异常做一个抛出异常时提示缺少依赖的处理。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 没有markitdown-no-magika依赖时知识库中上传txt文件会报错

2 participants