Skip to content

feat: add llms markdown in doc pages#891

Open
shreya1510ss wants to merge 4 commits into
keploy:mainfrom
shreya1510ss:add-llms-markdown
Open

feat: add llms markdown in doc pages#891
shreya1510ss wants to merge 4 commits into
keploy:mainfrom
shreya1510ss:add-llms-markdown

Conversation

@shreya1510ss

@shreya1510ss shreya1510ss commented Jul 17, 2026

Copy link
Copy Markdown

What has changed?

Every docs page is now also available as plain Markdown, and llms.txt / llms-full.txt are auto-generated at build time — so AI agents (and humans) can grab clean per-page content instead of parsing HTML.

  • Added docusaurus-plugin-llms, configured to generate llms.txt, llms-full.txt, and a .md twin for every doc page (same URL, .md extension).
  • Added a "Copy as Markdown" / "View as Markdown" control under each page title, so the .md file is actually discoverable, not just guessable. Styled with Tailwind utility classes to match this repo's existing convention.
  • Patched docusaurus-plugin-llms (via patch-package, auto-reapplied on install) to fix two real content bugs found during review:
    • Some pages reuse content from another file via MDX imports (e.g. import WhatIsKeploy from '../concepts/what-is-keploy.md'); the plugin only resolved these when the filename had an underscore, so ~7 pages were leaking that raw import/JSX line into their .md output instead of the real text.
    • ~106 pages were also leaking raw React component code (<Tabs>, <ProductTier>, <HowTo>, etc.) into their .md output. Fixed by stripping self-closing component tags entirely and unwrapping paired ones while preserving their real inner content — carefully excluding <YOUR_INGRESS_HOST>-style fill-in-your-own-value placeholders, which look similar but aren't components.
  • Fixed a timer leak in the copy button: a setTimeout resetting its label was never cleared, so it could fire after the component unmounted (React warning) or race itself on repeated clicks.

Fixes https://github.com/keploy/enterprise/issues/2272

How Has This Been Tested?

Ran npm run build then npm run serve — confirmed the build succeeds, llms.txt/llms-full.txt generate correctly, each page's .md twin loads at its matching URL, and the Copy/View buttons work in both light and dark mode.

Preview.Video.mov

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you and congratulations 🎉 for opening your very first pull request in keploy

…Markdown button

Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
@shreya1510ss shreya1510ss changed the title Add llms markdown feat: add llms markdown in doc pages Jul 17, 2026
@manasmanohar
manasmanohar requested a review from Copilot July 17, 2026 06:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class “LLM-friendly” outputs for the docs site by generating per-page .md twins plus llms.txt / llms-full.txt, and exposes the per-page Markdown via UI actions under each doc title.

Changes:

  • Add and configure docusaurus-plugin-llms to generate llms.txt, llms-full.txt, and per-page Markdown outputs at build time.
  • Add a MarkdownPageActions UI control to “Copy as Markdown” / “View as Markdown” for eligible doc pages.
  • Wire the new control into the DocItem theme and update lockfile for new dependencies.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
yarn.lock Locks new transitive dependencies required by docusaurus-plugin-llms.
package.json Adds docusaurus-plugin-llms dependency.
docusaurus.config.js Configures the plugin to generate llms*.txt and Markdown per doc page.
src/theme/DocItem/index.js Computes .md twin URL and renders Markdown actions under the doc title.
src/components/MarkdownPageActions.js Implements “Copy as Markdown” / “View as Markdown” UI and clipboard logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/components/MarkdownPageActions.js
…leak

Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.

…tripping

Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>

@manasmanohar manasmanohar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the comments below. Specifics left inline; this one isn't tied to a diff line:

Collides with the llms.txt we already ship.
We already have curated static/llms.txt and static/llms-full.txt (content team's — About Keploy, awards, positioning, curated links). The plugin generates the same two filenames, and its postBuild runs after static/ is copied, so it overwrites them: build/llms.txt is the generated version and the curated content is gone. Net effect — the content team's file is silently replaced, and editing static/llms.txt later would have no effect. The static file also drifts today since it's hand-maintained.
Option: merge both — feed the curated text into the plugin via rootContent/fullRootContent (+ title/description) so it stays on top and the auto-generated index below stays fresh. Needs the curated source moved out of static/ (so it isn't overwritten) and the static copies removed. Worth confirming the exact split with the content team.

// and only for real doc pages, not root/category-index pages.
const showMarkdownActions = isLatestVersion && !suppressArticleSchema;
const markdownUrl = showMarkdownActions
? `${permalink.replace(/\/+$/, "")}.md`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy/View as Markdown 404s on the 5 section landing pages (concepts, dependencies, operation, server, server/sdk-installation). For an index page the permalink is /docs/concepts/, so this produces /docs/concepts.md — but the plugin writes the file to /docs/concepts/index.md, so the link 404s and Copy shows "Copy failed". Normal pages are fine.
Options: special-case folder permalinks (trailing slash → append index.md instead of .md), or derive the .md URL from the plugin's output instead of reconstructing it from the permalink.

Comment on lines +43 to +44
+ // Prop values can legitimately contain '<' or '>' inside quoted strings (e.g.
+ // tools={["Linux kernel >= 5.10"]}), which would otherwise be mistaken for the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two regexes run over the whole document including fenced code, so real code gets stripped. e.g. installation/windows.md: wsl --install -d <Distribution Name>wsl --install -d (~5 files); quickstart/java-spring-boot-xml.md: the <User>...</User> wrapper is removed and the XML sample ends up malformed. This is new — those tags were preserved before (the plugin only stripped a fixed HTML allowlist).
Option: skip fenced code blocks when stripping (mask fences → strip prose → restore).

Comment thread package.json
"@tailwindcss/typography": "^0.5.0",
"autoprefixer": "^10.4.0",
"docusaurus": "^1.14.7",
"docusaurus-plugin-llms": "^0.4.0",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plugin is ^0.4.0 but the patch is pinned to 0.4.0 — patch-package will fail the install if a 0.4.1 publishes. Consider pinning the exact version.

Comment thread docusaurus.config.js
[
"docusaurus-plugin-llms",
{
docsDir: "versioned_docs/version-4.0.0",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docsDir is hardcoded to versioned_docs/version-4.0.0 — will need updating when we cut the next docs version, since the buttons show on whatever the latest version is (gated on isLatestVersion).

? "Copied"
: copyState === "error"
? "Copy failed"
: "Copy as Markdown";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add fixed width to prevent the div from shrinking when changing text

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants