How to serve a Markdown version of your pages
Serving a Markdown copy when a client sends Accept: text/markdown gives AI agents a clean, token-cheap version of each page. Add content negotiation at the server or edge, return the text/markdown type, and keep the Markdown in sync with the HTML.
By ratemysite.ai editorial team. Updated .
How do you fix it?
- Generate a Markdown version of each page from the same source as the HTML, with headings, lists, tables and links.
- Add a rule at the server or CDN edge: when the Accept header prefers text/markdown, return the Markdown with Content-Type: text/markdown.
- Add Vary: Accept to both responses so caches keep the two versions apart.
- Optionally expose the same content at a .md URL such as /page.md.
- Test with curl -H "Accept: text/markdown" and re-run the audit.
What is content negotiation?
Content negotiation is the part of HTTP that lets one URL return different formats depending on what the client asks for in its Accept header. Browsers ask for HTML, while an agent can ask for text/markdown. If the server supports it, the agent gets the page without navigation, scripts or styling. A typical article drops from tens of thousands of tokens as HTML to a few thousand as Markdown, which makes it cheaper and more reliable to read.
How do I keep both versions consistent?
Generate both from one source of truth, such as your CMS fields or the original Markdown files, rather than converting HTML afterward. Include the title as an H1, keep tables and lists, and carry over the canonical URL. Set Vary: Accept so a CDN does not serve Markdown to a browser. Check a few pages each month by comparing the headings in both versions, because drift between them can mislead agents. Also keep the Markdown free of hidden text that the HTML does not show, since agents may treat both versions as the same page.
How much does this check matter?
- This check is worth about 1.2% of the overall score.
- A finished report is reused for 24 hours before you can re-analyze it.
- Each page is fetched with a timeout of 10 seconds, and HTML larger than 2 MB is not audited.
What else do people ask?
- Does Markdown replace my HTML page?
- No. Browsers still get HTML. Markdown is an extra representation for agents that ask for it.
- Why is Markdown cheaper for AI?
- It drops layout markup, scripts and styles, so the same page often takes a fraction of the tokens of its HTML.