@benword Ben Word

Serving Markdown Instead of HTML to LLM User Agents

Bun announced they started serving their documentation as raw markdown to Claude Code instead of rendered HTML. The results were a 10x reduction in token usage.

It’s a great idea. I immediately went to implement it on our roots.io docs, which uses Laravel routes within our WordPress site thanks to Acorn.

I’ll show an implementation in Laravel, but the concept applies to any web framework.

Getting started

Create a Laravel app and install the CommonMark package for converting markdown to HTML:

composer require league/commonmark

Creating the files

You’ll need to create a few files to get this working:

  1. Create the controller at app/Http/Controllers/DocsController.php:
<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use League\CommonMark\CommonMarkConverter;

class DocsController extends Controller
{
    public function show(Request $request, $slug)
    {
        $slug = basename($slug);
        $filePath = resource_path("content/docs/{$slug}.md");

        if (!file_exists($filePath)) {
            abort(404);
        }

        $markdown = file_get_contents($filePath);

        if ($this->isLLMRequest($request)) {
            return response($markdown, 200)
                ->header('Content-Type', 'text/markdown; charset=UTF-8');
        }

        // Convert to HTML for regular users
        $converter = new CommonMarkConverter();
        $html = $converter->convert($markdown);

        return view('docs', ['content' => $html]);
    }
}
  1. Create a sample markdown file at resources/content/docs/example.md:
# Example Markdown File

Howdy!

## Markdown example

This content is written in **Markdown**.
  1. Create the view at resources/views/docs.blade.php:
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Documentation</title>
</head>
<body>
    <div class="docs-content">
        {!! $content !!}
    </div>
</body>
</html>
  1. Add the route in routes/web.php:
<?php

use Illuminate\Support\Facades\Route;
use App\Http\Controllers\DocsController;

Route::get('/', function () {
    return view('welcome');
});

Route::get('/docs/{slug}', [DocsController::class, 'show']);

Now if you visit /docs/example in your browser, you’ll see the rendered HTML. But if you make a request with a user agent containing “Example”, you’ll get the raw markdown instead.

Testing it out

To test your implementation, you can use cURL with the “Example” user agent:

curl -H "User-Agent: Example" http://localhost:8000/docs/example

You should see the raw markdown returned instead of HTML. Without the custom user agent header, you’ll get the full HTML page.

Expanding user agent detection

You can expand the user agent detection to include multiple LLMs. On the roots.io docs, I fired up Claude Code, Claude Web, and Gemini CLI to see their user agents in the access logs:

I just used axios, Claude-User, and node as the substrings to check for in the user agent header. These checks are broad and may catch non-LLM clients, but serving Markdown to them for docs should be harmless.

Check your own access logs to discover which LLM tools your users are using.

Here’s the updated user agent check:

protected function isLLMRequest(Request $request): bool
{
    $userAgent = $request->header('User-Agent');
    $llmAgents = ['axios', 'Claude-User', 'node'];

    foreach ($llmAgents as $agent) {
        if (str_contains($userAgent, $agent)) {
            return true;
        }
    }

    return false;
}

Why this matters

LLMs are typically charged based on token usage, and HTML is very verbose compared to markdown. By serving markdown directly, you can significantly reduce the number of tokens processed, leading to cost savings and improved performance.

Also note that this approach means LLMs miss your site’s navigation, styling context, and any interactive elements. Sometimes that context is valuable, but for many documentation use cases, the raw content is sufficient.