What AI agents get wrong when writing docs

Frances Liu

Writers tell us they have a hard time assessing the quality of doc changes proposed by AI agents. We built a benchmark with technical writers. Docs maintainers at Helm, PostHog, and Mautic chose the tasks from their own repos and wrote criteria we graded the agents against. Writers from across this community read the paper before we published it. The bar in these results is one you could apply in your own review queue. We ran frontier coding agents (Claude Code, Codex, and OpenCode with open mod