The files that decide whether agents can use your site.
A handful of new specs now sit between a business and the AI systems summarising it. Most are twenty minutes of work and poorly documented. These are plain explanations of each, written from shipping all of them on this site, including the parts we got wrong first.
- llms.txt
What is llms.txt, and does it actually do anything?
A markdown file at the root of your site that tells language models what your business is and where the canonical pages are. Cheap to add, easy to get wrong, and not the ranking factor people assume.
- agent-card.json
What is agent-card.json, and do you need one?
The A2A agent card: a business card for agent-to-agent contact, published at a well-known path. Who you are, what you can do, how to be reached.
- WebMCP
What is WebMCP? Tools on your web page, no server required
A W3C draft that lets a page expose callable tools to an agent browsing it. Chrome and the ChatGPT desktop browser already discover them, and almost nobody has implemented it.
- ARD catalog
What is an ARD catalog (ard.json)?
Agentic Resource Discovery: one file listing every machine-readable thing your site exposes, so an agent does not have to hunt for it page by page.
- Markdown twins
How to serve markdown to AI agents (and why .md matters)
Append .md to any URL and get a clean markdown version. The simplest thing you can do to make a site readable by machines, and the easiest to get subtly wrong.
- robots.txt for AI
How to write robots.txt for AI crawlers
A blanket User-agent: * tells an agent nothing about your intent. Naming the AI crawlers says the access is deliberate, which is the thing being measured.
- pricing.md
What is pricing.md, and should you publish your prices?
If an agent cannot find what you charge, it cannot put you in a comparison. A commercial decision before a technical one.
- Web Bot Auth
What is Web Bot Auth? Letting good agents identify themselves
A public key directory that lets agents cryptographically sign their requests, so you can tell a real assistant from something wearing its user agent string.
- MCP server card
What is server-card.json, and how do agents find your MCP server?
Publishing an MCP server is not enough. An agent has to be able to find it, and know what it does, before connecting.
- auth.md
What is auth.md, and what if nothing needs authentication?
A markdown walkthrough at /auth.md telling an agent how to get access to your surfaces. It scores even when the honest answer is that everything is open.