What This File Does, In Plain Terms
llms.txt is a plain text file at your site root that hands AI systems a curated map of your site: what it is, what matters on it, and where the important pages live. Markdown format, one H1 with your site name, a one-line summary, then sections of links. AI assistants that fetch pages on demand can use it to find your best content instead of guessing from navigation menus, and since May 2026 Google's Lighthouse includes an llms.txt check in its agentic browsing audit, which moved the file from nice-to-have to expected.
Three facts worth knowing before you upload one. First, llms.txt is advisory: it cannot force any AI system to read or obey it, and it is not a replacement for robots.txt, which controls access rather than describing content. Second, adoption is real but uneven, so treat the file as a low-cost bet with asymmetric upside rather than a ranking lever. Third, a bloated llms.txt is worse than none, because the point is curation. A dump of every URL you own tells a machine nothing about what matters. Our own file on this site lists a fraction of our pages, on purpose.
How This Generator Builds Your File
No templates and no guessing. The tool fetches your homepage through our server-side proxy, reads your real title tag and meta description for the H1 and summary line, then pulls your sitemap.xml and groups your URLs by their first path segment, so /blog/ pages become a Blog section, /services/ pages become a Services section, and so on. Link names are derived from your slugs. If you have no sitemap, it falls back to the internal links on your homepage. Everything renders into an editable preview, nothing is stored, and your file never touches a database.
Edit Before You Upload: The Three Cuts
One, delete utility pages: privacy, terms, login, cart, tag archives. Two, cut any section down to its ten best links, ordered by importance, because position signals priority. Three, rewrite the link names the generator derived from slugs into real descriptions where it matters, since "Complete Guide to AI Crawler Access" tells a model far more than "Ai Crawler List". Fifteen minutes of editing is the difference between a site map and a recommendation.
Uploading and Verifying
Upload the file to your web root so it resolves at yoursite.com/llms.txt, exactly like robots.txt. Serve it as plain text. Then verify three things: the URL returns 200 and not a redirect to your 404 page, the first line is a single # heading, and the links inside actually resolve. If your site runs on a framework with a public or static folder, the file goes there. Teams that want to go further add llms-full.txt, a longer companion file with expanded descriptions per link, which is what we run on this site alongside the short version.
One thing this file will not do is make AI systems cite you by itself. Citation comes from content worth citing plus crawler access plus entity clarity, and llms.txt is the map, not the territory. Our complete llms.txt guide covers what the standard genuinely does and does not do, and the AI crawler checker tells you whether the bots that would read your new file can even reach your site. If you want the whole picture done for you, that is our AI visibility audit.
Frequently Asked Questions
Is llms.txt required for AI search visibility?
No. It is advisory, and no AI platform requires it. But Lighthouse now checks for it in agentic browsing audits, adoption among technical sites keeps growing, and it costs minutes to add. The realistic framing: it will not rescue weak content, and it gives strong content a cleaner path to being found by fetch-on-demand AI agents.
Does this tool store my URL or my generated file?
No. The fetch happens through our proxy purely to read your public homepage and sitemap, the file is assembled in your browser, and nothing is written to any database. Close the tab and it is gone.
What is the difference between llms.txt and robots.txt?
robots.txt controls access: which crawlers may fetch which paths. llms.txt describes content: what your site is about and which pages represent it best. They are complementary. A bot blocked in robots.txt will not be reading your llms.txt, which is why checking crawler access first matters.
How big should my llms.txt be?
Small. The strongest files we see are under a hundred lines: one heading, one summary, and a few curated sections of roughly ten links each. If a page would not make your own top-twenty list, it does not belong in the file. Use llms-full.txt if you genuinely need the longer version.
Read next
LLMs.txt: The Complete Guide to What It Does (and Doesn't Do)
Read guide
Next tool
AI Crawler Checker: can AI bots reach your site at all?
Check your site