Short answer
Llms.txt is an optional, Markdown-formatted text file that gives compatible AI tools a curated map of a website's important content. It's a community proposal for organizing information, not a requirement for AI visibility or a guarantee of recommendations.
On this page
- 1. What Is llms.txt and What Does It Actually Do?
- 2. How llms.txt Works From File Structure to Retrieval
- 3. When llms.txt Is Worth Maintaining
- 4. llms.txt Versus robots.txt and Why Access Comes First
- 5. Practical Changes That Make SaaS Content Easier to Use
- 6. How to Measure Whether AI Tools Use the Content
- 7. Common llms.txt Mistakes and How to Avoid Them
- The seven topics at a glance
- Your Next Steps, From File to Feedback Loop
- FAQ
Why publish another index file if an assistant still can't read the pricing page it points to? That's the gap SaaS teams should resolve before treating llms.txt as an optimization priority.
For founders and marketing teams, the practical question isn't just what llms.txt contains. It's whether AI systems can access accurate product information, find useful comparisons, and describe the software correctly in buying recommendations. Crawlability, buyer-facing content, and measurement deserve attention before the file does.
The priorities below move from understanding the format to fixing access, improving source pages, and checking actual answers. Each ends with an action a team can take today, without assuming that publishing a file changes how any assistant behaves.
1. What Is llms.txt and What Does It Actually Do?#
Llms.txt is a publisher-maintained reading list for AI tools. It introduces a website or project and points toward resources that explain it, such as documentation, product specifications, pricing, and policies.
Jeremy Howard proposed the format on September 3, 2024, according to the llms.txt proposal, which describes its purpose as helping language models use website information at inference time. It remains a proposal rather than a requirement that every AI provider follows.
For a project management SaaS, a useful file could link to its plan comparison, Salesforce integration guide, API documentation, and comparisons with Asana and Monday.com. Each description should explain what the linked page contains, rather than repeat a marketing slogan.
The trade-off is maintenance. A short index is inexpensive to create, but someone must keep its links and descriptions aligned with the live product.
Practical rule: Treat llms.txt as documentation infrastructure, not as a ranking switch.
Action today: A documentation owner should draft a compact list of authoritative pages in a UTF-8 text document. The selection should favor pages that answer real product questions, not every URL in the sitemap.
2. How llms.txt Works From File Structure to Retrieval#
A CRM vendor's index might introduce the product, explain which documentation covers the current release, and group links under pricing, integrations, and security. A compatible assistant could fetch that index and follow the resources relevant to a question.
The published format specification requires an opening H1 heading naming the project or website. It recommends a summary blockquote next, followed by optional explanatory content and H2 sections containing Markdown links with brief descriptions.
A valid file still needs useful destinations#
The distinction between required syntax and recommended content matters. An opening project heading can satisfy the minimum format without giving an assistant much useful context.
A practical implementation should include:
- Product context: A factual summary of what the software does and who uses it.
- Grouped resources: Clearly named sections for buying information, implementation, and policies.
- Descriptive links: Labels such as Salesforce synchronization requirements rather than Learn more.
- Consistent terminology: Product and plan names that match the public website.
An assistant's ability to read the format doesn't mean it will request the file during every conversation. Discovery, retrieval, and use in an answer remain separate steps.
Action today: The site owner should open the deployed file directly, follow every listed resource, and check that each description matches its destination.
3. When llms.txt Is Worth Maintaining#
Documentation-heavy SaaS products have the clearest practical use case. An API platform with authentication guides, SDK references, migration instructions, and integration documentation has a genuine navigation problem that a curated index can help address.
A small workflow tool with a concise marketing site may have little to simplify. If its pricing page is vague and its integrations are described only in screenshots, improving those pages deserves the available time.
Choose a workflow before choosing the file#
Consider a helpdesk vendor with an extensive support library. Its index could direct an assistant toward current plan limits, single sign-on requirements, and the integration guides needed for implementation.
That's a more concrete purpose than hoping the file makes the brand appear in general software recommendations. The team can test the workflow by giving an assistant the index and checking whether it reaches the correct documentation.
Adoption alone isn't proof of effectiveness. RankGarage's llms.txt guide distinguishes file presence from actual assistant use and reports no measurable relationship between increased presence and AI citation frequency.
A worthwhile index saves a reader or compatible tool from choosing between confusing, duplicate, or outdated documentation.
Action today: The product or documentation team should name the task the index will support. If no clear task exists, the file can wait while the team improves the pages buyers already need.
4. llms.txt Versus robots.txt and Why Access Comes First#
An index can't make an inaccessible page readable. A SaaS vendor can publish a carefully organized llms.txt file while its comparison guides remain blocked by crawler rules, authentication, or firewall challenges.
The files serve different purposes. Robots.txt expresses crawler access preferences, while llms.txt supplies context and selected resources. Neither should be treated as a substitute for authentication when information must remain private.
For example, a software team might discover that an inherited rule blocks its entire resources directory, including its strongest alternative and comparison pages. Another team might find that pricing details appear only after client-side JavaScript executes.
Check the complete access path#
- Crawler preferences: Review relevant user-agent groups and broad disallow rules.
- Delivery: Check for successful responses, redirect loops, login requirements, and bot challenges.
- Readable content: Inspect raw HTML for product names, plan details, and feature descriptions.
- Intentional restrictions: Preserve security controls and document which public crawlers should have access.
The AI crawler access checker offers a starting point for reviewing access. The distinction between crawler rules and broader visibility is also covered in this robots.txt and SEO guide.
Action today: Engineering should test the pricing, comparison, integration, and security pages first, then record any intentional restrictions so future firewall changes don't reverse them.
5. Practical Changes That Make SaaS Content Easier to Use#
A finance software vendor's pricing page might say that plans scale with growing teams without explaining included integrations or usage restrictions. An llms.txt link won't resolve that ambiguity.
The destination page must contain the answer. Clear source material helps human buyers evaluate the product and gives retrieval systems explicit information to work with, without requiring them to infer details from slogans.
Publish facts that support buying decisions#
Pricing pages should state plan names, billing conditions, included capabilities, and relevant limits in readable text. Integration pages should distinguish native integrations from API-based connections and explain material setup requirements.
Comparison pages need equal care:
- Shared criteria: Evaluate every product against the same requirements.
- Fair limitations: Identify situations where a competitor is a better fit.
- Specific evidence: Link claims to current documentation rather than unsupported superiority statements.
- Clear audience: Explain which team or workflow benefits from the differences.
Third-party listings deserve attention too. A project tracking vendor should correct outdated product descriptions and integration claims on relevant review platforms and directories, particularly when those pages appear in observed assistant citations.
The practical approach in this guide to visibility in ChatGPT belongs alongside access and content work, not in place of it.
Action today: Marketing should rewrite an ambiguous buying page and inspect the external listings cited in its category's AI answers. Changes should be logged before measuring subsequent answers.
6. How to Measure Whether AI Tools Use the Content#
A request in a server log proves retrieval, not influence. A citation proves that an answer references a page, but it doesn't establish that llms.txt caused the assistant to find it.
Teams should separate technical observations from buyer-facing outcomes. Otherwise, a crawler visit can be mistaken for a recommendation improvement.
Track repeatable buying questions#
A project management vendor could monitor prompts about agency workflows, client approvals, and alternatives to Asana. The prompt wording should remain stable so changes in the questions don't get confused with changes in visibility.
A spreadsheet can record:
- Run context: Date, assistant, location settings, and exact prompt.
- Brand outcome: Whether the product is mentioned, recommended, or absent.
- Answer accuracy: Correct and incorrect statements about features, pricing, and fit.
- Evidence: Cited domains, destination URLs, and the captured answer.
For implementation details, this AI citation tracking guide explains the monitoring workflow.
Say competing helpdesk products repeatedly receive citations from an industry directory. The next investigation should examine that directory's coverage, rather than assume a larger llms.txt file is the answer.
Measurement caution: Matched periods help organize a comparison, but they don't eliminate model changes, answer variability, or unrelated website updates.
Action today: The team should save a baseline prompt set, capture complete answers, and date each access or content change before reviewing subsequent results.
7. Common llms.txt Mistakes and How to Avoid Them#
The main risk is publishing a second, less accurate version of the website. An automatically generated index can look complete while pointing to retired documentation, restricted pages, or outdated product names.
Common Crawl's content analysis of llms.txt files distinguishes basic Markdown formatting from the fuller expected structure. For practitioners, the useful lesson is that file presence alone isn't a quality check.
Avoid completeness without prioritization#
A developer tools company might export its entire documentation tree, including deprecated SDK instructions. A shorter index pointing to current setup, authentication, and migration guidance would be easier to interpret and maintain.
The same caution applies to llms-full.txt, which bundles content rather than merely linking to it. A generated bundle needs a deliberate public-content filter so internal runbooks and draft documentation don't enter the output.
- Exclude restricted material: Build from an explicitly public collection.
- Validate destinations: Flag broken links, unexpected redirects, and authentication requirements.
- Remove stale descriptions: Update summaries when plans or capabilities change.
- Avoid promotional instructions: Describe source material instead of telling assistants to recommend the brand.
Keep the index factual enough that a documentation owner can verify every line against a public source page.
Action today: The maintainer should compare the file with live pricing and documentation, assign an owner, and connect validation to the publishing workflow. A generated file still needs editorial responsibility.
The seven topics at a glance#
| Topic | Effort to act on | Impact on AI visibility | Best first step |
|---|---|---|---|
| What llms.txt Is | Low; create a curated text file | Optional and uncertain; no guaranteed citation gains | List the most important public product and documentation pages |
| How llms.txt Works | Low; use a simple Markdown structure and maintain links | May help supporting tools find relevant pages; adoption varies | Add an H1, a short summary, and clearly described links |
| When to Use llms.txt | Low for documentation-heavy sites; optional for small sites | Most useful as a supporting asset for content-rich products | Check whether a curated index would help readers navigate your technical content |
| llms.txt, robots.txt, and Crawlability | Moderate; requires checking access rules and page rendering | Crawlability can have a more immediate effect than llms.txt | Audit crawler access and confirm key pages load with readable text |
| Practical Steps to Improve AI Readability | Moderate; involves content updates and ongoing checks | Can improve how AI tools access and describe your product | Check access first, then clarify pricing and product information |
| Measuring Whether AI Tools Use Your Pages | Moderate; requires recurring prompt checks or tracking tools | Provides evidence of mentions, recommendations, and cited sources | Create a fixed set of buyer questions and record answers and citations |
| Common llms.txt Mistakes | Low but ongoing; requires periodic maintenance | Avoids stale, inaccessible, or misleading guidance | Review links, descriptions, and public-only content regularly |
Your Next Steps, From File to Feedback Loop#
Access first, content next, optional indexing after that. This order keeps a small documentation experiment from displacing work that buyers and retrieval systems both need.
A practical sequence is:
- Check crawler access. Review robots.txt, firewall behavior, authentication, and readable page content.
- Fix buying information. Make pricing, features, integrations, security requirements, and comparisons explicit.
- Publish a curated index where useful. Add llms.txt when documentation depth creates a clear navigation need.
- Measure actual answers. Keep prompts stable, distinguish mentions from recommendations, and inspect cited pages.
- Act on observed gaps. Correct inaccurate source pages and compare matched reporting periods.
The team should also distinguish file testing from visibility testing. Giving an assistant an index directly can reveal whether the index is useful, but it doesn't establish that the assistant discovers it independently.
Kliqo runs nightly prompt checks across ChatGPT, Google AI Overviews, Google AI Mode, Gemini, and Copilot, with a seven-day trial requiring a payment card available through its subscription options.
FAQ#
Does llms.txt improve rankings?#
There's no guaranteed ranking or citation benefit from publishing the file. Teams should treat it as an optional navigation aid and evaluate observed answers rather than assume installation improves visibility.
Where should the file be placed?#
The normal location is the website root at /llms.txt, served over HTTPS. A subpath can also be used where an index describes a particular documentation area.
Should llms.txt include every page?#
No. A curated selection of authoritative pages is more useful than an undifferentiated sitemap export, especially when older pages contain superseded guidance.
How is llms.txt different from robots.txt?#
Robots.txt expresses crawler access preferences. Llms.txt introduces the site and links to useful information, but it doesn't grant access, enforce restrictions, or protect confidential content.
How can a team tell whether AI tools use its content?#
Server logs can show requests, while citations can show which pages an answer references. Repeated prompt checks reveal how assistants describe and recommend the product, but none of these observations alone proves that llms.txt caused an outcome.
Kliqo AI monitors how AI services describe, cite, and recommend brands, with recorded answers supporting the analysis. Teams evaluating llms.txt can visit Kliqo AI to establish a baseline and assess access and content changes against actual buying answers.
