Tools / llms.txt Generator & Checker (plus llms-full.txt)

llms.txt Generator & Checker (plus llms-full.txt)

Make any website AI-ready. Input: websites. Output (JSON/CSV): one row per item with website, siteName, pagesIncluded, existingLlmsTxt, llmsTxtUrl, llmsFullTxtUrl, llmsFullTxtTokenEstimate, recommendation.

Run it on Apify Store →

What it's for

Generate an llms.txt for your own or a client's site in one run
Audit whether competitors' sites are AI-ready
Package a docs site into one Markdown file for an LLM context window

Example input

{
 "websites": [
  "https://therealtour360.com",
  "https://docs.anthropic.com",
  "nonexistent-domain-zzq81.com"
 ],
 "maxPagesPerSite": 12
}

Example output

{
  "website": "https://therealtour360.com",
  "existingLlmsTxt": {
    "found": false,
    "httpStatus": 200,
    "note": "URL returns an HTML page, not a Markdown file"
  },
  "existingLlmsFullTxt": {
    "found": false,
    "httpStatus": 404
  },
  "llmsTxtUrl": "https://api.apify.com/v2/key-value-stores/xgqjg2VctotpkdWdF/records/therealtour360.com-llms.txt?signature=13cSr8HdDVO7e8NbRugej",
  "llmsFullTxtUrl": "https://api.apify.com/v2/key-value-stores/xgqjg2VctotpkdWdF/records/therealtour360.com-llms-full.txt?signature=DV8V5Anm9xda2l2jLKaj",
  "llmsFullTxtSizeBytes": 68703,
  "llmsFullTxtTokenEstimate": 17150,
  "siteName": "Real Tour 360",
  "pagesIncluded": 12,
  "sections": 2,
  "generatedLlmsTxt": "# Real Tour 360\n\n> Real estate photography, video, drone and Zillow 3D tours in Southern Middle Tennessee and North Alabama. Compare package…",
  "recommendation": "No llms.txt found; upload the generated file to https://therealtour360.com/llms.txt",
  "generatedAt": "2026-10-12T04:44:10.731Z"
}

Input reference

Field	Type	Notes
`websites` required	array	Homepages or docs roots.
`maxPagesPerSite`	integer	Max pages per site
`useSitemap`	boolean	Use sitemap.xml
`buildFullText`	boolean	All pages as one Markdown file.
`mainContentOnly`	boolean	Main content only
`includeLinks`	boolean	Keep links in llms-full.txt
`respectRobots`	boolean	Respect robots.txt
`maxConcurrency`	integer	Max concurrent sites

Pricing

Pay per event: llms.txt generated for a site $0.02, Page crawled and included $0.0005. Plus Apify's small per-run start fee. Apify's free plan includes monthly credit to try it.

How to run

Open the Store page and click Try for free.
Fill in the input (or paste the example JSON in the JSON tab).
Click Start, then export results as CSV, Excel or JSON, or call it via the Apify API.
Optional: add a Schedule and a webhook/Slack/email integration for ongoing monitoring.

Open llms.txt Generator & Checker (plus llms-full.txt) on Apify →