# Paid-ads landing pages under /lp/* are intentionally NOT disallowed here. # They are de-indexed with `noindex` (LpLayout + the X-Robots-Tag in _headers) # and excluded from the sitemap. A `Disallow: /lp/` would block the crawl, so # Google would never read the noindex — and could still index the URL from the # ads that link to it. Crawlable + noindex is what actually de-indexes a page. # WordPress-era endpoints (Session 62). The Astro site has no /wp-admin/ or # /wp-login.php; Google kept requesting them from the old site's links, so it # is told to stop. /wp-content/ is NOT disallowed — three ranking PDFs still # live under it (public/wp-content/uploads). # AI crawlers and answer engines (GPTBot, OAI-SearchBot, ChatGPT-User, # ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, # CCBot, Bingbot …) are ALLOWED on purpose (Session 68, AI search readiness). # Do not add per-bot groups: a named group replaces `*` for that bot, so it # would silently lose the two WordPress Disallow lines below. A plain-text # brief for them lives at https://kitchencabinetsandbeyond.com/llms.txt User-agent: * Allow: / Disallow: /wp-admin/ Disallow: /wp-login.php Sitemap: https://kitchencabinetsandbeyond.com/sitemap-index.xml