# AI crawler policy: every AI crawler is deliberately allowed.
#
# The major operators separate their crawlers by purpose, and the three
# categories carry different trade-offs (verified against first-party docs,
# Aug 2026):
#
#   search/discovery  OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot,
#                     bingbot — these are how Juno gets surfaced and cited in
#                     AI search. Blocking them removes Juno from those answers.
#   user-triggered    ChatGPT-User, Claude-User — fetch a page because a person
#                     asked about it. Blocking them breaks a live buyer question.
#   model training    GPTBot, ClaudeBot, Google-Extended, CCBot — corpus
#                     collection. Blocking these costs no search visibility
#                     (Google states Google-Extended is not a Search ranking
#                     signal), but Google-Extended also gates Gemini *grounding*,
#                     so blocking it does cost answer visibility there.
#
# Juno's public marketing content is positioning we want quoted, so all three
# are allowed. Revisit deliberately rather than by adding a blanket AI block.

User-agent: *
Disallow: /admin
Disallow: /api
Disallow: /app
Disallow: /settings
Disallow: /profile
Disallow: /response-quality
Disallow: /onboarding
Disallow: /chat
Disallow: /demo
# NOTE: there is deliberately no `Disallow: /marketing` here. That rule was
# added for a retired `/marketing/*` route, but the route is gone and the
# prefix also covers `web/static/marketing/` — which holds the brand logo shown
# in the header of every marketing and documentation page, and both DM Sans
# webfonts the whole marketing surface renders in. The rule therefore told
# every crawler not to fetch the site's own logo and body typeface while
# protecting nothing, since the paths it was written for now 404 on their own.

# Tokenised participant surfaces (/room, /join, /invite, /embed) are
# deliberately NOT disallowed here. They answer `X-Robots-Tag: noindex` from
# hooks.server.ts, and a noindex only works if the crawler is allowed to fetch
# the page and read it — a disallowed URL can still be indexed from an external
# link, and can never be seen to carry the directive. robots.txt is a crawl
# control; noindex is the index control. Neither is the privacy boundary: that
# is the token check in the interview API.

# A plain-text index of what Juno is and where to read more, written for a
# model reading the site rather than a crawler ranking it. Generated from the
# same sources as the sitemap, so it cannot drift from the pages it lists.
# https://www.heyjuno.co/llms.txt

Sitemap: https://www.heyjuno.co/sitemap.xml