# robots.txt — reputoo.com # # Read this before filing the blocks below as a mistake. # # Nothing here is public content hidden from search. Every Disallow is one of # three things: an authenticated surface behind a login, a URL that carries a # per-recipient token, or a machine endpoint that returns JSON rather than a # page. All of it is either sensitive or worthless to index, and in several # cases both. The public marketing site — every page in the sitemap — is open, # and that is the whole of what we want crawled. # # The reason is stated against each group. These paths also return # X-Robots-Tag: noindex (see next.config.js), so a URL that reaches an index # by some route other than a crawl of this file still stays out. User-agent: * Allow: / # --- Authenticated product surfaces ----------------------------------------- # Signed-in application UI. Behind auth, so a crawler gets a redirect to the # login screen and nothing else; the real content is customer data. Disallow: /admin/ Disallow: /dashboard/ Disallow: /agent/ Disallow: /team/ # --- Credential entry points ------------------------------------------------ # Linked from every page (the header CTA), which is why an audit finds them. # They are forms, not content: nothing to rank, and no reason to send search # traffic at a password field. Disallow: /login Disallow: /signup # --- Per-recipient token URLs ----------------------------------------------- # Each of these resolves against a token that identifies one customer, one # invoice or one referral. They are sensitive by construction: the URL IS the # credential, so indexing one publishes a private document. Crawling them would # also fire real-world side effects — marking a review request as opened, # attributing a referral click. Disallow: /wa/ Disallow: /r/ Disallow: /invoice/ Disallow: /review/ Disallow: /referral/ # --- Machine endpoints ------------------------------------------------------ # JSON APIs and the embeddable widget bundle. No document, no title, no text — # a crawler that follows these logs a page with nothing on it. Disallow: /api/ Disallow: /widget/ # /docs is deliberately NOT blocked. It is a public, unauthenticated API # reference with no secrets in it, and developer documentation is among the # most-retrieved content on AI answer engines. It was disallowed once; that # removed a maturity signal and protected nothing. Sitemap: https://www.reputoo.com/sitemap.xml