Auth0 Universal Login Pages, such as /u/login?state=..., sometimes appear in Google Search Results. This occurs when external sources post direct links on public websites, bypassing the Auth0-hosted robots.txt restrictions. Resolve this issue temporarily using the Google Search Console, or permanently by hosting the own robots.txt file for customers using custom domains with a self-managed certificate.

• Universal Login
• Custom Domains

External sources post direct links to the Universal Login Pages on public websites or social media platforms. Search engines index these links despite the Auth0-hosted robots.txt file disallowing crawler access.

What is the temporary method to remove Universal Login Pages from search results?

Access the search engine tools to temporarily remove the indexed pages.

1. Open Google Search Console.
2. Submit a request to remove the indexed Universal Login Pages. This removal remains effective for a certain period.

What is the permanent method to remove Universal Login Pages from search results?

Host the own robots.txt to permanently remove the pages from search results.

1. Ensure the tenant uses a custom domain with a self-managed certificate.
2. Create and host the own robots.txt file to allow crawlers to access the Universal Login Pages.
3. Route the URL path /robots.txt to the file in the system to let the crawlers to read the HTTP response header or meta tag on the page that explicitly forbids indexing. This action permanently removes the pages from search results.

Related References

• Block Search Indexing with noindex