Crawler and bot policy
Which automated clients are allowed, which are challenged, which are blocked, and why the site does not use User-Agent as a security boundary.
Written for people and for machines, and deliberately not gated behind an account.
Public documentation: the crawler and bot policy, the HTTP API reference with its OpenAPI schema, our security practices, and the brand and entity reference.
Which automated clients are allowed, which are challenged, which are blocked, and why the site does not use User-Agent as a security boundary.
The public HTTP API behind the site: endpoints, versioning, rate limits, authentication and error format. Machine-readable schema at /openapi.json.
How this site and the systems behind it are secured: headers, transport, secrets, dependencies, monitoring and disclosure.
The canonical name, description, logo, domains and social handles. Use these verbatim in directories, articles and structured data.
Last reviewed · Site changelog