There is currently a public discussion (notably on Hacker News) regarding observations that Google appears to be under-indexing or failing to index specific, technical information related to the AT Protocol (atproto), such as lists of public relays.
There is no official statement from Google explaining why this might be happening. However, the community discussion highlights several potential factors often associated with indexing challenges:
* **Indexing Protocols:** Google does not support the "IndexNow" protocol, which is used by many modern web services to push updates to search engines instantly. Google relies on its own crawling infrastructure and its own Indexing API, which may lead to differences in how quickly or thoroughly it indexes content compared to search engines that support IndexNow.
* **Technical Nature of the Content:** The AT Protocol is a decentralized, federated network. Content on such networks can sometimes be difficult for traditional crawlers to discover if it is not well-linked from established, static web pages.
* **Crawling and Indexing Processes:** Google's indexing is a complex, automated process that relies on many factors, including site authority, crawl budget, and link structure. It is possible that specific technical documentation or community-generated lists for atproto do not currently meet the criteria Google uses to prioritize content for its index.
In summary, while Google does index some information about the AT Protocol (such as its Wikipedia page), users have observed gaps in its coverage of specific technical resources, which is likely due to a combination of how Google's crawlers discover content and the decentralized nature of the protocol itself. There is no evidence of a deliberate policy to exclude this information.
1searchis Google indexing information about the AT Protocol