OpenAI, Google, Microsoft and Anthropic say they will adopt a new standard that publishers can use to opt out of being removed by AI, as previously reported Roundtable on search engines. With this control, editors can apply their AI training preferences across multiple AI companies, rather than setting them individually.
The shared “policy token” token is proposed by Kevin Kelley, standards technical lead at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live inside a website’s robots.txt file, which has long signaled which web crawlers can and cannot crawl a site, whether for search indexing, archiving, or…
Read the full story at The Verge.
