In a significant move, Google has unveiled a new feature that empowers website publishers to decide whether their data should be used for training the tech giant’s artificial intelligence (AI) models. This decision comes amidst growing concerns about data privacy and the ethical use of information in AI training.
Key Highlights:
- Google launches the “Google-Extended” tool for publishers.
- The tool allows websites to be indexed by Google without their data being used for AI training.
- Google-Extended is accessible via robots.txt, a standard used to guide web crawlers.
- The initiative aims to provide more control and choice to web publishers.
- Concerns arise as major sites block AI data scraping, leading to potential legal challenges.
Google’s new tool, named “Google-Extended,” ensures that while sites continue to be scraped and indexed by search engine crawlers like Googlebot, their data won’t be utilized to train evolving AI models. The company emphasizes that this feature will enable publishers to manage the contribution of their sites in enhancing Google’s BARD and Vertex AI generative APIs. This means that publishers can now use the toggle to regulate access to their site’s content.
Earlier in July, Google confirmed that it was training its AI chatbot, BARD, using publicly available data scraped from the internet. The introduction of Google-Extended, which is integrated into robots.txt (a text file that instructs web crawlers on site accessibility), is a testament to Google’s commitment to providing more machine-readable choices and control mechanisms for web publishers.
However, the road to this decision wasn’t without its bumps. Several prominent websites, including The New York Times, CNN, Reuters, and Medium, had already taken steps to block the web crawler that OpenAI employs for data scraping to train ChatGPT. The challenge was that completely blocking Google’s crawlers would mean these sites wouldn’t appear in search results. This predicament led some, like The New York Times, to resort to legal means, updating their terms of service to prohibit firms from utilizing their content for AI training.
In conclusion, Google’s latest move with the introduction of the “Google-Extended” tool is a significant step towards addressing data privacy concerns in the realm of AI. By providing publishers with the choice to opt out of AI training data collection while still being indexed on the search engine, Google strikes a balance between technological advancement and ethical data use. This initiative not only underscores the importance of data autonomy but also highlights the tech giant’s commitment to fostering trust and transparency in the digital age.
