Meta Delays Controversial Web Scraping Terms Until November 20 After Publisher Backlash

Meta has delayed the implementation of its controversial updated commercial terms following criticism from publishers, advertisers, and other businesses concerned about how their online content could be accessed and used. The changes, originally scheduled to take effect on October 30, will now become effective on November 20, 2026.

In statements to Reuters and Fast Company, a Meta spokesperson said the company was extending the deadline to give partners and stakeholders more time to understand what is changing and what will remain the same.

The decision comes after concerns emerged over language that appeared to give Meta broad permission to use automated tools to access and collect content from websites associated with business accounts. The proposed terms initially included wording about using that content to train AI systems, prompting criticism from media organizations and other content owners.

Why Meta Delayed Its Updated Commercial Terms

Meta's proposed commercial terms drew attention because of the permissions they described for web crawlers and automated data-collection tools.

The original wording stated that the company could use automated tools to access, index, cache, analyze, collect, retrieve, or crawl websites and their publicly available content associated with business accounts.

The terms also initially included language allowing Meta to use content from those websites to develop, improve, research, train, and maintain its products and services.

Publishers and other businesses questioned whether these provisions could allow their articles and other online material to be collected without adequate compensation or meaningful control over how the information was used.

Following the backlash, Meta revised the language to remove the explicit reference to AI training. However, the change did not eliminate all concerns about the scope of the company's proposed web-scraping permissions.

Publishers Raise Concerns About AI and Copyright

News organizations have been among the most vocal critics of the revised terms.

The News Media Alliance, which represents news publishers in the United States and Canada, sent Meta an open letter demanding that the company withdraw the updated terms before the original October 30 deadline.

The organization argued that publishers could be compelled to provide valuable journalism that might subsequently support products competing with their own businesses for readers and advertising revenue.

For publishers, the dispute involves more than website traffic. News articles, reporting, photographs, and other editorial material represent significant investments in research, staff, and production. Businesses want clarity about whether automated collection could allow their work to be used in commercial products without permission or compensation.

The controversy also reflects wider tensions between digital publishers and technology companies over the use of online material in AI development and other automated services.

Meta Says Publishers Can Use Robots.txt

As part of its response, Meta said it would explain how its web crawlers operate and clarify the options available to content owners.

The company stated that publishers and website operators would be able to use robots.txt, a standard text file that communicates crawling preferences to automated bots.

Website owners commonly use this file to tell cooperating crawlers which sections of a website they should or should not access. It can be useful for managing search engine crawling and communicating preferences to other automated systems.

However, robots.txt is not a technical access-control mechanism. It relies on crawlers respecting the instructions provided by a website.

Reports have indicated that some AI-related crawlers have continued accessing websites despite robots.txt restrictions. Consequently, publishers may still have questions about how effectively the mechanism can protect their content from unwanted collection.

Website owners should also understand that robots.txt instructions do not automatically remove content that has already been collected, guarantee that every bot will comply, or replace legal and technical access controls.

Why the New Deadline Matters for Businesses

Moving the effective date to November 20 gives Meta additional time to communicate the changes, but it does not necessarily resolve the underlying dispute.

Businesses that maintain websites connected to their Meta accounts may want to review the updated commercial terms and understand how the provisions apply to their online content.

Publishers, in particular, are likely to look for clearer explanations about the information Meta's crawlers collect, the purposes for which that information can be used, and the controls available to website owners.

The revised deadline could also give industry groups and regulators more time to assess the concerns raised about the proposed terms.

For smaller websites and independent publishers, the issue is especially relevant because producing original content can require substantial time and resources, while the terms governing automated data collection may be difficult to negotiate individually.

Meta's Web Crawling Policy Remains Under Scrutiny

The controversy highlights a broader challenge facing the internet: how technology companies can collect and analyze publicly accessible information while respecting the interests of the businesses and creators who produce it.

Meta's removal of the explicit AI-training reference was a response to criticism, but publishers remain concerned about the wider permissions described in the revised terms.

The company's promise to clarify its crawling practices and support the use of robots.txt may provide additional information, although the practical effectiveness of those controls will depend partly on crawler behavior and the final terms.

Final Thoughts

Meta has pushed the planned start date for its updated commercial terms from October 30 to November 20, 2026, following criticism from publishers and other businesses. The company says the extension will give stakeholders more time to understand the changes and its web-crawling practices.

Although Meta has removed the explicit reference to AI training from the revised wording, questions remain about website scraping, content ownership, and publishers' ability to control the use of their material.

The next important development will be whether Meta provides enough clarity to address those concerns before the new deadline.