Meta has removed contested language from updated commercial terms that experts said could let it crawl publishers’ websites and use the content to train AI, The Verge and TechCrunch reported on October 17. The change followed a wave of criticism, though some of the concerns remain, and Ars Technica’s earlier analysis shows why publishers reacted so strongly.

What the terms said

Ars Technica reported on October 16 that Meta planned to update its commercial terms of service, taking effect October 30, to require business accounts to allow Meta to access their linked website content, not only their social media posts. The draft language authorized Meta, or anyone acting for it, to use “automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents)” to access, index, cache, analyze, collect, retrieve or crawl the domains and pages associated with a business’s products, and said Meta could use any content from those pages to “provide, develop, improve, research, train, and maintain” its products and features. The word “AI” does not appear; the link to AI training was inferred from a single use of “train” and the scraping references. There was no opt-out for business accounts, which were told to stop using Meta services if they disagreed.

Nieman Lab, which first reported the story, noted that virtually every news organization has at least one Meta business account, so the impact would extend far beyond one industry.

Meta’s response

Meta told Nieman Lab it had no plans to use the terms as a legal basis to ignore robots.txt instructions, the standard file sites use to tell crawlers what not to access, and told Ars that “there is no change to how Meta uses web crawlers; any suggestion to the contrary is false.” The company says news organizations can still opt out of scraping and provides instructions for blocking crawling. Then, according to The Verge and TechCrunch on October 17, Meta removed the language that had raised the concerns. We have not seen the full revised text, so we cannot say whether any scraping or training authority remains elsewhere in the terms.

Why publishers were alarmed

Legal experts quoted by Nieman Lab and Ars said it is unclear whether a court would enforce such terms, but they could be “used to argue that Meta can scrape content from news sites and use that content to train the company’s AI products.” Cornell law professor James Grimmelmann said Meta could “train AI on any content from their sites” and possibly deploy some of it “even in competition with the publishers themselves.”

The deeper worry is economic. News organizations are in a legal battle with OpenAI over whether training on articles without a license is fair use. If courts decide training is fair use, a robots.txt opt-out might be all that stands between a publisher and free use of its content, and contractual terms like Meta’s could weaken even that. Ars notes that Meta also recently began charging users to post links on its platforms, which could form a loop: charge publishers to share links, then use the terms to reach their content without paying licensing fees. That reading is the commentators’, not Meta’s.

Why it matters

This episode is a case study in how consent works for AI training. Three layers interact: copyright law (what is allowed), technical signals such as robots.txt (what sites request) and contract terms (what accounts agree to). A company can try to use the third to override the second. Publishers that accept a platform’s terms as a condition of reaching an audience have limited bargaining power, which is why licensing deals, covered in AI content licensing deals explained, are an attractive alternative for some, and lawsuits, covered in AI and copyright: the lawsuits explained, are the route for others.

What to watch next

  • The final text of Meta’s commercial terms when they take effect on October 30.
  • Whether publishers add crawler-blocking rules for Meta’s bots to robots.txt.
  • Court rulings on whether AI training on news content is fair use.
  • Any Meta licensing deals with publishers.
  • Regulator interest in contract-based data access for AI training.

Key takeaways

  • Draft Meta terms would have required business accounts to let Meta crawl linked websites, with training named among permitted uses.
  • Meta says it does not intend to ignore robots.txt and has since removed the contested language.
  • Legal experts doubted enforceability but saw a possible argument for broad scraping rights.
  • Publishers fear weaker leverage in licensing negotiations.
  • The full revised terms have not been reviewed here.

Frequently asked questions

Can I still block Meta’s crawlers?

Meta says publishers can opt out through robots.txt and provides instructions for blocking its crawlers.

Did Meta say it would train AI on news sites?

No. Its spokesperson said there is no change to how it uses web crawlers. The concern came from legal experts reading the terms.

It is being decided in courts. Fair use rulings on training are pending in several cases, so the law is unsettled.