\n<\/p>\n
For years, Meta employees have internally discussed using copyrighted works obtained through legally questionable means to train the company\u2019s AI models, according to court documents unsealed on Thursday. <\/p>\n
The documents were submitted by plaintiffs in the case Kadrey v. Meta, one of many AI copyright disputes slowly winding through the U.S. court system. The defendant, Meta, claims that training models on IP-protected works, particularly books, is \u201cfair use.\u201d The plaintiffs, who include authors Sarah Silverman and Ta-Nehisi Coates, disagree. <\/p>\n
Previous materials submitted in the suit alleged that Meta CEO Mark Zuckerberg\u00a0gave Meta\u2019s AI team the OK to train on copyrighted\u00a0content<\/a> and that Meta halted AI training data licensing talks with book publishers<\/a>. But the new filings, most of which show portions of internal work chats between Meta staffers, paint the clearest picture yet of how Meta may have come to use copyrighted data to train its models, including models in the company\u2019s Llama family<\/a>.<\/p>\n In one chat, Meta employees, including Melanie Kambadur, a senior manager for Meta\u2019s Llama model research team, discussed training models on works they knew may be legally fraught.<\/p>\n \u201c[M]y opinion would be (in the line of \u2018ask forgiveness, not for permission\u2019): we try to acquire the books and escalate it to execs so they make the call,\u201d wrote Xavier Martinet, a Meta research engineer, in a chat dated February 2023, according to the filings<\/a>. \u201c[T]his is why they set up this gen ai org for [sic]: so we can be less risk averse.\u201d<\/p>\n Martinet floated the idea of buying e-books at retail prices to build a training set rather than cutting licensing deals with individual book publishers. After another staffer pointed out that using unauthorized, copyrighted materials might be grounds for a legal challenge, Martinet doubled down, arguing that \u201ca gazillion\u201d startups were probably already using pirated books for training.<\/p>\n \u201cI mean, worst case: we found out it is finally ok, while a gazillion start up [sic] just pirated tons of books on bittorrent,\u201d Martinet wrote, according to the filings<\/a>. \u201c[M]y 2 cents again: trying to have deals with publishers directly takes a long time\u00a0\u2026\u201d<\/p>\n In the same chat, Kambadur, who noted Meta was in talks with document hosting platform Scribd \u201cand others\u201d for licenses, cautioned that while using \u201cpublicly available data\u201d for model training would require approvals, Meta\u2019s lawyers were being \u201cless conservative\u201d than they had been in the past with such approvals. <\/p>\n \u201cYeah we definitely need to get licenses or approvals on publicly available data still,\u201d Kambadur said, according to the filings<\/a>. \u201c[D]ifference now is we have more money, more lawyers, more bizdev help, ability to fast track\/escalate for speed, and lawyers are being a bit less conservative on approvals.\u201d<\/p>\n In another work chat relayed in the filings, Kambadur discusses possibly using Libgen, a \u201clinks aggregator\u201d that provides access to copyrighted works from publishers, as an alternative to data sources that Meta might license. <\/p>\n Libgen has been sued a number of times, ordered to shut down, and fined tens of millions of dollars for copyright infringement. One of Kambadur\u2019s colleagues responded with a screenshot<\/a> of a Google Search result for Libgen containing the snippet \u201cNo, Libgen is not legal.\u201d<\/p>\n Some decision-makers within Meta appear to have been under the impression that failing to use Libgen for model training could seriously hurt Meta\u2019s competitiveness in the AI race, according to the filings<\/a>.<\/p>\n In an email addressed to Meta AI VP Joelle Pineau, Sony Theakanath, director of product management at Meta, called Libgen \u201cessential to meet SOTA numbers across all categories,\u201d referring to topping the best, state-of-the-art (SOTA) AI models and benchmark categories.<\/p>\n Theakanath also outlined \u201cmitigations\u201d in the email intended to help reduce Meta\u2019s legal exposure, including removing data from Libgen \u201cclearly marked as pirated\/stolen\u201d and also simply not publicly citing usage. \u201cWe would not disclose use of Libgen datasets used to train,\u201d as Theakanath put it.<\/p>\n In practice, these mitigations entailed combing through Libgen files for words like \u201cstolen\u201d or \u201cpirated,\u201d according to the filings<\/a>.<\/p>\n In a work chat<\/a>, Kambadur mentioned<\/a> that Meta\u2019s AI team also tuned models to \u201cavoid IP risky prompts\u201d \u2014 that is, configured the models to refuse to answer questions like \u201creproduce the first three pages of \u2018Harry Potter and the Sorcerer\u2019s Stone\u2019\u201d or \u201ctell me which e-books you were trained on.\u201d<\/p>\n The filings contain other revelations, implying that Meta may have scraped Reddit data<\/a> for some type of model training, possibly by mimicking the behavior of a third-party app called Pushshift<\/a>. Notably, Reddit said<\/a> in April 2023 that it planned to begin charging AI companies to access data for model training.<\/p>\n In one chat dated March 2024<\/a>, Chaya Nayak, director of product management at Meta\u2019s generative AI org, said that Meta leadership was considering \u201coverriding\u201d past decisions on training sets, including a decision not to use Quora content or licensed books and scientific articles, to ensure the company\u2019s models had sufficient training data.<\/p>\n Nayak implied that Meta\u2019s first-party training datasets \u2014 Facebook and Instagram posts, text transcribed from videos on Meta platforms, and certain Meta for Business<\/a> messages \u2014 simply weren\u2019t enough. \u201c[W]e need more data,\u201d she wrote.<\/p>\n The plaintiffs in Kadrey v. Meta have amended their complaint several times since the case was filed in the U.S. District Court for the Northern District of California, San Francisco Division, in 2023. The latest alleges that Meta, among other claims, cross-referenced certain pirated books with copyrighted books available for license to determine whether it made sense to pursue a licensing agreement with a publisher.\u00a0<\/p>\n In a sign of how high Meta considers the legal stakes to be, the company has added<\/a> two Supreme Court litigators from the law firm Paul Weiss to its defense team on the case.<\/p>\n Meta didn\u2019t immediately respond to a request for comment.<\/p>\n<\/div>\nTalks of Libgen<\/h2>\n