\n<\/p>\n
OpenAI has been accused<\/a> by many<\/a> parties of training its AI on copyrighted content sans permission. Now a new paper<\/a> by an AI watchdog organization makes the serious accusation that the company increasingly relied on nonpublic books it didn\u2019t license to train more sophisticated AI models.<\/p>\n AI models are essentially complex prediction engines. Trained on a lot of data \u2014 books, movies, TV shows, and so on \u2014 they learn patterns and novel ways to extrapolate from a simple prompt. When a model \u201cwrites\u201d an essay on a Greek tragedy or \u201cdraws\u201d Ghibli-style images, it\u2019s simply pulling from its vast knowledge to approximate. It isn\u2019t arriving at anything new.<\/p>\n While a number of AI labs, including OpenAI, have begun embracing AI-generated data to train AI as they exhaust real-world sources (mainly the public web), few have eschewed real-world data entirely. That\u2019s likely because training on purely synthetic data comes with risks, like worsening a model\u2019s performance.<\/p>\n The new paper, out of the AI Disclosures Project, a nonprofit co-founded in 2024 by media mogul Tim O\u2019Reilly and economist Ilan Strauss, draws the conclusion that OpenAI likely trained its GPT-4o<\/a> model on paywalled books from O\u2019Reilly Media. (O\u2019Reilly is the CEO of O\u2019Reilly Media.)<\/p>\n In ChatGPT<\/a>, GPT-4o is the default model. O\u2019Reilly doesn\u2019t have a licensing agreement with OpenAI, the paper says.<\/p>\n \u201cGPT-4o, OpenAI\u2019s more recent and capable model, demonstrates strong recognition of paywalled O\u2019Reilly book content\u00a0\u2026 compared to OpenAI\u2019s earlier model GPT-3.5 Turbo,\u201d wrote the co-authors of the paper. \u201cIn contrast, GPT-3.5 Turbo shows greater relative recognition of publicly accessible O\u2019Reilly book samples.\u201d<\/p>\n The paper used a method called DE-COP<\/a>, first introduced in an academic paper in 2024, designed to detect copyrighted content in language models\u2019 training data. Also known as a \u201cmembership inference attack,\u201d the method tests whether a model can reliably distinguish human-authored texts from paraphrased, AI-generated versions of the same text. If it can, it suggests that the model might have prior knowledge of the text from its training data.<\/p>\n The co-authors of the paper \u2014 O\u2019Reilly, Strauss, and AI researcher Sruly Rosenblat \u2014 say that they probed GPT-4o, GPT-3.5 Turbo<\/a>, and other OpenAI models\u2019 knowledge of O\u2019Reilly Media books published before and after their training cutoff dates. They used 13,962 paragraph excerpts from 34 O\u2019Reilly books to estimate the probability that a particular excerpt had been included in a model\u2019s training dataset.<\/p>\n According to the results of the paper, GPT-4o \u201crecognized\u201d far more paywalled O\u2019Reilly book content than OpenAI\u2019s older models, including GPT-3.5 Turbo. That\u2019s even after accounting for potential confounding factors, the authors said, like improvements in newer models\u2019 ability to figure out whether text was human-authored.<\/p>\n \u201cGPT-4o [likely] recognizes, and so has prior knowledge of, many non-public O\u2019Reilly books published prior to its training cutoff date,\u201d wrote the co-authors. <\/p>\n It isn\u2019t a smoking gun, the co-authors are careful to note. They acknowledge that their experimental method isn\u2019t foolproof and that OpenAI might\u2019ve collected the paywalled book excerpts from users copying and pasting it into ChatGPT.<\/p>\n Muddying the waters further, the co-authors didn\u2019t evaluate OpenAI\u2019s most recent collection of models, which includes GPT-4.5 and \u201creasoning\u201d models such as o3-mini and o1. It\u2019s possible that these models weren\u2019t trained on paywalled O\u2019Reilly book data or were trained on a lesser amount than GPT-4o.<\/p>\n That being said, it\u2019s no secret that OpenAI, which has advocated for looser restrictions<\/a> around developing models using copyrighted data, has been seeking higher-quality training data for some time. The company has gone so far as to hire journalists to help fine-tune its models\u2019 outputs<\/a>. That\u2019s a trend across the broader industry: AI companies recruiting experts in domains like science and physics to effectively have these experts feed their knowledge into AI systems<\/a>.<\/p>\n It should be noted that OpenAI pays for at least some of its training data. The company has licensing deals in place with news publishers, social networks, stock media libraries, and others. OpenAI also offers opt-out mechanisms \u2014 albeit imperfect ones<\/a> \u2014 that allow copyright owners to flag content they\u2019d prefer the company not use for training purposes.<\/p>\n Still, as OpenAI battles several suits over its training data practices and treatment of copyright law in U.S. courts, the O\u2019Reilly paper isn\u2019t the most flattering look.<\/p>\n OpenAI didn\u2019t respond to a request for comment.<\/p>\n<\/div>\n