In July and September, 15 leading AI companies pledged to uphold the White House’s voluntary commitments aimed at mitigating the risks associated with AI. One of these commitments was a promise to enhance transparency by sharing information across the industry, with governments, civil society, and academia. These companies also pledged to publicly disclose the capabilities and limitations of their AI systems. While these commitments appear promising in theory, what do they actually mean in practice? How do these AI companies, with their massive and powerful models, define and implement transparency?
Thanks to a report led by Stanford’s Center for Research on Foundation Models (CRFM), we now have answers to these pressing questions. The foundation models under scrutiny are versatile, general-purpose creations like OpenAI’s GPT-4 and Google’s PaLM 2, trained on vast datasets and adaptable for a wide range of applications. The Foundation Model Transparency Index assessed ten of the largest models based on 100 different metrics of transparency.
Remarkably, the highest score, though still modest, was achieved by Meta’s Llama 2, with a score of 54 out of 100. However, this score would equate to a failing grade in an academic setting. The researchers noted in a blog post that “no major foundation model developer is close to providing adequate transparency,” revealing a fundamental lack of transparency within the AI industry.
Rishi Bommasani, a PhD candidate at Stanford’s CRFM and one of the leaders of the project, emphasized that the index’s purpose is to address a troubling trend in recent years: as the impact of AI models increases, the transparency provided by these models and their respective companies decreases. Notably, when OpenAI transitioned from GPT-3 to GPT-4, the company decided to withhold essential information about the model’s architecture, size, hardware, training process, and dataset construction.
The 100 transparency metrics cover various aspects, including factors related to the model’s training, details about the model’s properties and functions, and downstream factors related to the model’s distribution and usage. Kevin Klyman, a research assistant at Stanford’s CRFM and a coauthor of the report, emphasized that transparency goes beyond merely releasing the model; it also involves sharing information about the resources invested in the model, evaluating the model’s capabilities, and disclosing what occurs after its release.
To evaluate the models using these 100 indicators, researchers examined publicly available data and assigned a 1 or 0 to each indicator based on predetermined thresholds. Subsequently, they reached out to the ten companies to inquire if they contested any of the scores. In some instances, additional information was obtained that was initially missed.
The source of training data for foundation models has become a contentious issue, with several lawsuits claiming that AI companies included copyrighted material from authors in their training datasets without authorization. Unsurprisingly, the transparency index revealed that most companies were not forthcoming about their data sources. The model Bloomz from developer Hugging Face scored the highest in this category, with 60 percent, while none of the other models scored above 40 percent, with some even receiving a score of zero.
Companies also remained largely silent on the topic of labor, which is significant since human workers play a crucial role in refining these models. For instance, OpenAI uses a process called reinforcement learning with human feedback to teach models like GPT-4 how to provide appropriate and acceptable responses to humans. However, most developers do not disclose information about these human workers’ identities or their wages, raising concerns that this labor may be outsourced to low-wage workers in places like Kenya. Bommasani noted that “Labor in AI is a habitually opaque topic, and here it’s very opaque, even beyond the norms we’ve seen in other areas.”
Hugging Face is one of three developers in the index that the Stanford researchers consider “open,” meaning that the models’ weights are widely accessible. These open models, including Meta’s Llama 2, Hugging Face’s Bloomz, and Stable Diffusion from Stability AI, are currently setting the bar for transparency, outperforming or matching the best-performing closed models.
However, not everyone agrees that open sourcing such powerful models is the most responsible approach. Controversy surrounds the debate over whether such models should be open sourced, potentially falling into the wrong hands. Recently, protests took place at Meta’s San Francisco office, opposing the “irreversible proliferation” of potentially unsafe technology.
Bommasani and Klyman confirmed the commitment of the Stanford team to regularly update the index, planning to do so at least once a year. They hope that policymakers worldwide will rely on the index as they draft legislation concerning AI, given ongoing regulatory efforts in many countries. If companies make improvements in transparency across the 100 areas highlighted by the index, lawmakers can gain better insights into where intervention may be necessary. As Bommasani put it, “If there’s pervasive opacity on labor and downstream impacts, this gives legislators some clarity that maybe they should consider these things.”
It’s essential to remember that a high transparency score in the current index does not necessarily indicate that a model is a paragon of AI virtue. Even if a company discloses that a model was trained on copyrighted material and refined by underpaid workers, it can still earn points for transparency in data and labor. Bommasani emphasized that “We’re trying to surface the facts as a first step. Once you have transparency, there’s much more work to be done.”
