OpenAI is actively seeking third-party contractors to submit actual assignments and tasks from their current or former workplaces. This initiative aims to utilize the data for evaluating the performance of next-generation AI models, as revealed in records obtained by WIRED.

Establishing a Human Baseline for AI Comparison

This project is part of OpenAI’s broader efforts to create a human performance benchmark across various tasks, allowing comparisons with AI models. In September, the company introduced a new evaluation process to measure AI performance against human professionals, a crucial step toward achieving Artificial General Intelligence (AGI)—an AI system that surpasses human capabilities in economically valuable tasks.

Data Collection from Diverse Occupations

Confidential documents from OpenAI indicate that they have enlisted professionals from various fields to help compile real-world tasks based on actual job experiences. “We need detailed examples of long-term or complex work that you’ve accomplished in your occupation,” the document states, encouraging contractors to convert these tasks into measurable units.

Submission Guidelines for Contractors

Contractors are instructed to describe personal tasks and provide authentic work examples in various formats, including Word documents, PDFs, and spreadsheets. OpenAI allows for the submission of fabricated examples as a means to illustrate responses to specific scenarios, expanding the diversity of data collected.

Components of Real-World Tasks

The OpenAI presentation elaborates on two key components of real-world tasks: the task request (instructions from a manager) and the task deliverable (the resultant work). OpenAI emphasizes that the examples provided should reflect genuine work experiences.

Privacy Considerations and IP Risks

Contractors are advised to remove any corporate intellectual property and personally identifiable information from their submissions. OpenAI includes reminders to anonymize sensitive information, as the sharing of confidential documents poses potential legal risks, including trade secret misappropriation claims.

Legal Implications for Contractors

Expert Evan Brown, an intellectual property lawyer, warns that sharing scrapped documents could lead to breaches of nondisclosure agreements. “The risk of leaking confidential information is significant,” Brown states, cautioning that AI labs depend heavily on contractors to determine what constitutes confidential material. This trust could expose AI labs to substantial legal risks.