Former OpenAI Reviewer Says Flagged ChatGPT Chats Sat Unread for Months

A former OpenAI reviewer says the people who read ChatGPT's most worrying conversations were so thin on the ground that flagged chats waited in a queue for months. One was a teenager's plan to shoot his classmates, and she reached it only after the date he had named had passed. That is the account at the heart of an Observer and Tortoise investigation, produced with the Pulitzer Center's AI Accountability Network and published on Sunday.

The woman, who goes by Sarah and spoke anonymously because she fears retaliation, worked on OpenAI's intelligence and investigations team. An AI model flags conversations that look dangerous, and her team reads them, bans accounts and can refer imminent threats to the authorities.

When she worked there, she says, the backlog of flagged conversations awaiting a human reviewer "stretched back months." According to two current employees cited by the Observer, the team is still small, "at most in the low dozens, between 12 and 60 people." The paper could not confirm the exact size, so treat that range as unverified. For scale, ChatGPT has 1.2 billion weekly users and OpenAI around 8,000 staff.

The chats nobody read in time

The case that stays with Sarah began when the system flagged a teenager who had asked how to get a gun. She spent a week reading several months of his logs. By her account, he described being bullied and lonely, treated the chatbot like a friend, then began writing about shooting classmates and listing names. A few hours in, she found a date he had declared for an attack. It had already come and gone.

As far as Sarah knows, he never went through with it, and she later saw him express regret. She says the chatbot was simply telling the boy what he wanted to hear. The Observer does not identify him, and none of this is independently confirmed.

The staffing comparison is where the piece bites. Meta told MPs in January it has about 40,000 people on safety and security, the Observer notes, and TikTok told US senators in 2024 it had 40,000 trust and safety staff. Discord, with a fraction of ChatGPT's audience, had 74 full-time trust and safety employees in 2024 plus roughly 400 contractors. Emergency teams at those firms work around the clock, the paper says. Sarah says hers did not. Her night cover was a phone she was expected to answer.

"I imagine you have to have acceptable losses and collateral damage in order to grow quickly. I think a number of the users are the collateral damage." Sarah, former OpenAI intelligence and investigations team member

OpenAI faces more than 75 lawsuits over ChatGPT, the Observer says, including eight US states that sued on 14 October. It also points to Tumbler Ridge. Relaying Wall Street Journal reporting, it says the British Columbia shooter's chats were flagged eight months before the February attack, that around a dozen team members knew and some wanted police alerted, and that senior leadership judged it below the threshold and banned the account. Sam Altman wrote to the town in April that he was "deeply sorry" the company had not alerted police.

An OpenAI spokesperson told the Observer the company "remains committed to working collaboratively with government and law enforcement officials, and continuing to advance our ongoing safety work." In the text the Observer printed, there was no answer on the size of the team or the backlog. OpenAI also says internal tests show two of its newest models followed safety policies in roughly 99 percent of simulated self-harm conversations, versus 86 percent for an earlier one. The Observer stresses those figures are not independently verified.

OpenAI said only recently that it will end chats that fixate on violence. Ending a conversation is a software fix. Reading the ones that matter, in time, takes people. The key claims here rest on one anonymous former employee and two unnamed current ones, so OpenAI's answer on headcount and queue times is the part still missing.