Who tested this AI, and could the company edit the results?
When you read that an AI was "independently tested", six questions tell you how much that test is worth. Here they are, what a strong answer looks like, and DecisionIndex's own answers with where to check each one.
The six questions
1What could the testers see?
Some testers work inside the company, with access to unreleased models, training data and staff. Others only see what the public gets. Inside access finds problems outsiders cannot; outside testing shows what actually reaches people.
A strong answersays plainly what the testers could and could not see, so you know which problems the test could ever have found.
DecisionIndexWe see only the answer a released AI gives, the same one you would get. We cannot see how a model was trained or what it does inside. Every question we test with is open: the list of sets.
2Who paid for the test?
Testers can be paid by the company they test, or take free computing credits from it. Payment does not have to bias a result, but you should be told.
A strong answernames who paid, whether the pay depended on the findings, and any credits or free access.
DecisionIndexNo AI company pays us or gives us anything, credits included, and we will not accept it. SolvingHealth, which makes DecisionIndex, pays its costs; there is no outside funder today. Any future funder will be named on this page before its money is used, and none will be an AI company.
3Could the company edit, delay or block the findings?
Some agreements let the company remove confidential details from a report, or ask for publication to wait while it fixes a problem. That can be reasonable for testers who saw private systems. You still need to know it happened.
A strong answersays whether anything was removed or delayed, and who decided.
DecisionIndexNo one outside DecisionIndex can remove, delay or veto a result, and no company is told before a result is published. We only test public answers, so there is nothing confidential to hold back.
4Who settles a disagreement?
When testers and the company disagree about a finding, someone has the last word. If that someone works for the company, or sits on its board, that matters.
A strong answernames who decides and how a disagreement is shown to readers.
DecisionIndexNo company takes part. Each answer key is checked by two qualified reviewers on their own, recorded by role and never by name, without seeing any AI's answer. If they disagree, that question is left out of scoring until they settle it. You can see the reviewer step at /score#review.
5Who decided what to test?
A company that chooses the scope can steer testers away from its weak spots. A test can be careful and still look in the wrong place.
A strong answersays who set the scope and what the testers asked for but did not get.
DecisionIndexWe pick the questions from decisions real people face: benefits, care rules, end of life, a denial, a health worry. No company sees them first. Anyone can turn a question they faced into a test question.
6Did an AI grade the answers?
Many AI tests use another AI to decide whether an answer was good. That grader can share the blind spots of the AI being tested, and its judgment cannot be rechecked exactly.
A strong answerpublishes the grading rules so anyone can rerun them and get the same result.
DecisionIndexNo AI grades anything. Every answer is checked by fixed text rules that anyone can read and rerun in their own browser: the scorer. Each check shows what it found and why. You can score an AI's answer yourself.
Two kinds of testing, side by side
On November 16, 2026, OpenAI published how four outside groups (Apollo Research, METR, Transluce and Goodfire) will test its models from the inside. It is a real step toward openness, and inside access can find failures no outsider could. The two kinds of testing answer different questions. This table uses OpenAI's own published terms.
| Question | OpenAI's embedded testers | DecisionIndex |
|---|---|---|
| What they look at | Hidden misbehavior, safeguards and alignment tests during training and internal use | The answer a released AI gives a person facing a real decision |
| Access | Unreleased models, training data, source code, monitoring, staff; on site, on company devices | Public answers only |
| Paid by the company? | Two of the four are paid (not tied to findings); the other two may take credits or compute | No, and no credits |
| Company can redact | Yes, confidential information | No |
| Company can ask for a delay | Yes, to fix safety or security problems first | No; no company is told first |
| Disputes go to | OpenAI's safety group and executives, then a committee of the OpenAI Foundation's board | Two reviewers; a disputed question is left out |
| Who sets the scope | Agreed with the company; one tester's scope is narrower than it proposed | DecisionIndex and the people who write test questions |
Source: OpenAI, "Our first phase of embedded assessments", November 16, 2026. We compare how testing is arranged, not how good any model is. DecisionIndex does not rank models or products.
Our conflicts
SolvingHealth, which makes DecisionIndex, also builds health-AI products. One is ClinicalSwipe, where a licensed physician signs or declines what an AI drafted, and our example page links to it. So that this cannot tilt a result:
- SolvingHealth's own products are never scored on DecisionIndex. A test we wrote cannot fairly grade work we sell.
- Test questions about SolvingHealth's own companies, people or products are left out of every set.
- No AI's score is published until a set's answer keys pass review, and no model or product is ranked.
Every set is still Draft today, so no score has been published under these rules yet.
This page is not a safety certification, and a good result on DecisionIndex is not a medical or legal endorsement. If someone is in danger now: call or text 988, or call 911.