How this content is checked
We write questions with AI from published clinical sources and check them with automated review. Items marked as reviewed by a subject matter expert were reviewed by a licensed clinician; no person has reviewed the others, and every question carries its label.
These checks test whether the cited page says what the item says and whether a second model, shown only the question, reproduces the keyed answer. They do not test whether the item is clinically right.
What you are reading
- Every question, explanation, clinical case and illness script is written by an AI system from a licensed or public source and checked by automated review before it is shown.
- An item marked clinician-reviewed has been read and kept by a licensed clinician. Every other item carries the label not clinician-reviewed, and that is the honest default.
- Content can be wrong, including the keyed answer. Where an item cites a source, the source is the authority, not our explanation. Anything that looks wrong can be sent through the Contact page.
The automated checks
- Tested fact. A model states the one clinical proposition the item turns on. Items that only test a study's statistics are rejected here.
- Cited page. That proposition is checked against a public page from an allow-list of government, national-health and NIH hosts. The page must be fetched, must agree, and must carry a recent date; the item then names that page.
- Second opinion. A second model from a different vendor, shown only the question, answers twice without seeing the options; both answers must match the key, and shown the options it must find exactly one defensible answer.
- Known-wrong keys. We deliberately break the key on items that cleared the checks and measure how often the same checks catch it. That number is below.
These checks test whether the cited page says what the item says and whether a second model, shown only the question, reproduces the keyed answer. They do not test whether the item is clinically right.
Wrong-key detection, measured
Not yet measured. The detection run has not been recorded on this site yet; the number appears here as soon as it has.
Clinician spot-check
A licensed clinician whose credential we have on record works through checked items and records what they find: the cited page confirms the item, says nothing about it, contradicts it, cannot be interpreted, the keyed answer is wrong, or no single option is defensible. The clinician can withdraw an item on the spot.
Not yet started. The queue is built and waits for the first attested clinician.
Items with a check on record
| Learning path | Items served | Checked | Cited page agreed | Rejected |
|---|---|---|---|---|
| Nurse Practitioner | 207 | 1 | 0 | 1 |
| Orthopaedic Surgery | 659 | 0 | 0 | 0 |
| Physician Assistant | 479 | 2 | 0 | 2 |
A rejected item is withdrawn from practice. "Checked" counts items with a recorded decision; the rest are served under the not-clinician-reviewed label while the checks work through the bank.
Weekly citation audit
Each week a person draws 20 checked items, spread across learning paths, source hosts and the newest batch, opens the cited page for each and records one of four outcomes: confirmed, not stated on the page, page contradicts the item, or cannot interpret. "Cannot interpret" counts as not confirmed. We act when more than 3% of the last 60 audited items were not confirmed.
No audit recorded yet. The first weekly draw appears here when it is logged.
Second opinion from a different model
Items are written by claude-sonnet-5 (Anthropic). The blind second opinion comes from claude-opus-5-5 (Anthropic), a different model from the same vendor. The code refuses to run the second opinion on the model that wrote the item.
Reports from learners
Every question has a "Report a problem" control. A reported question is withdrawn from practice until a person decides: kept, edited, or retired. Anything else can still go through the Contact page.
No report decided yet. The count and the time to a decision appear here after the first one.