Writing and Vetting Exam Questions¶
The Prompts This Guide Uses
Single Best Answer Item Writer
Drafts exam-quality single best answer items from your learning objectives.
Have ready: the learning objective each item must test and the blueprint slot it fills, the target level (preclinical or clinical), and your exam committee's item format conventions.
Item Flaw Checker
Audits draft exam items against standard technical flaw criteria.
Works with any capable assistant in the Tool Directory.
The Four-Minute Version
The Workflow¶
- Draft. Run the Single Best Answer Item Writer in an artificial intelligence (AI) assistant with one objective at a time. Generate two or three variants per objective; variety is cheap and your selection instinct is fast. Bring your own item-writing standards: the library prompts encode the common National Board of Medical Examiners style rules (vignette-dependent, lead-in answerable cold, homogeneous options, no absolutes or cues). For a team-based learning session, the Team-Based Learning Session Builder drafts the readiness questions and application cases together, in the American University of Antigua's format.
- Vet structurally. Feed each candidate through the Item Flaw Checker. Have it report flaws before proposing any rewrite, so you see the diagnosis, not just a polished surface.
- Vet for content. This step is entirely yours: clinical accuracy, currency of the underlying knowledge, blueprint fit, and difficulty for your cohort. The published comparisons find AI-drafted items falling short on these points: a wrong answer key, content that drifts from what the item should test, or an item easier than intended. Check that each wrong option stands for a mistake a student could plausibly make, and keep a one-line note of which one; it helps at item review and when you go over the exam with students.
- Pilot like any item. Send AI-drafted items through your normal exam review committee and post-exam item analysis, as you would any new item: a second reader and the statistics catch what one author misses. Flag their origin in your records so you can compare their performance statistics over time. The Post-exam Item Analysis Reader helps read those statistics, working from the numbers alone.
Good Practice for This Task¶
- Questions from a licensed question bank belong to their publisher, and their license usually limits reuse (the policy's intellectual property section covers licensed material), so new items written from your own learning objectives are the better route, and they test what you taught.
- Build vignettes from invented details, or from a real case abstracted until no one could recognize it. A recognizable case can identify the patient to anyone who was on that rotation. The AI Responsible Use Policy is the reference for patient information.
- If past exam performance shapes a new item, aggregate statistics (difficulty, how often each option was chosen) are all the drafting needs.
Before You Rely on It¶
For your own checking: ticks are not saved or sent anywhere.
About This Task¶
Use AI to draft and structurally vet multiple-choice questions (MCQs), cutting item-writing time while the calls that matter stay with you and your exam committee: clinical accuracy, blueprint fit, and what goes on a live exam.
Where AI Helps, and Where It Hurts¶
Models draft plausible vignette-based items quickly, and they make a useful first pass at structural vetting. In a 2026 comparison with expert reviewers, the stronger models agreed closely with the experts on objective flaws such as negatively worded stems, and poorly on judgment calls such as a conspicuous correct answer or options that do not match in structure, so your own read stays part of the structural check. Published comparisons, gathered in a 2026 review of 71 studies, find AI items usable but uneven, and the review concludes that the evidence supports them as drafts, not for unsupervised use on a summative exam (see further reading under the MCQ prompts). What models cannot do reliably is judge clinical accuracy, know your blueprint, or carry accountability for an exam. The workflow above keeps those with you.
Related: Module 4: Teaching and Assessment for the principles, and the item-vetting prompts in the prompt library.
Content last reviewed October 12, 2026. Review dates are maintained as described on the About page.