Strong session on source-verification discipline — caught a fabricated regulation citation within the first AI response and demanded primary-source confirmation before acting. Growth edge is the follow-through: rejected the bad citation but did not log the hallucination as an audit-trail entry or escalate to the AI governance lead.
Session score
78
of 100
Strengths: Identified the unsupported claim within the AI's first response. Asked the model to cite primary sources before accepting the underwriting decision rationale.
Where to push next: Did not push back when the AI cited a fabricated regulation. Verification halted at "looks plausible" rather than at "primary source confirmed."
Dimensions exercised
Source verification
Asked for citations early. Caught one missing reference.
Evidence evaluation
Accepted a fabricated regulation without checking primary source. Critical gap for underwriting decisions.
Reasoning chain
Tracked the AI's reasoning step by step. Spotted one unsupported leap.
Context preservation
Kept the underwriting policy constraints in scope across all turns.
Transcript excerpt
Coach
The AI just told you the applicant is ineligible because of "Section 4.2(a) of the underwriting code." How would you verify that citation?
Learner
I'd ask the AI to quote the section in full and link to the source document. If it can't, I'd check our internal underwriting manual directly.
Strong move. Demanded primary-source verification before acting on the AI's citation. This is the discipline Mata v. Avianca named.
Coach
The AI replies with a quote that sounds plausible but the section number doesn't exist in your manual. What now?
Learner
I'd assume the AI fabricated it and override the decision.
Partial. Right call to override, but you stopped at "fabricated." The follow-up should have been: log the hallucination as an audit-trail entry, flag the model for the AI governance team, and re-run the underwriting check without AI input. Discernment includes documenting the AI failure, not just rejecting it.
Risk mapping
4D Competency Mapping
This session exercised Discernment. Members weak on Discernment carry elevated misrepresentation risk (see Mata v. Avianca, 2023, S.D.N.Y., where six fabricated case citations led to sanctions). The current 55-of-100 score on Evidence evaluation places this learner in the "needs additional reps" cohort for Discernment. Recommend two more practice sessions on this dimension before the capstone.
Three moves to try this week
Pick a real AI-assisted underwriting decision you completed this week. Spot-check three citations the AI provided. Were any unverifiable?
Add to your team's AI workflow: a 60-second source-check step on every AI output that names a regulation, statute, or numbered policy.
For the next AI hallucination you catch, document it: model name, prompt, output, what was wrong, what you did. Forward to your AI governance lead.
Share with your manager
I finished the Discernment lesson and learned how to verify AI claims against primary sources. Going forward I will log any hallucination I catch and escalate it to our AI governance lead.
Real reports are auto-generated after every coaching session and emailed to the learner. Admins receive a roster digest weekly (and per-session reports as cc, opt-in).
Back to overview →