INFO 4340 Analytics and AI

Week 8: Failures, Incidents, and Risk

Evaluate | AI failures are usually sociotechnical - not just technical

Module: Evaluate  |  Deliverable: AI incident brief (due before Week 9 begins)  |  See assignment →

Learning objectives

By the end of this week, you should be able to:

  1. Identify eight common AI failure modes and the sociotechnical failure chain
  2. Analyze a real AI incident using a structured diagnosis worksheet
  3. Compute subgroup error rates and formal disparity metrics (disparate impact ratio, equalized odds, chi-square)
  4. Estimate quantitative harm at production scale (errors/month × cost per error)
  5. Map failure modes to safeguards with quantitative trigger thresholds

Sessions

Session 1 · Class 15
When AI goes wrong

Eight failure modes, the AI Incident Database, and an incident diagnosis workshop. Groups analyze real incidents and estimate harm quantitatively: how many affected, for how long, at what cost.

Eight failure modes AI Incident Database Incident diagnosis + harm estimation
Session 2 · Class 16
Subgroup analysis, disparity metrics, and safeguards

Using your Week 6 classification results: compute accuracy by subgroup, disparate impact ratios, equalized odds, and chi-square tests. Then project error rates to production scale with dollar-cost harm estimation. Safeguard mapping with quantitative trigger thresholds.

Formal disparity metrics Python: subgroup analysis Quantitative harm estimation Safeguard thresholds

Prepare before class

Before Session 1:

Before Session 2:

  • Bring your Week 6 classification results/notebook - you’ll compute subgroup accuracy and disparity metrics on them

Key concepts

Concept What it means
Disparate impact ratio Error rate for one group divided by the reference group’s rate. If > 1.25x (inverse 4/5 rule), flags potential bias.
Equalized odds Whether the classifier has the same recall for each subgroup - equal accuracy across groups
Quantitative harm estimation Project error rates to production volume: X errors/month × $Y cost = $Z monthly impact
Safeguard threshold A quantitative trigger for intervention: “human review when confidence < 0.7” or “alert when subgroup error rate > 15%”

Readings and resources

  • AI Incident Database - Discover - browse real incidents here before Session 1; you’ll need one for your brief
  • LLMs Visual Cards: Bias in LLMs - how bias shows up as a statistical pattern rather than an explicit rule, and the benchmarks (BBQ, StereoSet) used to detect it

Deliverable

AI Incident Brief
~2–3 pages | 30 points | Due before Week 9's first class | Submit via Canvas

Analyze a real AI incident with failure chain, quantitative harm estimation, disparity analysis, and a recommended safeguard with a quantitative trigger threshold.

Full prompt and rubric →

Looking ahead

Next week: Governing GenAI in Organizations - translating evaluation and failure analysis into practical governance rules with quantitative risk scoring.