Week 5: Analytic Design Critique + Implementation
3 components | Due before Week 6's first class | Submit via Canvas
Assignment
Use AI as a design partner, then implement with formal model selection and validation.
Component 1: Design document (1–2 pages, submit as PDF)
Four sections, in this order:
- Business question - the question your analysis will answer, framed for a business audience
- AI-proposed plan - prompt an AI tool as a helpful analyst; include its proposed approach
- Adversarial critique - prompt an AI tool (the same one or a different one) as an adversarial reviewer attacking the plan; include its critique
- Your reconciliation - what you kept, changed, or rejected from each round, and why
Component 2: Python implementation (choose one track)
Submit as a single Jupyter notebook (.ipynb) rendered as .html that runs top to bottom without errors. Use markdown cells to label each required piece (model selection, validation, etc.) so it’s easy to find.
Segmentation track: RFM features from a customers dataset, outlier analysis, train/test split, model selection with three metrics (elbow, silhouette, gap), justified k, holdout validation, ANOVA, cluster profiles.
LMM track: 4+ specifications, AIC/BIC comparison table, justified final model, residual diagnostics, ICC, effect sizes with 95% confidence intervals.
Component 3: AI use log
See the AI use log guide.
Rubric
| Criterion | Excellent (5) | Adequate (3) | Needs revision (1) |
|---|---|---|---|
| Design process | All four roles, substantive reconciliation | Some roles | No design process |
| Model selection | 3+ metrics or AIC/BIC with justified choice | One metric | No formal selection |
| Validation | Holdout or diagnostics | Some validation | None |
| Statistical rigor | ANOVA/CI/effect sizes, assumptions checked | Some statistics | Descriptive only |
| Interpretation | Named clusters or interpreted effects in context | Some interpretation | None |
| AI use log | Detailed | Present but incomplete | Missing |
Total: 30 points