Investigate a sudden metric spike or drop

Systematically diagnose why a metric suddenly changed, ranking likely causes and giving the exact checks to confirm each one.

LLazyPrompt(@lazyprompt)Data & AnalysisAnalyzeWorks with any model
No ratings yetSign in to rate
Save 0 Fork Report
Prompt template
A metric changed unexpectedly and I need to find out why. Act as a methodical data investigator.

Metric: {{metric}}
What happened: {{change}}
When it started: {{timing}}
What I know about the business, product, or system around that time: {{context}}
Dimensions I can break the data by: {{dimensions}}

Produce an investigation plan:
1. Restate the anomaly precisely: size, shape (step, spike, slow drift), and timing, and what it should be compared against (same weekday last week, same period last year).
2. First, rule out measurement problems: tracking or pipeline changes, delayed or duplicated data, timezone or definition changes, bots. Give the checks in order of cheapness.
3. Then list 6-8 candidate real causes, ranked by prior likelihood given my context. For each: the evidence that would support it, the evidence that would contradict it, and the exact breakdown or query to run.
4. A decomposition approach: split the metric into its components (for example volume x rate, or by segment) and show how to find which component explains most of the change.
5. A decision tree: "If X is true, go to Y."
6. What to tell stakeholders right now while the investigation is ongoing, in 3 sentences, without claiming a cause I have not confirmed.
7. Follow-up: an alert or monitor to catch this sooner next time.

Say what extra information would most narrow down the cause.
Example output
**1. The anomaly**: A step change (not a spike or drift), about -35%, starting Tuesday morning and persisting five days. Compare with the same weekdays of the previous three weeks, since signups are likely weekly-seasonal. **2. Measurement checks (cheapest first)** - Is the signup event still firing on the new form? Compare front-end events with database user creation counts. - Did the dashboard filter or metric definition change? - Did bot filtering or duplicate suppression change in Monday's release? **3. Candidate causes** 1. Redesigned form broken on a browser or device. Supports: drop concentrated in one segment. Contradicts: uniform drop. Check: signups by browser and device, before vs after. 2. Added friction in the new form. Check: funnel completion by step; a sharp drop at one field points to it. 3. Paused ad campaign. Check: signups by traffic source; if the lost volume matches that campaign's old volume, it explains the drop. 4. Tracking missing on the new form (measurement, not real). Check: database counts vs analytics. 5. Country-specific issue, such as a payment or validation rule. Check: signups by country. 6. Seasonality or a holiday. Check: same period last year. **4. Decomposition**: signups = visits x visit-to-signup rate. If visits are flat, the rate fell; segment the rate by device and funnel step. **5. Decision tree**: Visits down -> traffic source breakdown. Visits flat, rate down everywhere -> form change. Rate down in one browser -> bug. Database flat but analytics down -> tracking. **6. Stakeholder note**: "Signups are down about a third since Tuesday. We are checking whether it is a tracking issue, a form problem, or lower traffic. We will update you by end of day." **7. Follow-up**: alert when daily signups fall more than 20% below the same-weekday 4-week average. **Most useful extra info**: signup counts from the database vs analytics, split by device and browser.

Why it works

Ruling out measurement problems before hunting for real causes avoids the most common waste of time. Add a screenshot description of the trend or sample numbers to sharpen the ranking.

Licensed CC0 (public domain) Version 1 · history