Level 4 - Cross-Validation & Reality Checks
Level 4 - Cross-Validation & Reality Checks
Section titled “Level 4 - Cross-Validation & Reality Checks”Goal: Learn to triangulate data across systems (intermediate/advanced).
Never trust one system. Always cross-validate.
Overview
Section titled “Overview”This level teaches you to compare data across multiple systems to find truth.
You’ll learn:
- The 3-system cross-check (ad platform vs attribution vs billing)
- Why Facebook undercounts (consent gaps)
- Why 5-20% unattributed sales is normal
- Daily/weekly reconciliation routines
By the end, you’ll know: “Facebook shows 80, my system shows 100, billing shows 95—so reality is 95, and FB has a 16% consent gap.”
Modules
Section titled “Modules”Module 4.1 - Three-System Cross-Check
Section titled “Module 4.1 - Three-System Cross-Check”Lessons:
- Lesson 4.1.1: Why Facebook under-counts, why billing is truth
- Lesson 4.1.2: Typical pattern: FB 80, your system 100, billing 95
- Lesson 4.1.3: Exercise - Pull numbers from all 3 systems and compute gaps
Based on: Encyclopedia - Cross-Validation (The Three-System Cross-Check)
Key exercise: For one day or week, get conversion numbers from Facebook Ads, your attribution system, and billing. Compare and explain the gaps.
💡 See Real Example: Moon attribution queries showing sales by traffic source →
Module 4.2 - Consent Gaps & Unattributed Sales
Section titled “Module 4.2 - Consent Gaps & Unattributed Sales”Lessons:
- Lesson 4.2.1: 5-20% unattributed is normal; >30% is a red flag
- Lesson 4.2.2: How to compute consent gap and adjust ROAS
- Lesson 4.2.3: Exercise - Calculate unattributed % for a recent week
Based on: Encyclopedia - Cross-Validation (Facebook Consent Gap, Unattributed Sales)
Key skill: Adjust expectations based on known gaps. Don’t panic when Facebook shows lower numbers.
Module 4.3 - Weekly Reconciliation Routine
Section titled “Module 4.3 - Weekly Reconciliation Routine”Lessons:
- Lesson 4.3.1: Checklist for weekly ops
- Lesson 4.3.2: Exercise - Perform one full reconciliation
Based on: Encyclopedia - Cross-Validation (Reconciliation Best Practices)
Key deliverable: Create your own weekly reconciliation checklist.
Key Concepts You’ll Learn
Section titled “Key Concepts You’ll Learn”The 3 Systems:
- Ad Platforms (Facebook, Google): Undercount due to consent (50-80% of actual)
- Attribution/Analytics (PostHog, Moon): First-party tracking, most accurate for attribution
- Billing (Stripe, payment processor): Ultimate truth for revenue
Facebook Consent Gap:
- iOS ATT: ~60-70% of iOS users decline tracking
- GDPR: Many EU users decline cookies
- Ad blockers: Block Facebook pixel
- Result: Facebook sees 50-80% of actual conversions
- Don’t panic and stop profitable campaigns
Unattributed Sales:
- 5-20%: Normal (direct sales, cookie clearing, device switching)
- 20-30%: High but manageable
-
30%: Tracking is broken, investigate
Daily Workflow:
- Morning: Check FB vs PostHog (campaign decisions)
- Weekly: Check PostHog vs Billing (validate tracking)
- Spot-check 5-10 events daily (verify parameters)
Level 4 Complete
Section titled “Level 4 Complete”After this level, you’ll be able to:
- Cross-validate data across ad platforms, attribution, and billing
- Calculate and adjust for consent gaps
- Identify when unattributed % is normal vs broken
- Perform weekly reconciliation to catch issues early
You now make decisions based on triangulated data, not single systems.
Next Steps
Section titled “Next Steps”👉 Continue to Level 5: Level 5 - Advanced Hunting & Diagnosis
Now you revisit earlier topics but go much deeper.