Time-Varying vs. Nonlinear Effects
Think about studying for an exam:
- Nonlinear Effects: Studying for one hour improves your score significantly, but studying for 10 hours doesn’t make you 10 times smarter (just like how doubling ad spend won’t double sales forever).
- Time-Varying Effects: Some days, your brain absorbs information better, while other days, you’re too tired or distracted (just like how ad effectiveness fluctuates).
In your data, these two can look the same. If sales slow as you spend more, is it because of diminishing returns? Or is ad effectiveness dropping over time? Each explanation leads to a different game plan:
- A nonlinear model might say, “Stop increasing spend—you’ve hit the limit.”
- A time-varying model might say, “Shift spending to when ads work best.”
Why Models Get Confused
- Autocorrelated Spending: Companies tend to adjust budgets slowly (e.g., adding 10% per month). That smooth pattern makes it hard to tell if results are due to nonlinearity or shifting effectiveness.
- AdStock Variables: MMMs use AdStock to track lingering ad effects (e.g., last month’s TV ad still impacts sales) but smoothing past spending makes it harder to isolate true nonlinear vs. time-varying dynamics.
- Noisy Data: Real-world sales data is messy. Even sophisticated models struggle to separate real patterns from random fluctuations.
The Cost of Misreading the Data
Making the wrong call can be expensive:
- Example 1: A company thinks ads have diminishing returns (nonlinear) and cuts spending. But if effectiveness was actually dropping over time, they missed chances to reallocate budgets to better-performing periods.
- Example 2: A company assumes ad effectiveness is seasonal (time-varying) but ignores diminishing returns. They overspend in peak seasons, wasting money on saturated channels.
Studies show that confusing these models can lead to 20–40% errors in spending decisions. For a $1M budget, that’s up to $400K wasted.
How to Fix It: Test, Don’t Guess
Instead of blindly trusting the model, validate it with experiments:
- The Seesaw Test: Switch ad spending between high and low levels—say, $10K one week, then $50K the next. If sales respond predictably (always peaking at $50K), it’s likely a nonlinear effect. If results vary unpredictably, time-varying effects are at play.
- Maximal Separation: Run the model for extreme spending levels (e.g., $0 vs. $100K). The bigger the difference in predictions, the easier it is to pinpoint the real driver.
These tests work like stress tests for your model, helping you avoid costly mistakes.
Bottom Line
Time-varying models are useful but easily misinterpreted. Always back them up with real-world tests to confirm if you’re seeing true shifts in effectiveness or nonlinear saturation.
Inspired by Ryan Dew et al.’s research on MMM conflation. For deeper insights, explore their paper [here].
