Survivorship Bias: The Achilles' Heel of Last Touch Attribution

Introduction

Critical thinking is one of the most important skills we can possess, and it’s one of my favorite topics to explore. Recently, I found a very interesting parallel between one of the basic biases, survivorship bias, and last-touch attribution in marketing.

Learning from History

I've come to appreciate the power of historical lessons in shaping our understanding of current challenges. A particularly striking example comes from World War II, an era that fundamentally changed how we approach problem-solving in various fields, and as you'll see it applies to modern marketing analytics too. The story of WWII airplanes and the insightful work of Abraham Wald offers a compelling case study on the pitfalls of incomplete analysis.

During the war, military engineers were tasked with improving the survivability of airplanes. They meticulously recorded where returning planes were most frequently hit by enemy fire, assuming that these were the areas most in need of reinforcement.

It was a reasonable hypothesis, grounded in the data available. However, it took the keen insight of Wald, a statistician, to identify a critical oversight: the analysis only included the planes that made it back. The ones that didn't return, presumably hit in other areas not represented in the data, were absent from their calculations. This was a classic case of survivorship bias, focusing on the survivors without accounting for the full scope of the situation.

Ironically, by reinforcing the aircraft’s wrong parts, engineers were creating heavier, less maneuverable planes that were likely to survive and make it home regardless of the additional armor. This unintended consequence demonstrated how misinterpretations could lead to suboptimal outcomes.

But how does this example of survivorship bias apply to the field of modern marketing? Could we be making similar analytical missteps when evaluating our campaigns?

Connecting Dots

Imagine a world where it’s possible to track every twist and turn in a customer’s journey to conversion, understanding the full spectrum of influences that guide them to that final decision. Ideal, isn't it? However, the stark reality we face is much more confined, often limited to the actions within a single website session. But what about the journey that led to this point? We are usually limited to attributing success to the last touch point.

After reading about survivorship bias and WWII airplanes story, does this situation sound familiar?

It should, because much like the analysts in the WWII scenario who overlooked the bullet-ridden aircraft that never made it back, we too often discount the multitude of influential touchpoints a customer has encountered before the reported "last-touch".

By increasing the budget on the channels that contribute to the final conversion, we may unknowingly overlook the other significant influences, akin to reinforcing the armor of the airplanes while the overall strength of the army diminishes. Thus, this could inadvertently lead to an ineffective marketing strategy.

Practical Implications

So, what are the channels to be aware of, especially those typically encountered beyond the consideration phase of the funnel? Understanding these touchpoints can significantly alter our approach to attributing conversions and optimizing our marketing strategies.

To be more specific, let’s consider direct traffic. When a customer arrives at our site directly, it’s challenging to pinpoint exactly what motivated their visit. Was it a memorable ad they saw days ago, an impactful social media post, or perhaps a conversation with a friend? Similarly, with paid search, there’s a high chance the customer was already poised to convert, making it a critical but not solitary factor in their decision-making process.

These scenarios highlight a critical gap in traditional last touch attribution models: they fail to acknowledge the depth and complexity of the customer journey. By focusing predominantly on these visible endpoints, we might be undervaluing the myriad of interactions that have subtly guided the customer to this decisive moment.

Let’s cast a critical eye on Google PMax campaigns and Meta’s Advantage+. How do these algorithms truly work? In terms of ROAS, they often shine, but are we evaluating them through the appropriate lens? There’s a possibility that these sophisticated algorithms excel primarily at pinpointing customers that would convert anyway. This perspective invites us to question the depth of their impact, reminiscent of the insights drawn from survivorship bias.

Is last-touch attribution any good?

After delving into the nuanced relationship between survivorship bias and last-touch attribution, it might seem as though I’m discrediting last-touch attribution entirely. But let’s not throw the baby out with the bathwater. Despite its limitations, last-touch attribution plays a crucial role in our marketing arsenal, especially in measuring the effectiveness at the bottom of the funnel.

However, relying solely on last-touch to inform all our marketing decisions is where we stumble. Its strength lies in pinpointing what led the customer to the point of conversion, offering valuable insights into the effectiveness of our final engagement tactics. Yet, this approach falls short of revealing the entire narrative of the customer’s journey. The customer might already be convinced; the last touch simply facilitates their final action.

Enhancing Your Stack

The insights drawn from Abraham Wald’s analysis during WWII extend a crucial lesson for today’s digital marketers: the importance of not overlooking seemingly insignificant channels. Each touchpoint, no matter how small it appears, plays a part in the intricate dance that leads to conversion.

While Multi-Touch Attribution (MTA) presents itself as a powerful tool to capture this complexity, its effectiveness is increasingly challenged. With its applicability narrowing to a smaller segment of customers and its reliability in decline due to privacy changes, marketers must navigate these waters with caution.

This brings us to the potential of Marketing Mix Modeling (MMM) as an alternative. MMM offers a broader perspective, enabling marketers to see beyond immediate interactions to understand the cumulative impact of their efforts. For those interested in diving deeper into the MMM versus MTA debate, check out previous post here.

Remember, there is no one-size-fits-all solution when it comes to attribution models, but rather a myriad of tools and techniques that when deployed effectively, can work in tandem to strengthen your marketing strategies. It is through a holistic approach, understanding the strengths and limitations of each tool, that we can achieve a well-rounded perspective of our marketing performance.