Your Marketing Models Can Predict Everything Except What Happens When You Act

I am responsible for marketing analytics at a Series C fintech, and here is a rather embarrassing admission: our industry tends to produce models which are very good at answering questions nobody asked, while being structurally unable to address the single question everyone asks.

This question is something along the lines of "what happens if we do X?", where X stands for several possible actions: "What happens if we offer this segment a discount?", "What happens if we reduce retargeting budget by 30%?" or "What happens if we add one more email touch?"

Instead, our models, including lifetime value predictions, return on ad spend forecasts, and even multi-touch attribution, answer a completely different question: "What happened to people who resembled this?" It is a prediction. We need an intervention. They are not the same, and the gap between them leads to millions of marketing budgets being wasted quietly every year.

Two separate camps

The field of marketing analytics can today be divided into two distinct communities that rarely refer to each other.

First camp: probabilistic customer models. Traditional examples include BG/NBD, Pareto/NBD, and Gamma-Gamma models, which constitute the "buy till you die" family of models. These models assume the customer to be a small stochastic machine with several parameters: frequency (how often he buys), monetary (how much he spends per transaction), and dropout (silent churn probability). They are beautiful, easily interpretable, and have proven themselves over two decades of use. Give them your transaction data and get the customer expected value of the future.

What they cannot do is intervene. They just describe the world as it is. You launch a promo campaign – the BTYD model readily incorporates that new information post hoc, but it has no understanding of "how the metrics would have been different without the promo."

No levers to pull there.

The camp number two – causal machine learning. Uplift modeling, double machine learning, causal forests, T and X learners. They answer questions about interventions. They try to calculate the impact of a particular treatment: how much did the campaign contribute to the growth of the metric, beyond the natural one.

They regard the outcome as a plain numeric scalar. 30-days revenue. Conversion flag, yes or no. Completely unaware of the internal logic of customer behavior. The causal forest will tell you what discount added five dollars per client. But the forest has no idea how it added five dollars: increased frequency of purchases, higher average order size or enhanced loyalty. And the understanding of such mechanism is crucial for marketing.

A Possible Hybrid Approach: Use the Do-Operator to Model Intervention Effects

Now that both approaches are clear, it becomes evident that it is reasonable to unite them. But not via adding causality to prediction (it is what everybody does nowadays and, hence, it inherits all the flat-scalar problems), but via including the intervention itself into the parameters of the model.

Specifically: in a Gamma-Gamma / BG/NBD framework, customer behavior is determined by parameters such as λ (frequency of purchase), ν (average value per transaction), p (probability of customer dropout), etc.

In our hybrid approach, the parameters become functions of the intervention:

where x_i are customer covariates, do (T) is Pearl’s do-operator (a formal representation of an intervention rather than observation). The target quantity is interventional CLTV, while the value of the campaign is the difference between two interventional regimes, calculated over the entire horizon.

What a Hybrid Approach Offers You that Either Option Doesn't

1. Incrementality at the mechanism level. Where one side says "the discount gave us 14% increase in revenue", the other says "the discount increased purchase frequency by 30%, reduced basket size by 12%, and had no effect on retention". In my simulation where there is known ground truth, a randomized discount with exactly that structure looks like an innocuous +14% to any scalar uplift model, while the parameter-level approach identifies all three effects separately, within confidence intervals. Those are three different business decisions hidden in one number.

2. Safety from confounded targeting. This is the ugly secret behind the extension of BTYD-type models by covariates: marketing exposure is not randomly assigned in reality. Promotions are sent to more engaged users; retargeting targets high-intent users. If you use marketing exposure as a covariate in a BTYD model, reading the coefficient of the regression as an effect is an egregious causal mistake. In my simulations, where there was realistic targeting (the probability of promotion increasing with engagement), the naive estimate of promotional effect on purchase probability was 4.4 times larger than the truth. However, using a Double ML-style estimator, which partially controls the confound and uses flexible models for nuisance functions, recovered the truth perfectly. Same data. Same model class. Different identification.

3. Effects of interventions on customer life cycle trajectories. Any intervention that has a component of retention effect works multiplicatively. The incremental customer life cycle curve keeps growing for 24 months, because it retains its customers who keep buying. A two-month A/B test underestimates that effect systematically. Viewing CLV as an interventional quantity makes the problem visible.

4. Causal Shapley-values. Shapley-value attribution systems have taken the marketing industry by storm (I've implemented such systems myself). The standard way of attributing correlation is what's used in all those systems. Retargeting always looks like a heroic effort in all the attribution reports precisely because it is shown to the users who would buy anyway.

However, if you define the Shapley value function based on interventional quantities, you will be attributing incrementality:

In my world simulation where there was selection into retargeting and therefore a bias in the observations of that channel, the Shapley value attributed twice the importance to the retargeting channel than it really had interventionally.

Isn't this just Double Machine Learning?

Not at all, and this is an important difference. DML is an estimation method, whereby you orthogonalize your estimation of the treatment effect to flexible nuisance functions to avoid bias.

But the usual DML pipeline computes an effect on the outcome variable. In our hybrid approach, we compute an effect on the latent parameters of a structured probabilistic model and propagate those effects to trajectory-level metrics such as CLV. DML is the engine; the behavioral model is the vehicle.

The closest prior art is the extension of Pareto/NBD models with covariates (Abe 2009; Bachmann, Meierer and Näf 2021, implemented in the CLVTools R package), where model parameters may depend on marketing covariates. This is real machinery that works. What this machinery lacks is the causal layer: no confounding correction, no overlap assumption, no identification claim. These coefficients are mere associations that pretend to be effects. Turning that machinery into causal machinery is, to my knowledge, an unexplored space after a literature survey.

This would look like in practice

To get started moving towards this approach before the academic research catches up, the practical steps to take would be:

Model the behavioral process, not just the outcome.Run the BTYD-type model to get frequency, monetary and drop out parameters per segment. Without the causality step, even an educated guess based on segmentation will be better than just revenue.
Think of targeting as the opponent of measuring.Record your targeting strategy. Estimate propensities. Check for overlap. Some segments might have been targeted always, and thus the counterfactual won't be estimable. No model will help.
Orthogonalize. Apply the DML-style residualization when estimating how your interventions impact the behavior of the customer. Correct parametric specification is too much to ask for; orthogonalization allows you to relax the assumption.
Report your results per parameter.Frequency, basket, retention. Three numbers instead of one; and your growth team will take other decisions because of that.
Test against available experiments.Use holdout geo's, staggered rolls, ghosting ads. The observational model has to confirm the experimental benchmarks before it can scale.

The bigger picture

In the 2010s, marketing analytics developed the skills to predict; and in the early 2020s we are developing the skills to measure incrementality. The skill trees remain distinct. The next level of toolset development, in my belief, has to be hybrid: probabilistic behavioral models with interventions explicitly incorporated, identification arguments instead of vibes, and attribution based on do-quantities instead of observational journeys.

Prediction tells you where the world is going. Intervention tells you what you can do about it. It is time our models learn the difference.