Measuring effectiveness: models, attribution, experiments and triangulation

Measuring effectiveness means answering one question honestly: what did our communication cause that would not have happened otherwise? No single method answers it completely.

• Brand and activation effects behave differently: one builds slowly and lasts, the other is quick and fades¹.
• Marketing mix modelling (MMM, called econometrics in much of the UK industry) estimates each activity's contribution from aggregate data over time. Google's Meridian and Meta's Robyn are open-source tools for it⁶⁸.
• Attribution assigns credit to the digital touchpoints before a conversion; it is fast but shows sequence, not cause, and privacy changes have reduced its signal¹⁰¹¹.
• Experiments (holdouts, geo tests, brand lift studies) establish cause for one question at a time¹⁴¹⁵.
• The Barcelona Principles set the standard for measuring PR and communication¹⁶¹⁷.
• The IPA's advice is to triangulate: use each method for the question it can answer, and let them correct each other³.

Les Binet and Peter Field's analyses of the IPA's databank of effectiveness cases, summarised in The Long and the Short of It, separate two kinds of effect. Brand building works on memory and preference among the whole category and produces growth that builds slowly and lasts; sales activation prompts people who are already close to buying and produces short-term sales effects that decay quickly. Their principles also hold that the two work best when integrated¹.

This matters for measurement because most fast metrics see only activation. A plan judged on this month's clicks will, over time, shift money from brand to activation. The IPA's ten-year review argues that this is what happened across the industry: short-termism kept growing, and effectiveness fell as activation's share of budgets rose². A sound measurement plan therefore includes long-term brand measures (awareness, associations, consideration, price sensitivity) alongside sales, and reviews them over a period long enough for brand effects to show.

MMM uses statistical models of aggregate data (weekly sales, spend by channel, price, distribution, seasonality, competitors, the economy) to estimate how much each factor contributed to an outcome and what would happen if the spend were changed. See the glossary entry for MMM. The IPA has published a beginner's guide to econometrics and payback measurement since 2004⁴, and its effectiveness awards guidance notes that typically just under half of recent winners used econometrics in their entries⁵. In the IPA's framework, MMM is the method that integrates the others, because it gives a view of true incrementality and its drivers and is used for budgeting and strategy³.

Two open-source tools have made MMM much more accessible:

• Meridian, from Google, was made available to all marketers and data scientists on 29 January 2025; Google describes it as a Bayesian model that measures effectiveness from aggregate data⁶. Its documentation shows how to turn the results of incrementality experiments into priors, the model's starting assumptions about each channel's return⁷.
• Robyn, from Meta Marketing Science, is an experimental, open-source MMM package⁸. Its documentation describes ridge regression for the model, Meta's Nevergrad library for searching the model's settings, Prophet for trend and seasonality, a budget allocator, and optional calibration of the model with experiment results⁹.

MMM's limits are real: it needs enough history and enough variation in spending to separate one effect from another, and it describes correlation unless it is calibrated with experiments. It is also only as integrated as its inputs; if PR, sponsorship or CRM activity is not in the data, the model will credit their effects to something else. Treat a mix model as a living tool: refresh it as new data arrives, and compare each new version with the last before acting on a change in its recommendations.

Attribution and its limits

Multi-touch attribution follows individual users across digital touchpoints and shares the credit for a conversion among them. The IPA places it where it is strongest: granular, tactical and reactive decisions such as campaign optimisation³. Its weakness is that it sees sequence, not cause. In a study of 15 large advertising experiments at Facebook, covering 500 million user-experiment observations and 1.6 billion ad impressions, Gordon and colleagues found that observational methods, even with detailed data, often failed to reproduce the effects the experiments measured¹⁰.

Privacy changes have also cut the data attribution relies on. Since 26 April 2021, apps on Apple devices must ask users for permission before tracking them across other companies' apps and websites¹¹. Google, after years of planning to phase out third-party cookies in Chrome, abandoned that plan in July 2024¹³, and in April 2025 said it would keep its current approach to cookie choice rather than introduce a new standalone prompt¹². Cookies remain, but consent rules, browser protections and app permissions mean user-level paths are incomplete, so attribution should be treated as a steering tool, not a verdict.

Incrementality and experiments

An incrementality test compares a group exposed to an activity with a comparable group that is not, and attributes the difference to the activity. See incrementality. Common designs:

• Holdouts: a random share of the audience is held back from a campaign or a channel.
• Geo experiments: whole regions are randomly assigned to test or control. Google researchers described the method in 2011 as conceptually simple and easy to interpret, and argued that standard online metrics such as clicks do not show incremental impact¹⁴.
• Conversion lift studies run by the ad platforms, using their own randomised control groups.

Brand lift

Brand lift studies apply the same logic to brand measures. Google's version shows short surveys to people who saw an ad and to a control group who were eligible but were not shown it, and reads the difference in answers on measures such as ad recall, awareness, consideration, favourability and purchase intent; it warns that a control group already exposed to the creative can hide the lift¹⁵. See brand lift.

The Barcelona Principles

For public relations and communication, the reference is AMEC's Barcelona Principles. Version 3.0, published in 2020, holds that measurement should cover outputs, outcomes and potential impact, should be holistic across all relevant online and offline channels, should be rooted in integrity and transparency, and that advertising value equivalents (AVEs) are not the value of communication¹⁶. Version 4.0, published in 2025, reorders the principles to follow AMEC's evaluation framework and puts more weight on ethics and governance, including data sourcing, the use of AI and transparent methods¹⁷. The out-takes and outcomes they ask for (what people understood, believed and did) are the same brand measures the rest of the plan needs, which is where PR measurement and marketing measurement meet.

Triangulation

The IPA's Making Effectiveness Work states that there is no silver bullet: MMM, experiments and attribution each have a role and limits, and using them separately fragments knowledge into silos³. It proposes a loop it calls MESI:

In practice: the mix model sets budgets and channel roles; experiments test the biggest assumptions and calibrate the model; attribution optimises day to day within the budgets; brand tracking and the Barcelona measures cover what the others cannot see.

1. Start from the objectives and KPIs: decide which questions measurement must answer, and by when.
2. Agree baselines and a long-term brand tracker before launch.
3. If spend and history allow, build or commission a mix model; include every discipline, not only paid media.
4. Plan at least one experiment a quarter on the biggest open question (a channel's real effect, a creative choice, a spend level), and feed the result into the model.
5. Use attribution for optimisation within channels, never as the sole basis for budgets across them.
6. Measure PR and communication by the Barcelona Principles: outputs, out-takes, outcomes and impact; no AVEs.
7. Bring the results together in one review, with one owner, so that the methods correct each other.

• Binet and Field's principles¹ and the IPA's ten-year review².
• The IPA's Making Effectiveness Work³ and Econometrics Explained⁴.
• Meridian's documentation⁷ and Robyn's⁹.
• Gordon and colleagues on experiments against observational methods¹⁰.
• AMEC's Barcelona Principles 3.0 and 4.0¹⁶¹⁷.
• On IMC Institute: objectives and KPIs, budgeting for IMC, the measurement objective and the glossary entry for attribution.