Campaign Attribution is Broken

I recently attended the Conversion Conference in London, thanks to a free ticket from Conversion Thursday.  It was an interesting event although my attendance at sessions was impacted by a request I received at the start of the first morning.  eMetrics was happening at the same time and was suffering from a few cancellations – I was asked if I could take a session.  Some frantic thinking later and I realised this was an opportunity to put together my thoughts on Campaign Attribution.

It was a topic I have wanted to explore for some time, building on discussions from my former employer Matthew Tod and extending with my own experiences and thoughts since then.  I disagree with the current approach using Campaign Attribution models (last click, first click, weighted attribution, etc) as I believe the data it is possible to collect will always be incomplete – leading to incorrect interpretations & decisions.

This presentation explains why Campaign Attribution is broken and then goes on to explore the real business problems/questions and suggest alternative approaches which can work – even if very difficult.  Following Matthew’s description of Last Click attribution as the goal scorer in football, all points are made using football as an analogy…

I expect a large degree of disagreement with my thoughts here and look forward to these discussions.  I managed to get a reasonable level of agreement at eMetrics on the issues raised, although this was followed by “I will continue using campaign attribution” (which confused me).  Please have a read through and let me know what you think.


Share and Enjoy:
  • Twitter
  • Digg
  • StumbleUpon
  • del.icio.us
  • LinkedIn
  • Facebook
  • Yahoo! Buzz
  • Google Bookmarks

Tags:

6 Comments

  1. Stephane Hamel December 17, 2012 at 12:34 am #

    I think your article provides a good explanation of what is attribution and what is the challenge. I feel you are not wrong, but not absolutely right either.

    It all comes back to your point: what is success? But I would add it also comes back to what you can control, manage and influence. If you are at a very tactical level and the only thing you can do is manage AdWords campaign, then the attribution you will look at will likely be last click and very narrow – but even then, you SHOULD gradually look at other channels and the impact of your PPC spending on the overall marketing performance.

    If you are the online marketing manager, there is absolutely no excuse to think last click is the only and best approach… you HAVE to look at the online marketing mix – even if you can’t make up for offline attribution.

    And lastly, if you are the marketing manager responsible for both online and offline marketing, there is no excuse not to look at the bottom line impact across all channels.

    May I say it depends on your “maturity”?

    The problem in your example is you are trying to attribute a single goal to an exact sequence of events. Taking a step back and looking at the performance of players throughout a whole season might give you a much better idea (and how much you pay each player vs his performance). Digital analytics isn’t about perfect data, it’s not about complete data, and it’s not about an exact answer – but it is certainly capable of providing good insight and guide decisions of smart managers.

    • Peter O'Neill December 17, 2012 at 3:18 pm #

      Hi Stephane – thanks for sharing your thoughts. I think we are actually closer in thoughts than it has come across, one of the perils of only posting a presentation. I need to follow up with a post (will likely be three) explaining my thinking on this topic. Hopefully something I can accomplish in the next month.

      I agree you have to look at the online marketing mix but I think you should be doing this by defining the objective for each campaign – where these objectives vary. And you can then evaluate all of these based on behaviour during that visit or through member analytics (if log in).

      I agree your “maturity” will impact the level of analysis you can perform. But while I agree analysis is not about perfect data or exact answers, incomplete data can lead to wrong answers. As a very simplistic example, your data could show that 80% of purchases were completed within a single visit. Due to multiple devices, the reality actually be 20% – and no technology would solve this issue. The problem is the strategy you would take for 80% single visit purchases is very different to 20% single visit purchases – that’s why I say we need to work on alternative approaches than campaign attribution models.

      Looking forward to continuing this discussion…

      • Andrew Strickland January 8, 2013 at 11:58 pm #

        Hi Pete,

        Guess we should probably rename the Gary Lineker model to a player that didn’t retire in 1994 (I will have a think!)

        I personally don’t think attribution is pointless although I think we are all in agreement it is flawed due to the reasons your presentation highlights.

        In Stephane’s example above, for channels that we consider acquirers such as PPC Generic, they have to be evaluated beyond last click otherwise from a ROI perspective the decision would always be to reduce spend. So if different credit allocation models show that an AdWords campaign (or ad Group or Keyword) is actually performing well then that insight is useful. Arguably the multiple device issue actually underplays the variance between model X and last click.

        The above example does not take a long time to analyse which leads onto a big issue in my opinion where companies want to invest lots of internal resource into attribution when:

        1) the existing marketing tracking is not sound
        2) there is little thought about what they are going to do based on the outcomes of the analysis e.g. testing (as per your slide)
        3) people who own an individual channel have a large % of their remuneration based on the performance of their individual channel
        4) they could be optimising their web site and giving all of their digital channels a boost regardless of credit allocation.
        5) The agency is paid based on a proportion of media spend as opposed to specific targets

        Plus a load of other examples…….I think your ending to your above reply is apt!

        • Peter O'Neill January 9, 2013 at 11:20 am #

          Thanks for the comment Andrew. In some ways I am a traditionalist – it will always be the Gary Lineker model to me…

          My main concern is that while channels such as PPC Generic need to be evaluated beyond a simplistic last click convert vs didn’t convert model. But if people use a different device to convert, every other attribution model will just be as flawed as last click. So I think people should be focusing on what the alternative may be.

          Agree with your points on other issues a company should be thinking about as well – beyond just buying the most expensive attribution tool / agency out there. This discussion will definitely be continued at MeasureCamp.

  2. Tim Wilson December 18, 2012 at 12:00 am #

    Great post (and great points by Stephane).

    I wish I could remember who made a point to me once that attribution tends to boil down to how you value the different steps in the process (this is separate from the fact that, Google’s claiming “Universal Analytics” notwithstanding, there are ton of influencing interactions that are not captured). Even pure play attribution management platforms like Clearsaleing, at the end of the day, just give you the flexibility to set up whatever you “want” for your attribution model.

    It’s always seemed like a circular expectation that, if you capture enough touchpoints, that you will be able to use some sort of analytics to capture the contribution of each individual touchpoint. The sum of the parts are greater than the whole — interaction effects and whatnot — so attribution is trying to assign a value per-touchpoint…when the combination of touchpoints should increase the total value.

    At the same time, isn’t this what media mix modeling has been “doing” for years? Granted, that’s at a much higher level (and was in simpler times, I would claim…and have claimed in the past; yes, of course I wrote a post about it — but I’m not going to logroll here!). But, it’s attribution management that marketing execs loved. And, it added value.

    But, attribution is something, in my experience, that marketers jump at as a Holy Grail where “the data will tell us the answer,” when they’d be better served focusing on how they think their business works and then analyzing and testing to see if that’s the case. That should absolutely include cross-channel work. But, if the goal is to “totally solve the marketing mix with math,” their goal is a pipe dream.

Trackbacks/Pingbacks

  1. Resumen Semanal Analítica Digital (IV) | Roberto Ballester - December 21, 2012

    [...] clic, distribución equitativa..) está en boca de todos y estará en el próximo año, Peter O’neill nos deja una presentación sobre ello y  RuthBurr otra en SEOMOZ. About Roberto Ballester OrtegaLicenciado en Publicidad y Relaciones [...]

Post Comment

Notify me of followup comments via e-mail. You can also subscribe without commenting.