JavaScript analytics / conversion script: expected failure rate?

0
votes

Question for developers who work with 3rd-party analytics tools:

is there an industry-standard or expected failure rate with JavaScript tracking?

Scenario: I have a one-page website. I install Google Analytics, Mixpanel, and Heap 3rd-party analytics JavaScript tracking. The page loads clean and error-free. I use Adwords to buy 100 clicks to my site.

Now, according to raw server logs, I receive all 100 visitors. However, my Analytics dashboards report:

  • GA: 97 unique visitors
  • MixPanel: 96 unique visitors
  • Heap: 99 unique visitors

Report latency isn't an issue (I've waited 48 hours). I don't want to quibble about which analytics tool's definition of a "unique visitor" is best.

What I'm trying to get to the bottom of is this: is there an anticipated error bar I should apply globally to any/all analytics reports? Say that each script loads properly 95% - 99% of the time? (That way I can ignore mismatching numbers so long as they fall into this expected error bar and focus on true outliers.) Additionally, if there's an expected failure rate, I can have greater confidence that, despite the mismatched numbers above, my scripts are reporting properly and save my IT team a lot tail-chasing.

File under Anecdotes Not Data: A colleague told me his ecommerce site uses a hosted, JavaScript-based, enterprise-level conversion tracking platform. Based on 400-500 transactions per day, his analytics under-reports conversions consistently by 4-5%. He has several years of data documenting this (99.9% confidence).

What I don't know is, does this hold true globally? Do everyone's analytics scripts misfire, fail to load, or otherwise go CLICK instead of BANG 4-5% of the time?

Here are potential issues I AM aware of:

  • Script errors
  • Script conflicts
  • Timeouts when pulling from a third-party server
  • User bounces before scripts complete loading

**Not to get all chemtrails on you but: **IF there's an expected fail rate, it's certainly not common knowledge. Nobody I've spoken to at any analytics companies admit to consistent failure. Neither do they guarantee 100% accuracy.

So I ask: in your experience, what's the expected accuracy rate of your JavaScript-based, hosted analytics platforms?

1
javascriptgoogle-analyticse-commerce
I don't feel I can give you an answer to this. I am a javascript developer, however. Your 4 bullet points are exactly your points of failure AFAIK and there is nothing you can do about them. "Script errors" should be very infrequent if at all, provided the vendor of the analytics does their job properly. That being said, of course they're not ever going to admit consistent failure or guarantee 100% accuracy, for the same reasons that those 4 points are beyond your control; they are beyond their control. In summary: Yes they all fail. They have to. it's the nature of javascript. - gforce301
Analytics generally depends on tracking cookies, and visitors can disable them. - Barmar
As well as the mentioned points there's also various ad-blockers which will disable JavaScript based analytics tools. For example if you are browsing in Firefox's inPrivate mode Google Analytics will be disabled. In general if you're within 5% of the true number of visitors/transactions then you can be reasonably confident there's nothing wrong with the code. - Joe Law
I'm voting to close this question as off-topic because its not programming related may be better suited for webapps.stackexchange.com - DaImTo
I appreciate your responses! - George Tucker

1 Answers

0
votes

I think your fourth bullet may be the most revealing. Paid media will have a very high bounce rate compared to organic/referrer/direct traffic.

What is the bounce rate reported by those various tracking tools? I would wager it is at least 80% which means the chances of users exiting prior to the scripts loading is high. You could correlate that with your page load time.

One thing you could try (since my expertise is with Google Analytics) is to use the Measurement Protocol to send pageview data on the server-side. Since that no longer requires JavaScript to load, it takes page load time and bounces out of the equation. I would not recommend using this method for production, but it could illuminate the issue.

To summarize, I think your issue is with load abandonment and not necessarily failure of the various tools.