August 18, 2007

EU and US JavaScript Disabled Index numbers + Web Analytics data collection impact

I had a great chat (as always) with Eric Enge about a set of web analytics research numbers of his; where part of the dialogue kept coming back to elements of JavaScript being Enabled or Disabled. During the chat I promised that we would do a cross analysis among a larger set of visits (not visitors), for us to get a decent attitude on what the JavaScript Disabled index number should be.

The whole reason that this might be of interest is; that most users of Web Analytics enterprise systems today deploy a page tagging solution such as Omniture or IndexTools – where the data collection methodology is in part based on a small JavaScript. It is therefore very likely that there will be a discrepancy in the collected data if you have visitors who have disabled JavaScript. Before concluding on the impact of this understood discrepancy, let me present the results:

JavaScript Disabled Index numbers
EU: 1.4%
US: 3.05%

Source: 1.000.000.000 visits across multiple industry web properties using IndexTools.
(VisualRevenue.com/blog – Dennis R. Mortensen)

Where one might, at first sight, be concerned about a 2% to 3% discrepancy, due to the fact that JavaScript is not enabled. This is normal and it is very common that people confuses or transmit to much value to JavaScript disabled numbers as they either expect NO data collection at all or very limited data collection. The fact is that most (all the ones I know) of the enterprise vendors have a noscript IMAGE “fall-back” data collection methodology – meaning that all of those who have JavaScript disabled will be tracked!

It is however valid to say that the IMAGE fall-back methodology do not have the same degree of nuance (the number of metrics) as the original intended JavaScript. But all the basic information is collected and tied together in a session. You might (dependant on your Web Analytics solution) even be able to configure the fall-back methodology to track custom actions, revenue and other information.

Therefore and as example, if we say you have a cool 4% sales conversion rate on your US visitors, the discrepancy in your Revenue Metric (KPI) should not be off more than 3.05% of your 4% and thus you cannot attribute JavaScript Disabled discrepancy to more than 0.12% of your Revenue Metric. On $1,000,000 in sales, that is only $1,220 – AND some vendors even give you an opportunity to track those last $1,220 as well :-)

Conclusion:
It is very likely for us to believe that we will see a continuous decline in visitors who have JavaScript Disabled and it is very fair to trust that visitors with JavaScript Disabled have little or no impact on your web analysis efforts. However; it is a fact that data might not be 100% accurate due to this data collection defect, but the impact is much smaller than the immediate believed 2% to 3% as first assumed.

So what are your JavaScript Disabled numbers; are they close to my index numbers or… am I completely off? :-)
Comments please. (especially if you have some RAW numbers on IMAGE disabled)

N.B.
I checked my blog and by pure luck, the JavaScript Disabled number for US visitors matched the Index Number almost spot on.

Dennis R. Mortensen
17
  • S.Hamel

    Very insightful post. There is always so much worry about JavaScript or cookies being disabled, but once you understand what’s going on, once you can “make a good story” to explain it, it’s no SO bad. In fact, while it is frequent to have informational areas of a site working without JavaScript, more and more transactions requires JavaScript to be enabled. It might be important to do a thorough analysis to see if those 3% or 4% of visits are really, indeed, your target customers and if they convert.

  • Dennis R. Mortensen

    Hi Stéphane,

    See, this is a perfect add-on to the post. As I (like you) to some extent believe (or at least believe it to be researched) that those who in fact disable e.g. Java Script or other key browser functionality – might not even be part of your target audience (those who convert) – which leads my suggested example 0.12% discrepancy being a WORST CASE scenario!

    Great comment. Thanks.

    See you at Emetrics Washington!?

    Dennis :-)

  • Avinash Kaushik

    >>
    I checked my blog and by pure luck..
    >>

    Pure luck or…. some clever cooking for the numbers!!! :)

    Just kidding.

    This is a great post Dennis (insert everything Mr. Hamel said above).

    See you in a couple days.

    -Avinash.

  • Dennis R. Mortensen

    Hi Avinash,

    He he… Me and Gordon Ramsay are cooking nothing but delightful numbers :-)
    See you shortly.

    Dennis.

  • Pingback: On using JavaScript | The Theme Museum

  • Jens

    Hello Dennis,

    Do you know if there is any discrepancy introduced by the use of noscript image “fallback” solution, since these images might be requested by a number of robots? (although most common robots such as googlebots ignore them).

    Sorry to comment on an old post but I searched information about this issue and stumbled upon your blog, which is by the way very interesting.

  • http://visualrevenue.com/blog Dennis R. Mortensen

    Hi Jens,

    We maintained a list of robots which were excluded from the reporting. Not to say that a non-human couldn’t call the pixel – however, when that did happen, our customers typically just out-segmented that themselves.

    Cheers
    d. :-)

  • Craig Macpherson

    Great blog, very valuable – may I ask why you say “It is very likely for us to believe that we will see a continuous decline in visitors who have JavaScript Disabled”?

  • http://visualrevenue.com/blog Dennis R. Mortensen

    Hey Craig,

    It very much seems like, most websites today, and most certainly tomorrow are deploying JavaScript for not only extended functionality, but for basic functionality. That being the case, I personally can’t see people disabling JavaScript anymore, than they do today, as surfing the web and using your standard services becomes difficult to the point of impossible – whether that being your Internet Bank or your favorite retailer.

    Perhaps it’s time for some updated research, instead of opinion data :-)

    cheers
    d.

  • http://www.diird.vic.gov.au Brendan Halloran

    Hi Dennis,

    I have set up some noscript tracking using Google Analytics on one of the websites I work on. It picks up accurate pageview data but the session data is not accurate because there is currently no way for the CMS to insert a unique value in the appropriate parameter of the utm.gif for each user session/visit (this will change soon).

    However, when I estimate the number of noscript visits based on the overall average pageviews per visit figure, the percentage of noscript visits out of total visitation is 1.07%. The data covers the period 18 July to 28 September 2010 and the data size is quite large (in excess of 100,000 visits).

    One question for you though is about cookies and their effect on web analytics data. If people have their browser/s set up to reject first and/or third party cookies, then it doesn’t matter if you have a noscript method of tracking installed (the analytics tool you use and its cookie methodology of course plays a part here). Is there some data around to suggest what percentage of browsers are set up to reject these cookies? I am unaware of a way to track this using browser based methodologies. I suppose there might be some way to do it in your log files, but I am very wary of log file data for many well known reasons.

    Kind regards,

    Brendan.

  • http://visualrevenue.com/blog Dennis R. Mortensen

    Hi Brendan,

    Thanks a lot for adding NEW data to the conversation. Super interesting. In regards to Cookie Rejection, you should go look up the studies that ComScore has done on this. I also commented on this, from a slightly different angle:

    http://visualrevenue.com/blog/2007/06/cookies-and-their-effect-on-unique.html

    Cheers
    d. :-)

  • http://www.diird.vic.gov.au Brendan Halloran

    Hi Dennis,

    I have found it very difficult to find any figures on estimates of first party cookie rejection/blocking – I couldn’t find anything specific through ComScore, the WAA or the IAB.

    However, I have finally found an article which says:

    “It is estimated that a very low percentage of people block first party cookies, less than 5%.”

    The reference for this quote is:

    http://www.opentracker.net/en/articles/all-about-cookies-third-party.jsp

    I wonder where they get their data from and if it is still accurate?

    Brendan.

  • http://www/ marian

    Disabling JS = stuck . Nothing works. No Gmail, no Yahoo mail, Facebook looks like Win 3.1, etc. I wouldnt worry about this.

  • Pingback: A Basic HTML5 Template » SitePoint

  • Pingback: The impact of Google moving to HTTPS for logged in users? Not a lot… | RER Group

  • Pingback: HTML5 « Window 8 Blog's by manoj

  • Pingback: A Basic HTML5 Template « Nchip