The Long Tail … and how to calculate missing Revenue

Inspired by Avinash’s “long tail” post today I thought it would be appropriate to take it a step further and indicate how you as an affiliate with revenue generation based on content can calculate how much revenue you are leaving on the table – spotting a drooping tail. I know this is going to be a bit long haired, but stay with me for a second.

First lets draw our own graph (based on an anonymous dataset from the perfect IndexTools client).

Graph:The number of visits per page” – the homepage got 16241 visits in the given period and was the most popular page

This is the traditional Long Tail graph, telling us that a few pages generates a lot of traffic, BUT the same amount of traffic is generated by a large pool of lesser visited pages. Without going into detail, I think we can agree that it is fair to assume (just by looking at it) that a standard long tail distribution as the above is inversely proportional and thus follows Zipf’s law. With that said, let’s try to plot the same information into a double-logarithmic chart, to confirm that we indeed have a straight line.

I think it’s fair to say (from this perfect dataset) that we do indeed have a straight line and thus a distribution that follows Zipf’s law. However; let try to look at a less perfect dataset, another IndexTools client, who will not know how much revenue they are leaving on the table, if they are using traditional long tail linear graphs visualizations. I will have Sales call them up tommorow … :-)

Graph:The number of visits per page” – the homepage got 319270 visits in the given period and was the most popular page

And by looking at it (using a linear graph) – everything looks “normal” and I could assume that this is the traditional long tail distribution. But using a double-logarithmic chart, we all of a sudden see that there is a drooping tail.

And if we keep the assumption that the long tail should be inversely proportional and as indicated follow Zipf’s law – we are missing something here. And what we are missing is more CONTENT! – this client simply do not have enough content to support the long tail.

We could therefore say that additional content would increase Revenue. With the above as an example, the perfect distribution would roughly add an additional 1.7M visits per week, it would also mean that they had to move from 3600 something content pages to more than 300.000+ content pages.

For the fun of it, let’s assume (based on my latest Google AdSense experience) that you have an average ECPM on $2 and that each visit resulted in 5 page views. That’s a $17000 per week revenue increase!! – whether this offset the cost of creating 300.000+ pages is another debate.

But we should have in mind that this approach is not just for Content Pages – it might as well be for keywords (as Avinash used in his post), referring URLs and other metrics generating the typical long tail distribution.

Conclusion:
There is a Revenue opportunity in the dropping tail that most content sites miss out on – because they do not have optimal reporting on their data.

NB:
If you are an IndexTools client and this does not make sense – Add a comment (including your account name) – and I, ..probably outsourced to someone else. :-) will show you how to do the right exports and Excel setup’s to see if YOU are missing out and whether you have a drooping tale opportunity.

.

  • Avinash Kaushik

    Dennis: Are you sure you want to be a COO and not a top notch ten-thousand-dollars-an-hour-analytics-god? :)

    This is a brilliant extension for the long tail and a much richer and actionable recommendation. Thank you for doing that.

    Ok here is a free idea for you:

    1) Take your top 50 clients.
    2) Apply this (and more of your insights) to their data.
    3) Tell them that you will charge them 10% for every $1 increase in revenue.
    4) Set up a swiss bank account to keep all the money that will surely pour in!!! :)

    Seriously though one thing often missing from Vendors is a serious Professional Services offering (and I don’t mean “let us come and help you install the application and set up reports” thing). Maybe there is something here with you in charge.

    -Avinash.
    PS: Of course if you do execute a “serious professional services” strategy then you owe me 10% of all the revenue increases!!! :) Just kidding!

  • Dennis R. Mortensen

    Thank you for the kind words Avinash. Setting up the dual-bank-account right now :-)

    – but seriously, I think you are 100% right when you say that the industry sometime focus too much on the tool and too little on REAL Professional Services.

    And I will look forward to repeat it at this year’s Emetrics; that it does not matter whether you deploy Omniture, CoreMetrics or IndexTools – it’s all the same and the only difference is that Omniture has a green interface. The tools have become basic ecommerce commodities where the positive business impact is based on the Analyst – NOT the tool.

    Choosing between:

    A) GA + Avinash
    B) HBX + A Junior

    I would go for solution A) – I would never go for the “better” tool, but definitely for the better Analyst! – Or solution C), IndexTools+Dennis ;-). Everybody is in Sales.. ha ha

    Cheers

  • michaelshearer

    At risk of looking dumb, how about a quick overview of how you created the double-logarithmic chart?

    Thanks!

    michael shearer

    michael@netexperienced.com

  • Dennis R. Mortensen

    Hi Michael,

    >>At risk of looking dumb, how about a quick overview of how you created the double-logarithmic chart?

    I still believe the dumbest people out there are those who are too proud to ask a question afraid of losing face. So here goes my best try at explaining it.

    A) Go to your “Most popular pages” – Report. (this typically show the “Title” and metrics like “#Visits”
    B) Export ALL data for the given period as CSV – not just the TOP 100, That the reporting interface might show you.
    C) Import ALL data into EXCEL and remove all columns but “TITLE” and “#VISITS”
    D) Then MARK your “#VISITS” column (e.g. all 800 ROWS, this should equal the number of pages you have on the site analyzed)
    E) Create a Linear Graph in Excel based on the above data. THIS will give you the standard long tail graph as we know it.
    – moving to double-logarithmic chart
    F) Change chart format to X Y (Scatter)
    G) Then FORMAT X AXIS and choose “logarithmic scale”
    H) Then FORMAT Y AXIS and choose “logarithmic scale”
    I) That’s it.. (you can draw a straight line to get a better view of the potentially drooping tail

    Sorry for the CAPITAL’s letters, but I simply don’t know how to bold in Blogger comments yet :-)

    A screenshot of the result is here (one where we have a drooping tail):
    http://www.visualrevenue.com/blog/images/double-logarithmic-chart.jpg

    Did this make sense at all? .. :-)

  • michaelshearer

    The only thing that doesn’t make sense is “mark your visits column”. Otherwise, I will try it out today. Thanks so much.

    To bold, put the html tag for bold at the beginning and end of word/sentence.

    See here.

    Hope that helps.

  • Dennis R. Mortensen

    You are right, I should have tested basic HTML tags :)

    Cheers

  • Daniel R

    Dennis,

    Excellent follow-post to Avinash! I’ve thought about how to best graph the long tail from time to time, but unfortunately (unlike you) my statistics and math learnings from college elluded me.

    With ideas like that, its great to know IndexTools is our analytics partner. :)

    Cheers,

    Daniel

  • Dennis R. Mortensen

    Thanks Daniel…

    I will surely let our Partner folks here at IndexTools know about that.

    You are based out of San Francisco right?

    If so; come by and say hi at AdTech or Emetrics which I (IndexTools) am attending..

    :)

    Cheers

  • Jens

    Dennis,

    you are absolutely genious!
    I tried this double-logarithmic chart on data of one of our customers and here we go: a straight line!

    This was just a first try for me, but a great way to show the customer where he´s gotta lot of work to do.. perfect!

    Once again, i learned a lot! Thanks for that!

    Jens from booming

  • Dennis R. Mortensen

    Genius is a big word :-)
    thanks a lot Jens.

    Cheers mate

  • Anonymous

    Martin, I am afraid this is a nice idea that was posted by Jakob Nielsen on August 14, 2006.

    http://www.useit.com/alertbox/visualizing-traffic-analysis.html

  • Dennis R. Mortensen

    Probably one of my favorite blogs and I suggest everybody working in Internet Marketing go read it (even though Jakob is very firm about NOT calling it a blog)

    ..and we Danes should stick together! :-)

    Cheers
    Dennis

  • Pingback: VisualRevenue | Indentify increased organic search traffic opportunities in a drooping tail (speaking at SMX)

  • Pingback: VisualRevenue | Do you have a “Fat head”? (Long tail)

  • Pingback: VisualRevenue | Use Web Analytics to determine the width of your Internal Search Query box

  • Pingback: VisualRevenue | 9 Ways to Make Money on Analytics

  • Steve P

    Hi there,

    I’m learning about this long tail and log scales and Zipf distributions. The hardest part for me getting that red trendline to show up in Excel.
    I’m assuming it is not a trend line but a perfect Zipf distribution.
    Can you explain how to create the perfect fit line in Excel?

    Many thanks,
    Steve

  • http://visualrevenue.com/blog Dennis R. Mortensen

    Hey Steve,

    You can do something as simple as apply a line to your Chart Manually (from e.g. x=10,000 to Y=10,000). Which is what I did in one of the above examples and what you call the perfect distribution.

    OR you can apply a POWER trend-line. But dependent on your first data points this is not always what you want.

    Cheers :-)
    Dennis