Guide To Website Statistics

Statistics applications - or stats, metrics, logs, access logs, error logs, stats packages, statistical analysis programs or web analytics software: these are all names for the important sources of information available in your website backend or site control panel, and the methods of viewing and analysing them.

The information here includes your traffic figures, sales figures if the site is an online store, keyword performance, site errors, best pages, worst pages, entry pages, referring sites - and a hundred other things you’ll want to know if your business has any form of online presence at all.

The trick is to get what you need to know, from what is often a complex and obscure interface - web metrics take a lot of practice before you are familiar with them.

Most web hosts install one or more free stats packages for you. These are impressive, considering they are free, and are often sufficient for basic analysis, especially for sites with low traffic and that do not sell online in any form. These free server stats packages are compared below.

Why you need web analytics

There are two reasons: commercial reality, and ease of interpretation.

The website statistics are the most important tool at your disposal for improving website performance. In fact you probably can't get within a 50% level of site performance if you ignore the stats.

In effect your stats tools will be used to sculpt your website - it's as simple as that. The website's statistics are a normal webmaster resource, and increased knowledge of SEO will help you make better use of them.

The second reason is, simply, ease of interpretation - all the data is there in your site logs but in essence it is of no use in that format. There are some circumstances in which it is necessary to read the raw logs, but most of the time what you need is the data analysed and presented optimally. This is where your stats application comes in. The way they analyse and present the data is what defines the difference between the applications - they all use the same data.

Both presentation and analysis are important. One might think that superior analysis would be a better feature - but unless the information is presented in an easy to digest format it doesn't really matter how good an application is at analysing the data.

In addition there are a lot of dead ends a web analytics solution can go down. Several of them have not changed since the 90s, and if you remember the software available and the state of the Internet at that time, you'll know just how much everything else has moved on. These stats apps haven't.

There are many things a statistics application can interpret that are a complete and utter waste of time for normal website performance tuning. Because there are many highly inefficient webstats solutions that prioritise for them, people naturally think these dead ends must have some significance. They don't. If you are tuning your website's search performance (ie doing normal, basic web performance inprovement work) you should ignore 'hits', 'pageviews', 'clickthroughs', 'requests' or anything of this nature. These figures are useless.

What you need to be looking at is visits, keyword performance, and referrers. And after this comes site errors. Nothing else is of any interest until you have doubled your website performance.

The best web analytics software would be an application that prioritises for these figures and presents the data clearly. If, in addition, it can give you alternative ways of looking at this data, then it will prove very useful.

Compare website stats applications

Webalizer
The first, and still the most widely used, of the free integrated website stats solutions. Considering it is provided free with just about every LAMP server on the planet, it is quite acceptable - especially from the cost / benefit point of view. It gives a reasonable basic view, in simple tabular form, of the most important things you will need to know at first.

The figure you need, visits, is not prioritised but can be found by examination. Unfortunately it is not measured correctly (in our opinion) and tends to overstate the case, from between 20% to 33% compared with every other web metrics application. Either they are all wrong or Webalizer needs throttling back a little.

Webalizer is unfortunately an example of a good stats package in the late 90's that is no longer relevant, because it hasn't changed. Basic server stats programs cannot be used for PPC tuning, so they are better off prioritising for organic search tuning or other purposes. To do this, visits / uniques has to be a priority; it is a waste of time to strongly feature 'hits', which was of interest in the 90's but is no longer relevant (whatever it is now or was then). It might mean HTTP requests; it could be page requests; it may be visitors; no one knows, really, so as regards a statistics application that needs to provide an absolutely precise analysis of data on perfectly understood criteria - it's useless.

They need to ditch all that hits rubbish and prioritise visitor and error data.

Analog
A good flat data stats presentation, especially useful for those who just need to see the main figures, and prefer this to a graphical interface. Unfortunately the main figure you need to see, visits, is not provided. It seems Analog is still back in the 90's when hits was the number to know.

In fact their favourite metric is 'requests', followed by pages. Presumably this is the easiest figure to pull out of the logs, and for them to be able to present; unfortunately no one else much is interested in it. These kinds of numbers are handy for pinning down some types of unusual server or site malfunctions - but of no use for tuning traffic. Again, this is another stats app that got bogged down in the 90's and never improved. It has little relevance today, which is why people have turned en masse to webpage script-tagged hosted stats solutions instead - at least you can get your visitor numbers from them.

Analog has the advantage of being able to run on your own PC, to view the stats locally without having to be online. There are some advantages to this, for some clients, so we used to provide a customised set-up that showed the figures at your leisure. There are much better solutions now.

AWstats
The last in line of these free metrics solutions. This one has the best graphical display of the ‘big three’, so that many people now go to the AWstats page first, for their daily (or possibly weekly) dose of web statistics.

AWstats gives by far the best determination of visit numbers: it tells you clearly how many unique visitors the site had, and how many visits in total. A proportion of visitors return to a website, some more than once. For many sites, returning visitors is the key to sales as unless the products are fairly low-cost it is recognised that actual buyers make more than one visit - they usually don't buy first time round. In addition, AWstats figures look the most accurate of the bunch, since they tend to be lower than the other free webstats, and tie up better with the numbers from the big money apps. Visitor numbers are still too high though, compared to more accurate applications.

AW has a reasonable error log presentation (better than the others at any rate), that will help you get rid of those unwanted site glitches that can plague a webmaster. Outline details for broken links, missing files, and other problems can be found here; though actually locating the errors is a different matter and will require much more detailed info.

Google Analytics
A free web stats solution that comes from the Internet giant, angled specifically at PPC data rather than search traffic tuning. There are some major differences from the server-side apps we looked at: this is basically a hosted solution, i.e. the main works resides elsewhere. No site logs are used; instead, a JavaScript addition to each web page is used - the common method used as an alternative when site logs are unavailable. This method can be referred to as JS page script-tags.

There are many obvious disadvantages: page tag script data sourcing can never be as effective as reading the server access logs; someone else hosts all your traffic data; a major search engine is given full details of all your traffic from all sources; you need to add JS to all your website pages; and it's yet more JS you can do without.

However, the clear benefits of GA mean that vast numbers of people are willing to overlook those negatives. You get full traffic data for all pages, and the system is tuned specifically for PPC management. Normally, it would be necessary to buy an expensive stats package to get better detail results than GA provides.

GA is a sophisticated web analytics tool now and provides may useful pages of data. Unfortunately usability is not its strong point and it will take many hours of work before you are able to extract the data you need.

However there are some useful extra data presentations that no other stats package can provide - mainly because they would not dare to. For example Google knows all the traffic data for every website on the Net that uses its page script tagging - and they will tell you your competitors' traffic. No one else would dare to do this but you can't sue Google, they are too rich. They don't give the precise figures, but these are easy to work out by comparison. A graph is shown of your traffic compared to your competitors or any other site you want to compare.

Best visitor stats

AWstats easily wins the free on-server web metrics contest - they clearly prioritise visitor number figures, they discriminate between uniques and visits, and they get the numbers right unless there are complex data skew factors. They also give some clues to on-site errors - but far, far less than is needed in a commercial environment. AWstats isn't much use for PPC (online advertising) data, though, so is best used together with Google Analytics if you require both organic search and PPC data analysis.

Site hosts can buy a commercial upgrade for AWstats.
 

Interpreting web analytics

Website statistics interpretation is not easy for those just starting with these tools. What does it all mean?

The first and most important thing to grasp is that there is only one important figure: visits. Forget about hits (whatever that is), requests, pageviews, IPs, or anything else. Only 'visits' - also known as 'visitors', 'unique visitors' or even just 'uniques' - is of any importance, most of the time, for the normal basic website metrics analysis that you must prioritise in order to get better organic search results.

If you have a lot of spare time, there are other things to investigate that will reward you with more information, at the cost of a greatly-increased time input versus benefit ratio. Basic visitor tuning needs to be the fundamental core activity of your stats work.

If your web metrics also tells you when you have made a sale, and can relate that information to referrer, landing page, and maybe even click trail - then you have a very powerful analysis tool that will certainly make you (or save you) a deal of money. That is because you can tune your site and your marketing to improve sales.

The only time 'pageviews' and so on are of any use is specifically for tuning PPC results. Otherwise, concentrate your efforts on the 'visits' figure and forget the rest - there aren't enough hours in the day. You need to see the monthly visit number rising strongly; originating from the referrers you target; arriving via the search terms you promote; the traffic going to the pages you want; using the sales funnel you determine; and hitting the KPI scores you set. Anything else is largely irrelevant. Except the bottom line, of course: how much they spend!

You can also see an example of basic website stats practical use, on the Logaholic Guide page, on the Statistics menu at top right.

Website error analysis

The second important thing your stats app needs is a way to identify errors accurately; that is to say, to precisely identify:

• The HTTP error code
• The originating page
• The page (or file) requested
• The numbers of such errors; plus dates and times.
  

With that exact information you can eliminate errors almost entirely. And that will make you more sales and reduce the risks of search engine non-compliance.

Realtime stats

No free website statistics app provides realtime stats (as far as we are aware). Neither Webalizer or any other of the common applications has realtime stats, and in fact they are likely to be 36 hours behind. These all need the reprocessed logs from the server - they cannot process their own logs or provide their own data. This means they are effectively a day or more behind, since although site logs are normally processed every 24 hours there are certain lags and delays that mean data can be either missing or erroneous within a 36 hour period of the desired timeslot.

A webstats app therefore needs to be chosen specifically for this feature, if realtime or near-realtime data is needed. In practice, near-realtime data is all that is needed 99% of the time. With this you can look at figures from an hour ago or earlier in the day and see what the score is - but you can't do that with any of the Big 3 free website statistics solutions. This also applies if you wanted to run the logs locally (on your PC or a LAN server), in an attempt to get around this; you still have to wait till after midnight or whenever the webserver processes the logs.

Click trail data

Another often-requested feature is clicktrail or clickstream information. This refers to detailed path info for any given visitor: the referrer, search request (if any), entry page, exact pages viewed, items clicked, and the exit page. There are now some free stats apps that provide some measure of this, though you generally have to spend money to get a proper solution here. None of the three main free stats apps have this in any form; and it's also worth noting that a "yes, we have this feature" reply can have a different meaning to what you might expect. There are degrees of implementation.

The best results in this field are obtained by those website stats apps referred to as 'hybrid' applications: they get their data from both the server and on-page script tags. This approach gives the best of both worlds, and for instance allows custom cookies and IP logging to identify the visitor.

Web analytics packages that allocate Session IDs to visitors are best avoided, as in general they are death to organic search results - search engines hate them. This is because session IDs (together with the JavaScript they are based on) have been used as a major tool in cloaking and spider trap attacks on searchbots (and thus on the search results). As a general principle, therefore, session ID-reliant statistics applications, and those that are heavily reliant on large and obscure JavaScript page additions, are not the best choice for a website that will need search-traffic dependent website revenues.

Hosted website statistics applications

There are now more than twenty free or inexpensive hosted web stats solutions. The free versions usually have a commercial upgrade path. One of the longest established is Statcounter, who are perhaps worth investigating first.

'Hosted' means that the application resides elsewhere, on a server belonging to the providers, and your webserver communicates with them and sends the data there. Normally, these hosted solutions require script additions to your website pages, and do not use the server logs.

Naturally, there are advantages and disadvantages to these stats solutions.

Pro:

• Many are free, so the ROI is pretty good...
  
• JavaScripting a page can give additional features that server logs use cannot, such as extra cookie generation, and an alternative visitor number confirmation method.
  
• They often work in a completely different way to the normal server log type of stats app. This can be of benefit.
  
• For PPC tuning in particular, these page-tagged stats solutions work well.
• It may be possible to locate a provider who supplies a solution that suits your business well, presenting data in a more useful way for your particular purposes.
  

Con:

• Page tagging cannot provide any of the 'internal' data that the server knows.
• Someone else has access to all your website data. There is a price to pay (nothing's free, after all): that info may not be personalised, but it is likely to be repurposed and used elsewhere.
• In a comparison between server log data and page tag data solutions, and their relative abilities, the server log solution is always better for organic search tuning.
• You have to script every page on the site (or, at least, all the pages you want to see in the stats); and it's adding yet more JavaScript.
  

So: the free page-tagged hosted applications may have something to suit you, especially if your site needs better PPC tools, and you are not sensitive about your traffic data ownership.

Hybrid website statistics apps

A combination of server log and on-page script-tag data acquisition has some attractions: this is called a hybrid solution. In the absence of serious funds to obtain such a solution commercially, a combination of free or inexpensive applications of the two basic types may well suit. Many site owners use this method very successfully.