Statistics and Appreciation
Monday, December 21. 2009
So I've been working on the statistics tracking "RefStats" (which apparently I'll have to rename; naming conflict with another project) and have given a much fonder appreciation for those people who work with creating statistic tracking services for a living (or a side project). It's ridiculous.
You record a hit, and a time, and perhaps some possible extra information depending on what your application is actually tracking. But just with the hit and a time, there's an exorbitant amount of ways to display information gathered from this data. How many hits were received on Thursdays? What were the average number of hits per weekend days in the month of March? Which day/hour of the month had the most hits? What percentage of hits were counted on Monday, Tuesday, Wednesday, and Friday - comparatively?
As the designer/developer of such a tool, you need to be aware of these types of questions. But that's not even half of the challenge. You also need to try to anticipate how your customers/clients will be using your application, and which of these reports will be the most important to them (if you're going to release reports in stages rather than do it all at once). You also need to try to reuse your code, so hopefully one querying method can be used to gather multiple reports, rather than a single method for each report (which although would be nice for modularity, a lot of these things are doing practically the exact same query with just a different WHERE clause, or different fields in the SELECT clause).
There's also the thought of interoperability of the data. What if you want an API for your application? What if you want the charting tool to be able to pull data from your primary querying method that serves all other aspects of your site so you don't have to develop two (or more) separate entities that are essentially getting the same information but rendering it differently.
There sure are some intelligent people and teams out there, such as Dan Grossman (http://www.dangrossman.info) of W3Counter fame, Laurent Destailleur of the original king of web stats: AWStats, and of course there's the team behind Google Analytics. These are examples of some of the big boys of analytics and by no means on the same miniscule level that I am even attempting to recreate; but the sheer fact of the amount of planning and ingenuity that goes in to such a project is simply mind boggling to me.
Anyway, read more to view some example images of the web reports that I have thus far - some development is still needed (note the disabled form fields, and I need to add a range option even though it already works), and the UI is not complete.
You record a hit, and a time, and perhaps some possible extra information depending on what your application is actually tracking. But just with the hit and a time, there's an exorbitant amount of ways to display information gathered from this data. How many hits were received on Thursdays? What were the average number of hits per weekend days in the month of March? Which day/hour of the month had the most hits? What percentage of hits were counted on Monday, Tuesday, Wednesday, and Friday - comparatively?
As the designer/developer of such a tool, you need to be aware of these types of questions. But that's not even half of the challenge. You also need to try to anticipate how your customers/clients will be using your application, and which of these reports will be the most important to them (if you're going to release reports in stages rather than do it all at once). You also need to try to reuse your code, so hopefully one querying method can be used to gather multiple reports, rather than a single method for each report (which although would be nice for modularity, a lot of these things are doing practically the exact same query with just a different WHERE clause, or different fields in the SELECT clause).
There's also the thought of interoperability of the data. What if you want an API for your application? What if you want the charting tool to be able to pull data from your primary querying method that serves all other aspects of your site so you don't have to develop two (or more) separate entities that are essentially getting the same information but rendering it differently.
There sure are some intelligent people and teams out there, such as Dan Grossman (http://www.dangrossman.info) of W3Counter fame, Laurent Destailleur of the original king of web stats: AWStats, and of course there's the team behind Google Analytics. These are examples of some of the big boys of analytics and by no means on the same miniscule level that I am even attempting to recreate; but the sheer fact of the amount of planning and ingenuity that goes in to such a project is simply mind boggling to me.
Anyway, read more to view some example images of the web reports that I have thus far - some development is still needed (note the disabled form fields, and I need to add a range option even though it already works), and the UI is not complete.
Continue reading "Statistics and Appreciation"
