mysiteonline™

So I've been working on the statistics tracking "RefStats" (which apparently I'll have to rename; naming conflict with another project) and have given a much fonder appreciation for those people who work with creating statistic tracking services for a living (or a side project). It's ridiculous.

You record a hit, and a time, and perhaps some possible extra information depending on what your application is actually tracking. But just with the hit and a time, there's an exorbitant amount of ways to display information gathered from this data. How many hits were received on Thursdays? What were the average number of hits per weekend days in the month of March? Which day/hour of the month had the most hits? What percentage of hits were counted on Monday, Tuesday, Wednesday, and Friday - comparatively?

As the designer/developer of such a tool, you need to be aware of these types of questions. But that's not even half of the challenge. You also need to try to anticipate how your customers/clients will be using your application, and which of these reports will be the most important to them (if you're going to release reports in stages rather than do it all at once). You also need to try to reuse your code, so hopefully one querying method can be used to gather multiple reports, rather than a single method for each report (which although would be nice for modularity, a lot of these things are doing practically the exact same query with just a different WHERE clause, or different fields in the SELECT clause).

There's also the thought of interoperability of the data. What if you want an API for your application? What if you want the charting tool to be able to pull data from your primary querying method that serves all other aspects of your site so you don't have to develop two (or more) separate entities that are essentially getting the same information but rendering it differently.

There sure are some intelligent people and teams out there, such as Dan Grossman (http://www.dangrossman.info) of W3Counter fame, Laurent Destailleu​r of the original king of web stats: AWStats, and of course there's the team behind Google Analytics. These are examples of some of the big boys of analytics and by no means on the same miniscule level that I am even attempting to recreate; but the sheer fact of the amount of planning and ingenuity that goes in to such a project is simply mind boggling to me.

Anyway, read more to view some example images of the web reports that I have thus far - some development is still needed (note the disabled form fields, and I need to add a range option even though it already works), and the UI is not complete.

The Adobe AIR portion of the Reference Statistics Tracker is "done" (and has been for a week or two now). I just got approval over the extremely basic wireframe layout of the backend (web-based) tool that actually shows the stats data. Although it's basically only storing a very small amount of data, there are so many different ways that the data itself could be shown. If you think about how Web Analytics software can show so many different things; also think about what stats you have with it...it's basically a huge (textual) database of single line entries - the address, the referrer, the time, the visitor's IP, and a response code...yet there are tons of different ways to show that data.

It's a simplistic tool with huge possibilities. I aim to be rather simple for the first iteration out of the door.

Speaking of the first iteration out of the door - I ended up copying/pasting a LOT of the internal code in the Adobe AIR client to finish it. Having a local backup database in case of internet connection failure was a "last minute" additional "requirement" that I hadn't anticipated in the planning process, so I had to hack a lot of ugly stuff in to the code - and to finish it without a lot of it being rewritten, well... Thankfully, I do intend to upgrade it, as I left out one (actually two) important features.

1. Remote upgrade.
(2. Specify end-points to receive dynamic setup info.)

I left out the upgrade script because (1)I've heard it's a resource hog, and is a pain to work with, and...(2)I know I'll need to manually upgrade our library's PCs at the next version because of DeepFreeze anyway. (I left out the portion of the settings because I'm not yet ready to release the code as open source just yet, so I can hard code that stuff for now.)

I will post pictures of both the client and the web tool after it's been in use for a couple weeks - I'd gather that'll probably be about a month from now (I can't start work on the web panel until Monday, and it'll probably take me about two weeks to finish).

Most solutions that I've found for tracking outbound links with Google Analytics have been to send the link through a local file that would then be able to be tracked by Google, sending a query string as your end-result location. I never really liked that.

The other day, I accidentally found someone else's solution to use jQuery and Google Analytics for outbound link tracking.

The following code snippet is what I ended up with for my own use (so it doesn't add an actual visit/pageview to my own site's ranking):
$(document).ready(function(){ var domain = document.location.host; //includes port, whereas hostname does not $('a[href^="http"]:not([href*="'+domain+'"])').click(function(){ //can use the following for internal PDF/DOC hit tracking //pageTracker._trackPageview($(this).attr('href')); pageTracker._trackEvent('Outbound', 'click', $(this).attr('href')); }); });
What's happening here?
$(document).ready(function(){ - when the DOM is ready, do the following...

$('a[href^="http"]:not([href*="'+domain+'"])').click(function(){ - whenever a hyperlink is clicked that does not contain our domain anywhere in the URL, run the following... (this includes any subdomain, further string trickery would be required to keep subdomains listed in the same category)

pageTracker._trackEvent('Outbound', 'click', $(this).attr('href')); -- Run Google's _trackEvent method to register an event. Typically the 2nd parameter passed in is an "action". In my final version, I changed "click" to be $(document.location) instead, so that I can tell where people are clicking these outbound links from.

You can read more about the Google method "_trackEvents" from the Google Documentation on Google Analytics. Oh, and...about the commented out code, that's more of a reminder to myself, if I ever want to track PDF or DOC file clicks on my own domain, but the code would have to be revised a bit for that to work. Also, make sure you change the domain string to whatever you'd like to track (a specific subdomain, or your whole domain).

Work has finally progressed on the StatTracker. It's taken me a little bit of time to get accustomed to Event Driven programming, but I think I got the hang of it. To be honest, it was kind of nice after I understood what was going on (and created a flowchart to more easily see what I needed to code for). It forces me to go back to my OOP roots in C++. I've been doing procedural PHP for so long, I almost forgot how to spread procedure calls in to separate, more manageable blocks. I now call one function upon application instantiation that runs all other functions, depending on iteration.

Anyhow, the last post I had, I showed a mockup in Photoshop. Here's a live running example of the UI (built directly from the mockup):

What's done?

• CakePHP Web-based backend
• Adobe AIR application, can be installed on Linux, OSX, Windows
• Use jQuery JavaScript library with Adobe AIR
• XMLHTTPRequest to backend server from AIR app to dynamically build the UI

What small things do I need to fix?

• Currently, at least in Windows, if you have a :hover state on a button, moving the mouse to the transparent portion of the window does not trigger the mouseout event and disable the :hover state. I will have to overcome this either by JavaScript, AIR properties, or some HTML/CSS trickery.
• There is currently no indication that an action has occurred properly (did I press it properly? *click click click CLICK!!!*) as it's such a simple application, I felt it was unnecessary. My boss basically told me I was stupid (with a grin), and I agree.
• As of right now, the project only supports up to 9 different question types as the background colors are hardcoded in CSS. This will need to be dynamically coded at a later date. Our library only has a need for 4, so I'm not too overly concerned with this prior to me releasing the code.

What's left?

• Attaching onClick events to the buttons to submit a value to the server (very easy, and is the next step)
• Setting a modal dialog for the Text button to submit data to the server
• The hard part: If saving to the server fails for whatever reason, store the data in an internal SQLite DB - constantly check the server at predetermined intervals, and once connection succeeds again, upload all data in the internal DB back to the server, and empty the local DB store
• Backend will (eventually) provide numeric as well as graph/chart-type statistics (I think I'll use ezComponent's ezGraph, I was going to use Open Flash Chart, but Flash based data does not print, and I'd imagine our department heads will want to use the statistics in their monthly reports)

I switched tooltip libraries from SimpleTip to qTip, both jQuery tooltip libraries written by Craig Thompson (http://craigsworks.com/). qTip was a rewrite of SimpleTip. It didn't take much time to switch, but there was a bug that he only just fixed this morning that was preventing me from moving forward. Getting the window to properly display in the bottom right portion of the viewport was also "fun", but I'm learning, which is good. Unfortunately, it's all untested in Linux and/or OSX as our library only uses Microsoft Windows, but I can test OSX functionality at home at a later date; or better yet, once this goes up in SVN, anyone else can test it out too, and possibly supply fixes or updates (or if I use GitHub, create their own fork)!

Suggestions, comments, feedback?

I hate it. Until you start working with it and messing around with it, the documentation is terribly confusing. I kept trying to find an example (for quite some time) of how to use AJAX from within the main application sandbox (the initial "window" that opens upon run) to an external server. Almost all examples kept referencing local files from within the application, or used Flex which has a different mechanism in going out and grabbing information.

Jonathan Snook recently mentioned a talk he had given about plain old HTML/JS development of AIR applications in his blog, and...knowing it was my golden opportunity, I had to ask him if he knew of any examples of how to use AJAX in the main application window on an external domain with regard to the obscurities of the sandbox security.

He simply told me (paraphrased), "There is no security restriction on the main application sandbox. It should just work." Oooooooh, boy. Don't I feel stupid when I went and tried it and it worked just fine. I tried to PayPal him $10 for his time in responding, but apparently PayPal won't let me send money from my own account unless I link my credit card or bank account. :(

Some of you may remember my Reference Statistics Tracker Application that I have discussed in the past; that is the Adobe AIR application that I needed this assistance with... I had that part of the project (Reference Statistic Tracker) on hold simply because I couldn't figure out how to make a remote call, which sort of hinged on a majority of the GUI since the GUI is dynamically built. Awesome. At least I can continue now - especially now that the Staff Site's authentication is done. Luckily, I found a better tooltip plugin to use for the GUI in the meantime.

I'll probably create either a GitHub account, or create a public SVN repository on a subdomain here that can hold different projects I'm working on, if anyone's interested once this gets closer to a usable state. Any immediate preferences?