I want my data – plain and simple. When I first began to design web sites, I quickly learned about the information collected when visiting a website. Browser type, operating system, computer name, date/time, and IP address among many other pieces of data – and that’s just the tip of the iceberg. A well designed website will able be track visitor’s activity within the site and report back to the site owner. It would be very interesting to me to be able to view this data. I created it by entering input into the website, so why should this data be hidden in some back end administrative tool, and not readily available for me to view? I want to see what I have contributed to the database!
Kryder’s Law gives us an idea of what to expect in improvement rates for “annual bang for buck” over time when it comes to hard drive cost and capacity. Storage is really the only limitation a company faces when considering what data to keep. As annual bang for buck increases over time, it becomes less and less costly to store large amounts of data. New memory technologies are constantly pushing the edge of what is possible. We can only expect to have access to more and more memory for less and less money… and at some point data storage space will become a commodity.
Merriam webster defines commodity as “an economic good: as a mass produced unspecialized product”. As a good becomes commiditized, it’s relative cost approaches 0. Not just data storage is being commoditized. The processes by which that data is accessed is by nature commoditized over time as well. New query developments stand on the shoulders of those before them, and have a solid base of function to work with that only becomes more streamlined and feature laden over time as more brainpower is fed into the underlying logic structure. Once a query is built, anyone can use it and expand upon it. As storage space approaches a zero cost, and the language with which to query the data becomes more dense, the question for organizations when it comes to data retention and analysis is less of “What should we store and analyze?” and more of “Why NOT store and analyze EVERYTHING?!”.
So exactly what data is being stored as you visit a given website? A good place to check is the website’s privacy policy – here is an example. As far as I’m concerned, I should be able to give an administrator at whitehouse.gov my IP address, and they should be able to press a button to output my activity at their site over any time period. This data that has been stored about my input as I navigate their website was created by our joint effort – they have created a data collection mechanism, and I have used it to input data.
This phenomenon is not limited to website visitation. Every motion you make that is electronic is recorded and stored somewhere – charging a purchase to a credit card, making a cell phone call, even changing the channel on your TV.
Hey Verizon, do you think I could have the data that your company has stored from the GPS locater in my phone? It would be really cool for me to see where I have been and how much time I have spent there. The problem is, I don’t want to pay for this! Sure you can pay a third party to give you this data, but why should I have to pay for it? I created it! How about the cable company? Which channel did I spend the most time on between 4/2/09 and 4/8/09? I know that you know…
Now if every company that stores data about my actions had an API for me to create my own queries and analyze MY data as I see fit, then all of my problems would be solved. It bothers me that I just can’t get to you on demand, oh precious data. I predict that some day in the not too distant future, the data I have manifested will be at my finger tips again – right where it started. Does anyone else feel as strong a bond with the data that they create as I do?!
The good news is that sites hosted by WHardy include comprehensive reporting metrics – you can view the data that visitors to your site create on a daily or monthly basis. This data is essential to tracking visitor’s use of your site… where did they visit? Where did they come from? How much time did they spend while they were there? Graphical representation of these data points make it easier than ever for you to see who did what, how they did it, and for how long.