Lets face it, the users of your ShinyNewWebSite(beta) will never know exactly how it works. Perhaps that's by design (look, it's magic!), perhaps that's simply because they're not computer programmers, but this is the reality.
So there's this problem: how do I get users to provide informed consent to use my shiny new data collection web site? I want to do some really cool stuff, but I want the users of the site to know what's happening and feel in control.
This is hard. I think there's a ton of value in data mining and personalization, and it's not reasonable to expect users to comprehend the entire process of how their data is collected and used. We do however need to empower users to manage trust for the organizations who collect and use their data, and one way to do this is to get them closer to understanding what happens.
Here's one way I've been thinking about this: on one end of a spectrum are the users; they have values and want to assert protection over some of their data. On the other end of the spectrum are the web sites; they produce value from the users' data and want to be honest and compliant with users' desires. Right now there's often a huge gap between what users want and what sites actually do with their data. We need to shrink this gap.
I've talked about this gap from a user's perspective before (the privacy perception gap) and ultimately this gap leads to shock and discomfort. In Firefox 4, we deployed DNT as one feature to help shrink the gap from the user's informed-consent side.
Anything we can do to help make obvious users' preferences and privacy choices shrinks the gap from the user side, but we should work from the site's side as well, and hope the efforts meet somewhere in the middle. What else can we do to help bring site behavior into to the user's mental model of what's going on?
We need something new to improve upon privacy policies. We need something more objective than self-explanation. We need something empirical that can be measured, digested and shown to users. We need technology that makes it easier for people to peer into the opaque bits of the web and see what data is collected and how it's used. While it's not realistic to expect a silver bullet that makes all users instantly understand how sites work, we should still try hard; let's throw all the ideas that we have out on the table and approach this gap with as many tools as we have to try and shrink it.