Comments on: I’m wrong. NYT answers my challenge. http://selberg.org/2006/08/09/im-wrong-nyt-answers-my-challenge/ Erik Selberg's Homepage & Blog Sat, 30 Aug 2008 01:31:52 +0000 http://wordpress.org/?v=2.5.1 By: mc http://selberg.org/2006/08/09/im-wrong-nyt-answers-my-challenge/#comment-2244 mc Wed, 09 Aug 2006 20:00:45 +0000 http://www.selberg.org/2006/08/09/im-wrong-nyt-answers-my-challenge/#comment-2244 While there may be many privacy advocates who would agree that you can't ever anonymise data, and after the bad press from AOL probably no one else will; I think that it is possible to anonymise search logs for academic research by ensuring: * The queries are not linked to a unique user - this is the only way someones search *history* can be determined - now there is an argument you could still identify an individual from one search, but this is a hell of a lot harder, almost impossible and would reveal no more than the HTTP headers given to whatever website you click on (which most people probably don't realise they give out anyway) * Make sure the queries aren't time/date stamped so a webmaster can't linka query to personal information submitted to them, as I suggested in the comments to your previous article. Even without this, with the point above they still would have no more information on you anyway * Strip SSN, Credit Card Numbers, email addresses as Microsoft did, and queries of more than say 5 words where people are just pasting an email or whatever I don't know if such data would be useful to researchers, but if AOL had released data in this format, I think your challenge would have been almost impossible to meet. The question is then whether queries about identfiable people (you couldnt know who made the search but you could identfy the person being searched for) are accetpable - i.e. the query "Arnold, Lilburn, GA, gay" for example. While there may be many privacy advocates who would agree that you can’t ever anonymise data, and after the bad press from AOL probably no one else will; I think that it is possible to anonymise search logs for academic research by ensuring:
* The queries are not linked to a unique user - this is the only way someones search *history* can be determined - now there is an argument you could still identify an individual from one search, but this is a hell of a lot harder, almost impossible and would reveal no more than the HTTP headers given to whatever website you click on (which most people probably don’t realise they give out anyway)
* Make sure the queries aren’t time/date stamped so a webmaster can’t linka query to personal information submitted to them, as I suggested in the comments to your previous article. Even without this, with the point above they still would have no more information on you anyway
* Strip SSN, Credit Card Numbers, email addresses as Microsoft did, and queries of more than say 5 words where people are just pasting an email or whatever

I don’t know if such data would be useful to researchers, but if AOL had released data in this format, I think your challenge would have been almost impossible to meet. The question is then whether queries about identfiable people (you couldnt know who made the search but you could identfy the person being searched for) are accetpable - i.e. the query “Arnold, Lilburn, GA, gay” for example.

]]>