Paper long abstract:

Arguments for the Data Retention Directive and revelations from Edward Snowden reveal something of a Big Data fetish on the part of security forces. We are told that these forces need to know about all communications if they are to keep us safe from terrorism. And computational service providers seem eager to promise them that with enough data and the right algorithms, they will be able to predict and prevent crimes. Part of the argument behind this is that if N=all then sampling error is no longer an issue; if N=all then apparent correlations are true correlations; if N=all then no prior theory is needed as calculations are not about testing but about discovery. This paper will, in simple terms, discuss why these arguments do not hold water and why Big Data - while it may serve well for other purposes - is dangerous to rely on for security purposes.