Data Crush in Information Fusion Centers

Out of curiosity, I googled the term “Data Crush” this week and came upon a post on the website Homeland Security Today:

Government agencies use the term data crush to describe being overwhelmed by the volume of data that is at their disposal when performing their intelligence-gathering activities.

There were a couple of interesting points made in the article. First, the analytic tools that are being developed for commercial purposes clearly have value in the investigatory activities of government agencies; there are similar issues of scale, scope and velocity in the data sets involved.  Second, I found it interesting that 77% of law enforcement  agencies polled are currently using social media analytics in their criminal investigations. So, be careful what you post!

Third, I was struck that the example given in the opening of the post was pretty eye-opening.  Someone reported to authorities that they saw a threatening post by a minor, stating on a Friday that he was going to go to school on Monday and kill a bunch of classmates.  Given events like the Boston Bombing, or the shootings in Virgina several years ago, authorities must take such threats seriously, and must act upon them when they find that a credible threat exists.

This then, presents the challenge of both socialfication and appification.  In our hyper-connected world, hundreds of millions, or even billions of messages are being created and shared every day. Within this torrent of messages may lie clues to future crimes, attacks and threats, and if these clues are found ahead of time, these attacks may be thwarted.   However, finding such clues in real-time is extraordinarily difficult, both in terms of raw computing power and in resolving the context of each message, in which much of the message’s value lies.

Appification impacts this problem because we as consumers have extremely high expectations of all of the services in our world. It seems that after every attack such as in Boston, authorities are able to find evidence on social media platforms that could have and would have warned of the attack ahead of time.  The media seems to find these posts within hours after the attack, and begins to question why the police did not find the same evidence BEFORE HAND and take steps to prevent the incident.

This is leading to ever-greater expectations of prediction and prevention on the part of authorities, which is a classic case of appification at work.  As the article implies, our government agencies are attempting to adopt technologies and techniques which will improve their ability to meet these expectations, while they also struggle with the attendant intrusions upon our privary and free speech rights. It will be tough for authorities to balance these conflicting expectations, under the constant watch of an impatient public.



Leave a Reply