Big Data and Analytics for Responders

May 9, 2016

Some readers might remember the 2002 film Minority Report in which an Orwellian future police force would arrest would-be criminals before they committed their crimes. The data processing used in that film was based more on human precognition than on today’s predictive models, but the idea is similar: if we process the right data in the right way, BDA (Big Data and Analytics) can greatly enable the work being done to keep our communities safe and secure.

This short paper will briefly define BDA and provide examples of current applications and challenges faced by many organizations, then discuss potential future contributions BDA can make to the field of safety and security.

What is BDA?
Big Data has become a recent buzzword in vogue with technology vendors, statisticians and management gurus. The term refers to three different but related ideas: data of such size, volume and variety that special technologies are needed to store, manage and disseminate; the techniques used to analyze data, or the technologies themselves. For our purposes, I’ll refer to Big Data and Analytics (BDA) because analytics can contribute to better results regardless of the size of the data or the technology being used.

Although hundreds of analytic techniques exist, there are three key categories: descriptive, prescriptive and predictive.

Descriptive techniques are well-understood by most people: for example, the standard trend lines depicted in Figure 1 above. These techniques, commonly referred to as “visualizations”, help users understand what is actually going on with respect to the issues we care about.

Prescriptive techniques have been used since World War II to help optimize decisions. Rooted in operations research, these techniques help managers make informed decisions related to questions such as how to better deploy resources, how much to spend on different types of systems, or where to locate security facilities to best respond to threats.

And third, Predictive techniques are considered the holy grail of Big Data and Analytics. They create the ability to anticipate what is likely to happen and therefore act before situations get out of hand.

In the last few years, crime analytics has become an important element in the crime fighting arsenal of many police forces. The term “crime analytics” encompasses a wide range of analytic activities so a few examples will help clarify the point. Prescriptive techniques such as queuing models and linear programming are being used in the United States (St. Louis, New Britain, New York, and Dallas are a few examples) to better define where to situate patrol cars or police stations in order to reduce the response time to crimes. These analytic approaches are also helping the municipalities save costs related to fuel and wear and tear on response vehicles.

The cities of Santa Cruz and Memphis use predictive modelling to identify key locations. Police officers receive a feed in their cruisers with a probability estimation of where crimes are likely to occur coupled with a read out of past crimes within the specific area.

In the UK, a prescriptive technique called Data Envelopment Analysis is helping police services understand what combination of resources best leads to crime detection and resolution.

Prescriptive techniques such as queuing models and linear ­programming are being used in St. Louis, New Britain, New York and Dallas (as a few examples) to better define where to situate patrol cars or police stations in order to reduce response time.

On a broader scale, threat assessment is a well-known application of data in the field of safety and security. Analysts mine websites, social media outlets and other forms of communication between and among people to anticipate likely security threats. This form of analysis has been prevalent for some time and is not unique to the security field.

Private sector companies similarly mine online data to find out who will buy things and what they would likely buy next. Netflix for example, became somewhat famous for its million-dollar challenge to analysts: the task was to build a better recommender system (the system that recommends movies to you when you log onto Netflix). Many law enforcement agencies conduct similar analyses to determine social media chatter can help identify potential threats.

Location analytics is becoming an important factor in all of this. For example, working with IBM, the U.S. Navy has designed a system of embedded microphones around certain naval facilities. The microphones listen to sounds around the facility, while a bank of computers learns algorithms and analyzes the sounds to determine if they are made by animals, humans, wind or other phenomena. The system continually learns to distinguish which sounds security personnel should pay more attention to, thus providing an early warning notices of potential threats to the facility.


Seismic sensors are being installed in the retrofitted San Francisco-Oakland Bay Bridge span to help with early detection of earthquakes.

The Way Forward
While much progress has been made in the application of BDA, a number of challenges still remain. For many organizations, the vast amount of data available can be intimidating: it’s difficult to know where to start. The lack of a common data vocabulary is another problem: different parts of the organization use different terms to mean the same thing or the same term to mean different things. Analytic techniques need clean data if they are to provide any value to management. Therefore, unclear definitions frustrate the development of analytic programs. Moreover, the mix of data available can confound attempts to wring any meaning from the analytic process. For instance, how do we combine images with text, sound and numbers to draw a clearer picture of what is going on?

Despite these challenges, the use of BDA in the field of safety and security is growing. New technologies, especially related to the use of sensors have broadened the types and amount of data we can gather. For example, 199 seismic sensors are being installed in the retrofitted San Francisco-Oakland Bay Bridge span to help with early detection of earthquakes, and the Shanghai Tower, the second tallest building in the world at 632 metres, has 400 real-time sensors installed to monitor inclination and wind pressure. These sensors transmit hundreds of thousands of data bits per second.

Shanghai Tower, the second tallest building in the world at 632 metres, has 400 real-time sensors installed to monitor inclination and wind ­pressure. These sensors transmit hundreds of thousands of data bits per second. 

What might the future hold? Apart from deeper application of some of the techniques already discussed, Linked Analytics can help law enforcement agencies define clusters of potential perpetrators.  It’s not often necessary to curtail activities of everyone in the cluster so these types of analyses can help to identify the ringleaders; remove them from the cluster, and it will normally fall apart. This approach saves time and effort, but also speeds up the response to potential threats.  And, back to the notion of “thought crimes”, Jim Adler of Intelius has developed an experimental algorithm to predict the likelihood of someone committing a felony. Based on an analysis of 630 million criminal cases and 40 million defendants in the US, he has identified markers such as age, gender etc., that suggest which individuals are more likely to commit crimes. Adler insists that the algorithm is purely experimental, but with the power of large data sets coupled with advances in data processing speed and advanced analytics, it might be time to take the concept of thought crimes more seriously.

Gregory Richards is the MBA Director and Director of the Centre for Business Analytics and Performance at the University of Ottawa’s Telfer School of Management where he teaches courses on Organizational Performance Management and Management Consulting.