Transformational Insights with Messy Data

Data is the key currency of our times. It spurs innovation and defines the competitive landscape of the world’s most valuable companies including Facebook, Amazon, Apple, Microsoft, and Google. Much attention has been given to Big Data in the past decade. The digitalization of just about everything and the current wave of IoT yields enormous amounts of data for predictive analytics. In fact, the world’s data volume is currently on pace to double every four years. 

While there is an abundance of data, much of the world’s Big Data is fairly superficial. It consists of digital records oftentimes collected passively without the consumer paying much attention or  even being aware of it at all. In this sense we can think of this data as Thin Data. It contains information about what, when and how but typically lacks the underlying human motivation of why. Put it another way, we don’t necessarily understand the thought processes behind why the data exist in the first place. 

Thin Data is the opposite of Thick Data which is data that’s highly contextual, complex and often requires human intelligence to be analyzed and understood. This data is commonly referred to as qualitative data and can be in the form of spoken and written language as well as artefacts of consumption contexts and practices. Historically, market researchers have typically collected Thick Data from a limited number of respondents so as not to overburden the labor intensive data analysis. With the introduction of AI enabled technology, the ground rules for analysis of Thick Data changes. This leads us to the concept of Messy Data. 

Messy Data is made up of large amounts of Thick Data that isn’t immediately approachable for a market researcher to derive insights from. Messy Data is exactly what it sounds like. It’s difficult to deal with not just because it’s complex and multidimensional but also because it’s vast. The sheer volume of Thick Data makes it taxing. Without advanced technology it’s difficult to make inferences from messy data because the data material is complex and not immediately quantifiable. A qualitative data set can be coded manually to identify themes but it’s typically not very practical to do this for a larger sample as it’s time consuming and cost prohibitive. 

Harnessing the power of messy data has profound implications for business outcomes. When we use AI technology to derive insights from messy data and then connect these insights with other data points, we are able to connect the dots in ways never done before. By connecting the dots, we can develop transformational insights that enable a business to outperform its competitors. 

If Messy Data has sparked your interest you can read further in our white paper Messy Data Transforms Business Fortunes and learn how messy data can be collected and analyzed and integrated with other data sources  to develop insights for competitive advantage and outsize growth.