When Big Data Causes Big Problems #SMACtalk Ep 26

We all know that Big Data is a hot topic. As businesses and consumers we are constantly witnessing how data can be used to better understand our customers, our operations and of course our own buying behavior. 

However, big data in all of its glory isn’t always so perfect. What happens when we let data become a mere support structure for proving our own bias? With so much data out there, do we fall susceptible to data skew? 

While some point out that big data alone is rarely the answer (http://bloomreach.com/2014/04/we-get-it-big-data-alone-is-rarely-the-complete-answer/), the bigger question with so much data, especially unstructured data (meaning not numerical but other types of content), we may tend to use data to prove what we already believe rather than what the data is actually telling us. 

In this episode of SMACtalk, Brian and I dive into this topic and discuss how data, especially unstructured data can often drive us to our preconceived notions rather than to the best answer. Further, we discuss how important it is that we consider that data may wind up telling us something different than what we thought and we need to be open to that if we truly want to achieve the benefits of big data. 

Some of the examples given include how data can be misconstrued in…

• Social Media Analysis: Determining a hashtag is useful because of activity even if relevancy isn’t consistent or aligned with its use.
• Hiring Decisions: using only positive data to determine that a candidate is a good one. 
• Comparative Analysis: comparing two data points from different time periods to come to the conclusion of one data point being superior to another even though they aren’t truly comparable. 

Brian reiterates how important integrating data scientists can be into businesses that are seeking to use big data in their operations because we need those that are truly trained in how to interpret data for best outcomes and not just to prove our beliefs.

Of course just because data is available it doesn’t mean our intuition isn’t relevant. But what data can and should be used for is to enhance decision making, and when it is slanted to support what we want to believe, then data can become the cause of a much bigger problem. 

This episode is sponsored by IBM to help companies envision a #NewWayToWork.  IBM Verse, their new revolutionary social & email solution, can free up time for you to focus on this by calming your email. Sign Up Today at: IBM.com/Verse

As always we are grateful to be able to share our insights and views with you the listener, and welcome feedback, suggestions and open conversation at all times using the hashtag #SMACtalk.