The past few years have been signified by an increasing velocity of change, deluge of data in diverse formats and the rise of high performance computing. These drivers have shaped data and decision-making trends across industries in 2017.
In this article, we will peer into the future and cover the 5 key Data Science trends for 2018.
AI will be the new normal
The increasing velocity of change puts pressure on organizations to make better data driven decisions. What this means for the enterprise is to have the ability to derive insights from the data faster, and with greater accuracy. This will lead to cognitive/AI or machine learning becoming an essential ingredient in enterprise applications and data science tools for automated insight generation. Even at individual level, Data Scientists are now gravitating towards using Machine Learning algorithms over traditional statistical techniques. They realize that these algorithms are better and more accurate in achieving the outcomes they desire.
Unstructured Data gets its due
With enterprises realizing the potential of insight generation from unstructured data; the dominant data science architecture will include purpose built optimized solutions for both structured and unstructured datasets. Yes, you are looking at a deluge of Data lakes!. Also, NLP + ML + AI efforts will result in robust platform approaches to derive potent domain specific insights from these unstructured datasets. Deep Learning and Neural Networks, which extend the power of ML by learning from unstructured data in layers will become essential toolkits in the Data Science community due to their ability to process text, video, IoT ( all forms of unstructured data) and astounding prediction accuracy with these huge datasets.
Monetizing data will be a key imperative
As companies encompass more data, and figure how to derive insights better from it - this will lead to revenue growth from information based products. Data monetization will grow to be a more important objective for enterprises, and a lot of Data Science effort will go in discovering business value to create data products. This trend will be most prominent in sectors which are highly digitized e.g High Tech, Telecommunications and Financial Services.
Model building will become more transparent
As algorithms take over more of our lives, Data Scientists will be tasked with being responsible for the consequences of the algorithms they create. This will lead to more interpretable models. i.e If the models you are building are designed to be decision support systems for stakeholders, then how do you ensure they understand the way these decisions are made. For example - your models might be building internal decision trees for coming to the answer that you want to predict. Can you expose how your model is making these decisions? Can you ensure that there are no hidden variables, or inherent bias in your data or algorithms used to predict the desired outcomes? The need for interpretability will also be driven by the desire to make Data Science/ML/AI more acceptable within the organization.
Data Translators will become more valuable than Data Scientists
In a 2011 report on big data, the McKinsey Global Institute (MGI) foresaw an extreme shortage of data scientists (140,000 to 190,000 people with “deep analytical skills” in the U.S by 2018.) As more Data Scientists enter this industry – there will be a burgeoning need for ‘Data Translators’ who understand the intersection of Business + Data Science + AI to be able to translate value of the data, insight and work done to executive stakeholders, and the organization as a whole. Data Science after all is powerless unless actions are taken to derive value from insights; and there is a great cultural barrier to that happening if the executives do not understand the work of these data scientists well.
By Afrozy Ara,
Head of Data Science Practice at Incedo Inc.