Big Data Survey 2016: Does a budget increase lead to successful data projects?
Back in 2015, organizations expressed high expectations in regards to the use of data. Over two-thirds of organizations said they saw huge opportunities with data, nearly 70% expected a budget increase for 2016. Vision and management support were the large themes, operationally the challenges were building up teams and developing central data warehouses. Organizations pointed out that they used data mainly for marketing purposes, but that they expected to start developing use cases across their entire organizations soon. In Big Data Survey 2016, we have researched these these and more. Will organizations have been able to live up to their expectations this year?
Financial room for projects
Despite the fact that in 2016 organizations indicate that the opportunities with data are less big than they were last year, more organizations employ data scientists and use predictive models.
Since last year, more organizations have appointed budget, which has led to financial room for initial projects.
Beyond the hype
Let’s face the brutal facts: becoming a data driven enterprise is not done overnight. There is no such thing as a magic box that spits out an algorithm after you put some data in it. Developing successful data use cases is primarily about making data available, exploring the data and continue working on successful experiments and sinking unsuccessful ones.
When we take a closer look at industries, it becomes clear that organizations that have introduced a data driven process have different experiences. This might have to do with the fact that organizations have experienced first-hand that developing successful data use cases is not trivial.
All along the line, organizations in Energy, Travel, and Financial Services can be seen as frontrunners when it comes to becoming data driven, while organizations in Manufacturing, Healthcare, and Public Services, seem to be a bit behind the adoption curve. This has mainly to do with the limited room to experiment, data security, and the challenge of making data available.
The knowledge hurdle
This year, the demand for talented data scientists has reached new heights. Building up a data team remains a top challenge for many organizations. This necessity forces organizations to either recruit their own team and/or insource external know-how. Only half of organizations work with external consultants.
Remarkably, organizations that rate their own skill level the lowest, work the least with external consultants; they simply are not yet able to treat data as an import production factor, which leads to a situation where it is impossible to determine cause and effect. Do these organizations lack knowledge because they have not embraced a data driven process, or have they not been able to become data driven because they have no access to the right skills and knowledge within their organization?
Difference in perception between management and work floor
Ask a manager about the level of data security in the organization and it is quite likely that you will receive a positive answer. Ask an executive professional the same question and the answer will probably be slightly less positive. Big Data Survey shows that there is a difference in perception between managers and executive professionals when it comes to various aspects of the data driven way of working. Not just on data security, but also on the most valuable working conditions. Managers think that wage is the most important condition, but data scientists put more value in available time to experiment, a transparent organization, and especially the knowledge level of their colleagues.
No doubt that vision and support from the management are vital to becoming a data driven organization, but it is the executive professionals that can make or break a successful implementation. For example the know-how and natural curiosity of data scientists allows organizations to adapt to change more quickly. The role of the management should be rather to facilitate and to provide sufficient room to experiment than to dictate.
About Big Data Survey
Big Data Survey is an international research by GoDataDriven that studies the use of Big Data in organizations and offers organizations an independent insight in trends and developments in the field of data applications. Organizations can take better business decisions by better understanding what is going on within organizations in different industries and of different sizes.
After the success of the first survey in 2015, in this edition we can determine whether organizations have been able to live up to the expectations. 315 people, divided over operational roles (168 people) and managerial roles (147), all from a wide range of positions, company sizes, and industries. The research focuses on the key topics Organization, Data, Applications, Process, and Knowledge and Skills.
Follow us for more of this
Testing and debugging Apache Airflow
February 22, 2019
The Zen of Python and Apache Airflow
February 18, 2019
AWS Machine Learning Competency Status for GoDataDriven
February 14, 2019
GoDataDriven Open Source Contribution for January 2019, the Apache Edition
February 13, 2019
Our social responsibility as a company
February 08, 2019
Keras: multi-label classification with ImageDataGenerator
January 31, 2019