Data. It’s everywhere. It’s a Digital world out there and whether you are an individual or a company, in any given day huge amounts of data has been generated and no wonder we call it a ‘Digital Universe’. Amid the aggressive conversations on going Digital across the world, enterprises of all sizes are looking at how they can use the data for better outcomes and arrive greater business profitability.

Given the impact of data on our lives and on businesses, clearly the biggest challenge for both the vendors and the enterprise IT organizations is to mine the relevant data and achieve better business outcomes, but it is easier said than done, as more data means more complexity.

A World Economic Forum (WEF) observation stated: “This personal data – digital data created by and about people – is generating a new wave of opportunity for economic and societal value creation. The types, quantity, and value of personal data being collected are vast: our profiles and demographic data from bank accounts to medical records to employment data. Our Web searches and sites visited, including our likes and dislikes and purchase histories. Our tweets, texts, emails, phone calls, photos and videos as well as the coordinates of our real-world locations. The list continues to grow. Firms collect and use this data to support individualized service-delivery business models that can be monetized. Governments employ personal data to provide critical public services more efficiently and effectively. Researchers accelerate the development of new drugs and treatment protocols. End users benefit from free, personalized consumer experiences such as Internet search, social networking or buying recommendations.”

Why Is Data The New Oil?

For ages Oil (crude) is considered as an asset that runs the world, it’s a utility that we cannot live without. With the advent of the Internet age, the bulk of the earth’s population has a digital footprint in one way or the other. We create and consume data. We need to see this data evolution through multiple lenses, one as an individual we produce so much personal data and this personal data can be further granulated into many subsets. Data that are purely personal- the ones we have in our AADHAR, PAN and bank accounts.

These data are classified as mission critical in terms of security and privacy. This data becomes often a target for hackers. The second type of personal data relates to what we have in social media, online shopping sites, cloud services like photo sharing, email et al. This consumer data has huge value for the companies to understand many things about the individual and this data in many ways it is unstructured and most of the service providers data management and capturing algorithms capture these random data and structure it and suggest recommendations to the user and provide consumer-centric insights to the companies.

The third type of data relates to ‘business, productivity and process outcomes’. Enterprises use BI and Analytical tools to manage the business strategy for product lifecycle management. Say for instance, an FMCG company launching a new beauty product and post the launch, it wants to understand what kind of consumers have purchased it across age groups, it needs to go deep into its retail engine and do a 360 degree analysis culling buying data through an omni-channel approach to understand the customer demographics and satisfaction of the product.

To do this data analysis, one needs data scientists and analysts who through multiple touch points mine this data and create the consumer behavior for this product and the dynamics of that particular business category. So gaining insights out of data has multiple ramifications across the enterprises. This is where the vendor opportunity lies in.

Method Out Of Madness

The biggest challenge in mining the data is its spread and plurality, scattered across multiple sources. Experts say that many organizations have the equipment and expertise to handle large quantities of structured data, but with the increasing volume and faster flows of data, i.e., big data, they lack the ability to mine and derive actionable intelligence in a timely way.

Not only is the volume of this data growing too fast for traditional analytics, but the speed at which it arrives and the variety of data types necessitates new types of data processing and analytics solutions. Big data analysis involves making ‘sense’ out of large volumes of varied data that in its raw form lacks a data model to define what each element means in the context of the others. For example, one may have no idea whether or not social data sheds light on sales trends.

Besides, new types of BI inquiry entail not only what happened, but why. For example, a key metric for many companies is customer churn. It’s fairly easy to quantify churn. But why does it happen? Studying call data records, customer support inquiries, social media commentary, and other customer feedback can all help explain why customers defect.

With advanced analytics, similar approaches are being used with other types of data and in other situations. Why did sales fall in a given store? Why do certain patients survive longer than others? The trick now is to find the right data, discover the hidden relationships, and analyze it correctly to draw actionable intelligence from a haystack of information. Big Data is actually accelerating the democratization of BI.

So data is the resource like oil and it is up to the players in the fray to extract it for greater mileage. The biggest players in the data space Google. Amazon, Facebook, and Microsoft make the most out of the consumer data. Meanwhile, companies of multiple hues provide data analytics solution to the enterprise for an outcome-driven business model.

Business and IT decision makers in Indian enterprises see an investment in BI and analytics solutions as critical for their growth. Today enterprises need radical new BI and Analytical tools to provide pro-active insights and business outcomes. From start-ups to eCommerce companies, and banks to large manufacturing companies, almost every organization is beginning to invest in BI and analytics solutions to decode the changing consumer predicament and identify operational inefficiencies within the organization.

There is a greater understanding about the value of these insights to create better business strategies that can lead to enhanced productivity, a stronger competitive position, and greater innovation—all of which can have a significant impact on the bottom line and help find new avenues for top-line growth. In fact, according to an industry research, organizations that use analytics get $10.66 for every $1 they spend on analytics.

From a CIO’s Point Of View

In spite of the widespread implementation of analytics, enterprises struggle to fully realize the promise of operational effectiveness from their BI solutions. For many, the BI tools available are difficult to use and slow to respond and the content they deliver is of little relevance. Besides, the sophistication of many of today’s analytics applications, combined with larger and more complex data sets, is making it challenging for casual business users to interpret all the information they can now access.

As a result, business users default to making decisions based on incomplete information or their ‘gut feeling’. IT and business managers also say that most analytics and BI solutions are typically disconnected from planning systems and transactional applications so metrics viewed in BI reports and dashboards often have little or no relationship to target metrics established in financial plans. In the same way, the annual targets established in financial plans and budgets often have little or no relationship to operating plans and dynamically changing business conditions.

Going Forward

Indian organizations are increasingly moving from traditional enterprise reporting to augmented analytics tools that accelerate data preparation and data cleansing, said Gartner in a recent observation. This change is set to positively impact the analytics and business intelligence (BI) software market in India in 2018. Gartner forecasts that analytics and BI software market revenue in India will reach US$304 million in 2018, an 18.1 percent increase year over year.

“Indian organizations are shifting from traditional, tactical and tool-centric data and analytics projects to strategic, modern and architecture-centric data and analytics programs,” said Ehtisham Zaidi, principal research analyst at Gartner. “The ‘fast followers’ are even looking to make heavy investments in advanced analytics solutions driven by artificial intelligence and machine learning, to reduce the time to market and accuracy of analytics offerings.”

“We are witnessing a rapid shift to the cloud and hybrid data management through focused data management offerings, including integration platform as a service (iPaaS) tools for cloud integration and data preparation tools for self-service integration,” said Zaidi. “We are also seeing the emergence of data lakes and data hubs, as a new way to ingest and manage multi-structured data. However, unavailability of talent will continue to be a major inhibitor toward their adoption.”

As organizations embark on an aggressive digital transformation play, it is vital for them to understand data and equip themselves to cull insights and business outcomes. They need to constantly innovate and create a cohesive data management strategy and deploying analytics-driven workflow processes.

And going by Gartner, there is an avid demand from Indian organizations to integrate and manage unstructured data, and some are also experimenting with data science on real-time streaming data. As a result, data management software market revenue in India is on pace to total US$950 million in 2018, a 13.2 percent increase year over year.