Big Data: Coming of Age

Big data has often been described by its attributes—notably, volume, velocity, variety and veracity. More important than the definition of big data is its effective usage which could result in transforming data into insights and intelligence, and if delivered at the right time and place, could help making and implementing better strategic and operational decisions, drive new product development, facilitate innovation as well as improve customer relationships. Apart from the effective usage, it is also a problem of plenty—90% of the total data available today was created in the last two years—hence organizations have to deal with gigabytes, terabytes and at times petabytes of data.

This article seeks to understand the state of big data, how companies in the technology industry and some other closely knitted industries are benefitting from big data, and the key trends that are expected to be popular in the coming year.

Big Data: Changing the Way Business is Done

Analytics has become a key requirement in businesses with organizations creating dedicated teams to ensure that the insights from big data are seamless and available all throughout the organization, some going as far as embedding analytics into their organizational DNA. Whether it’s a company dealing with high-value customers, aiming to bring in new clients with aggressive marketing campaigns, or looking to overhaul its business structure, the need for agile data to make real-time decisions has become absolutely necessary.

Capitalizing on such an opportunity begins with understanding the client/customer issues, the needs and goals of a company and the inputs that the new data will bring to improve the decision-making procedure. Traditionally data analytics has mostly been performed in areas such as customer segmentation, marketing research studies, value driver analysis, etc, however in this era of informed decision making, in order to succeed, it is important for every key business division to use big data. The evolution of new data-driven business models has given rise to advanced analytics by creation of hyper-intelligent software platforms that address a variety of issues, ranging from sales to customer experience to product design and delivery, thereby helping build an organization that is more responsive to the disruptive changes.

Benefits from Big Data

While big data has had a profound influence on—and hence provided benefits to—the technology industry, it also has influenced the other closely associated industries, ie, Telecom and Entertainment & Media.

The e-commerce (technology) industry, for instance, was one of the first to have started using big data analytics extensively to drive competitiveness by redefining the online shopping experience by personalized assistance, tracking customer preferences and sending email updates from recent browsing data, using it for operations as well as delivery metrics, monitoring loyalty programs, segmentation/click analytics, building smarter operations and a better supply chain network. For e-commerce companies, on an average, only 2% of visitors end up spending on their sites, and in such a high customer acquisition cost scenario, Predictive Analytics becomes pertinent in order to improve economics of doing business.

For some other players in the technology industry viz. Software companies having millions of customers, use of big data to better understand how their customers use their products and services has become imperative. Leveraging big data insights, they are able to modify their software development in order to enhance the customer experience by improving product usability and friendliness. Analysis of real-time usage patterns allows service specialists to help customers before the customer even perceives a usability problem.

For telecom companies, on the other hand, the benefits are multi-faceted viz. reducing customer churn by using sophisticated time series analyses of customer behavior, optimizing routing and quality of service by analyzing real-time network traffic, using insights into customer behavior and usage to develop new products/services and monetizing opportunities, allowing call center representatives to flexibly and profitably modify subscriber calling plans immediately, tailoring marketing campaigns to individual customers using location-based and social networking (SMAC) technologies, and lastly analyzing real-time call data records to identify fraudulent activities.

For entertainment and media companies, big data has enabled them to gain a ‘single view’ of their customers, including their interests and buying habits. This knowledge, when combined with new mobile content delivery channels, allows new product development, content and promotion to be narrowly focused—resulting in improved sales and margins. Micro targeting of advertising to individuals has increased the value of ad impressions by orders of magnitude while creating new business models and companies.

Key Trends of Big Data Market in 2015

Innovation: Disruptive innovation has become a way of life for the big data companies. The increasing use of new technologies such as real time in-memory analytics, interactive dash-boarding, ETL (exits, serial extract, transform, load) processes, and decision support simulation tools exhibit that there is demand for newer versions of the big data technology. Several software firms are working on big data apps designed for analytics, ideally reducing an organization’s reliance on data scientists.

Big Data and integration with other emerging technologies: Social Media, Mobility and Cloud along with the Internet of things (IoT) and DevOps are amongst the emerging technologies that can be integrated with big data to make most effective use of the latter. For social media, more big data applications will be created to support unstructured data analysis helping improve customer intelligence and operational efficiency. With Bring Your Own device (BYOD) and Master Data Management (MDM) becoming popular and enabling frontline employees with real-time insights thus generating massive amount of data, new business processes will evolve which will mix big data and mobility. Cloud will allow organizations to automate the deployment of big data platforms to a virtualized infrastructure; major investment in products that make this happen is expected to happen in the coming year. Several organizations are planning to take IoT beyond the experimental phase, and aim to generate business value out of IoT; with the help of IoT and big data, businesses will be able to track usage patterns and help better the customer experience.

Convergence of big data and business intelligence: Big data technologies are an important tool, helping to transform the large amounts of data into usable business insights. However, from a Business Intelligence (BI) perspective, big data also creates some issues as it introduces disruption from new and underdeveloped technologies, as well as create the risk of data and application silos. To generate useful customer insights, BI solutions are shifting to meet big data scenarios. Data exploration, data wrangling, and predictive analytics will become key capabilities in a broad BI suite. Furthermore, embedded BI segment with reports and analytic content integrated into other applications will continue to grow.

Increasing use of big data in enterprise content management: Big data initiatives are severely affecting the traditional content and records management practices and several companies are rethinking their information management policies in light of big data analytics. It is expected that Enterprise Content Management (ECM) will evolve with insights from big data being combined with historical text-based content.

From a technology perspective, Open Source analytics platforms and Analytics-as-a-Service (AaaS) will gain traction as companies are focusing on building intelligent systems. It is expected that creation of analytics will become more commoditized and organizations are expected to compete on consumption of analytics, which will fuel competition. There has been an increased focus on analytics education by colleges and organizations, and the trend is expected to continue.

Outlook

In India, most organizations, as far as big data adoption is concerned, are still in the inception phase. Many may not yet comprehend the value that can be generated from big data, or that the insights generated from the analysis of big data can be used to improve productivity, enhance customer experience, help identify monetizing opportunities and alter decision making for overall profitability. Those set of companies that may not be considering adoption of big data are likely to not possess the technology or the necessary analytical skills. In term of operational issues, big data poses a variety of risk issues such as the reliance on third party data, risks of data on public cloud, data ownership and quality, privacy, storage and retention of data, information security and regulatory issues.

Despite the relatively low implementation of big data technologies in India currently, because of the promise big data analytics has to offer, many organizations are planning to invest in it in the near future. There are several of them which are looking at analytics to drive innovation. Dynamically changing customer trends and the rise of the new informed consumer will further drive the growth of analytics, forcing organizations across the TICE and other industries to analyze how information can be used effectively. Those who will adopt big data and effectively deploy it will move ahead of the curve.