Catalysed by the Covid-19 crisis, many organisations invested in digital transformation projects, fuelling a growing appetite for data streaming. As economic uncertainty looms large again in 2023, with global inflation and the threat of recession, intelligent investments in technology are needed to fortify long-term growth.
From cybersecurity modernisation to breaking the batch data habit, here are five key actions to take in 2023 to make your data worker smarter, not harder:
1. Creating operational efficiencies with data streaming
Overcoming data management and integration inefficiencies will prove key for tech executives keen to optimise the value of their data – and to protect bottom lines.
Across various industries, most data projects often run point-to-point to produce a certain output, sapping resources in merely integrating data and making it fit for purpose. Hence, busy in housekeeping, data scientists get very little time for high-value analytical work.
Once the data is out of the sources, flowing in a real-time stream, it becomes discoverable and reusable across projects. Data streaming – especially when managed via a third-party service – allows organisations to streamline their tech stack by consolidating software and mining the depth of machine learning. When the data is already in flow, developers just have to tap into it allowing data scientists to focus on innovating.
More and more companies are turning to fully managed solutions for help with real-time data. The IDC's 2021 Streaming Data Pipeline Survey reports that over 70% of companies have indicated that they plan on using managed services for their future data streaming projects.
2. Modernising cybersecurity measures as threats increase
Cloud-security vulnerabilities will continue to be a concern for organisations in 2023. One survey revealed that 48% find data protection more challenging to manage in the cloud than on-premises.
Given the outdated current systems, cybersecurity infrastructure modernisation is crucial to manage growing, sophisticated threats. The volume and different sources of data, as well data capture being closely coupled with other applications, further compounds the risk. Decoupling data capture in real-time and the ability to grasp and analyse a vast variety and volume of security data is definitely required for cybersecurity purposes.
With real-time data streaming, companies can rapidly sort, analyse, identify and deal with cybersecurity threats from large quantities of data. Anomalies can be quickly detected and dealt with before the damage is done.
3. Continuing investment and experimentation with IoT
IDC expects spending in APAC on the Internet of Things (IoT) to reach $433 billion in 2026, facilitated by expanding 5G coverage.
As manufacturing organisations further embrace AI for decision-making and IoT sensors collect more data, a powerful and highly secure data streaming infrastructure will be crucial to delivering the most value in the fastest possible time.
Car manufacturers, for example, will need to tie all the supply chain data together to facilitate just-in-time manufacturing workflows as well as predictive maintenance that could make the difference between profit and loss.
Certain communications service providers and food franchises are already leveraging new levels of connectivity from data streaming combined with 5G by providing notifications or offers to customers in the proximity of a recharge point or food outlet.
4. Secure data sharing and rethinking the ownership of data
The way governments and other bodies had to quickly figure out a way to share Covid health data safely, highlighted the critical need to share data in a secure, governed way.
Privacy and security concerns make customers, regulators, as well as institutions worry about sharing data. Yet, each industry has its requirements that demand data sharing. For instance, data shares allow financial service institutions to offer personalised services, address fraud and enable easier loan applications.
Markets are now shifting toward safer financial data sharing as data ownership is reconsidered. New consumer protection laws are also being drafted to give customers more power to share their data. Data privacy is now a fundamental right and consumers must have control over their data relationships with organisations. Companies that provide transparency and control for consumers will have a better brand reputation and more trust. This can be best enabled by having a secure, centralised data architecture powered by data streaming.
5. Breaking the batch data habit
Batch data processing, while fit for purpose, tends to diminish competitive advantage in the long term. With real-time data streaming, the Netflixes and Ubers of the world have strengthened their position as industry giants and also changed consumers' expectations forever.
Batch data processing is not good enough to provide data in real time. For example, bank customers will not think twice before switching to a provider that can reflect their transactions, update their address, or warn of a spam threat within seconds rather than hours. To remain competitive or indeed to survive, companies must break the batch data habit and turn to real-time streaming.
2023: The year of the data streaming transition
2023 will bring with it many challenges. Organisations that embrace the possibilities of real-time data streaming will stand a better chance of turning challenges into opportunities. This year will be a significant milestone for data streaming as businesses wake up to the value it creates.
The article has been written by Jill Macmurchy, APAC Vice President of Solutions Engineering, Confluent