Evidently, big data has become a bigger deal in the banking & financial services (BFS) domain. But while data has been growing in scale and complexity, there is now an additional source of data that BFS firms need to seriously consider: Big Interaction Data.
For many years, BFS firms have managed massive volumes of data, in the magnitude of several petabytes. This continues to grow exponentially at a rate of over 40%-under increasing scrutiny from outside regulators and diverse data sources, factoring in stringent availability and security requirements. From a strategic perspective, taking into account growth parameters like core competence and the extent of value realization, big data promises to underpin new waves of growth and consumer surplus. One common question that’s doing the rounds in BFS firms is: Do they regard big data as an investment opportunity, or as plain cost? An informal survey amongst BFS honchos threw up findings that are incredible, another survey of around 600 CXOs and senior executives conducted by Capgemini emphasized that the use of big data has improved the performance of businesses on an average by 26% and that impact is expected to escalate to 41% over the next 3 years.
The Big Data Drivers
The drivers for big data vary from one BFS firm to another-and between banking organizations and financial Institutions as well. That said, the overall BFS industry is pushing to get a hand on big data owing to 3 key aspects: (a) the amount of data continues to grow exponentially and unhindered; (b) the resolve to improve enterprise transparency, auditability and executive oversight of risks and (c) the escalating need for internal users (of BFS firms) to access larger and larger data sets to help uncover market opportunities, customer trends, product development possibilities and of course, for regulatory reporting and risk management purposes. In retail banking, the desire to better understand customer behavior, prevent fraud, and manage risks is increasingly propelling the big data opportunity paradigm.
Here, the business opportunity to better understand customers across a broad range of products and services (ie, data sources) is a very compelling proposition for executives and the business unit heads. In mortgage banking, the driver for big data is mainly focused on developing predictive credit risk models that tap into reservoirs of payment data-so as to ascertain the patterns of consumer and commercial collections and help prioritize collections’ activities. From the perspective of buy side and sell side firms, the regulatory environment on Wall Street is dealing with exponential costs-something that keeps increasing per year! It’s no surprise that they are looking at newer avenues of technology like big data.
Also underway is a paradigm shift in the way regulators are monitoring the operations of BFS firms and mandating transparency. In some countries, regulatory authorities have directed all banks to standardize their regulatory reporting by following an automated data flow (ADF) approach to ensure 100% accuracy and zero human intervention in every stage of the reporting cycle-right from data extraction at source systems till the actual submission of returns (reports) onto their central data repository. This is impelling BFS firms in the direction of regulation driven incentives, towards applying insightful data analytics. Some investment banks still rely on overnight batch data to make trading decisions and are considering big data analytics in real time to make better trading and risk decisions. Storage of market data time series are driving big data initiatives in Trading.
Examples include application in quantitative trading strategies, processing of social media for sentiment analysis, audio recordings of transactions negotiated and executed via phone, structured and unstructured reference data relating to transactions, varying from records of corporate actions, counterparty and legal entity information, contracts and income flows related to derivatives and structured products and the like. Unmistakably, there’s a growing need for larger market data sets and deeper granularity to feed predictive models, forecasts and trading.
Big data initiatives may be excellent technology enablers for business;
however, implementing them is quite complex as it deals with information
pools that back crucial decisions; deriving meaningful business interpretation from available information streams remains a challenge. The top 3 challenges are:
Lack of Competencies-Such as unavailability of data scientists, lack of Analytical domain knowledge, Intuitiveness, Program management;
Data Complexity-Current data often needs cleansing, verification, and reconciliation from diverse sources and
Scalability-Integrative analytics tool sets for voluminous data poses technical challenge of scalability
For instance, in-memory databases are often used to store high frequency trading (HFTs) data. When evaluating big data requirements, the firm needs to consider the growth-size of their data set, their business needs, and how to process. Many other financial transactions require complex computing and data-intensive processes. Some firms segregate storage across a continuum- comprising bulk storage, big data wells, analytics operations, relational databases, and in-memory databases. To leverage big data in diverse scenarios, many firms are using bulk storage for giant flat files (including emails and other files) that are stored on backup tapes and use platforms such as MongoDB, Hadoop, CouchDB for big data that might be relational, and non-relational or semi-structured. When applied to the financial markets, big data approaches need to cope with challenging performance requirements for both storage and extraction of data, allowing complex analytics to be performed on high frequency, varied data elements.
Clearly, big data is going beyond merely enabling decision-making. The scope is getting wider and deeper, something which BFS firms are realizing-and using for digital marketing, new insights in products and services, social network analysis and fraud detection as well as prevention besides regulatory compliance. Will big data become an organization’s greatest asset, or one of its gravest liabilities? This depends on what kind of skills, strategies, and solutions the organization has (or will) put in place to deal with this epic growth in data volumes-the complexity, diversity, and velocity of data notwithstanding!