Managing information explosion in the digital age

By: Chandra Sekhar Pulamarasetti, Co-founder & CEO, Sanovi Technologies

The year 2012 marked the beginning of the big data era. Big data turned into a watchword

much before it gained clarity as a concept. It became a catchphrase in almost every other

analyst report since that time frame. Today, there is a massive amount of data available—there are well up to 1 bn websites and near 5 bn webpages uploaded online. The digital world holds enormous volume of information, on an average more than 200 exabyte of

information is generated on digital platforms every month and is doubling annually.

Modern enterprises are overloaded with abundance of data and information and each year there is a subsequent expansion of more than 50% in this volume. There is a stark difference in data processing in comparison to the way it was done earlier. More than 60 terabytes of information is processed by enterprises annually today, 1,000 times more than it was a decade ago. Since there is an overflow of information, the focus has turned to discovery, integration, exploitation, and analysis of this data.

EXTRACTING MEANING FROM MASSIVE DATA SETS

Generally the data gathered, created and processed by companies today is largely unstructured. Most of this data is stored in word processing documents, spreadsheets,

images and videos, therefore complicating its retrieval or interpretation.

Unstructured data is basically information that lacks proper configuration. It is data that is neither a pre-defined model nor a pre-defined organized form. Most of the unstructured data is generated via popular social media forums such as Facebook, Twitter, Instagram, etc. On the contrary, structured data is more organized and is better defined. As it is very clear and analytical, it is easy to organize and is usually stored in databases. Despite its simplicity, industry experts estimate that there is only 20% structured data account that is actually available.

The elements in unstructured data are invaluable. This is why organizations are exploring different methods to extract meaning from these databases. These meanings hold the key to many crucial business connections and patterns that would otherwise result in missed opportunities.

The best way to draw the full potential of unstructured data is through cutting-edge analytics tools. These tools will give enterprise the much needed impetus to reduce fraud, ferreting out waste, and even confining cybercrimes to a certain extent. In spite of much dissimilarity, both these components of database work in tandem to create an effective big data operation. Therefore it is advisable for enterprises who wish to make the most out of information explosion to utilize tools that draw the benefits of both these components of database.

Big data has brought about a transformation in the way organizations analyze and optimize their internal and external business processes. Financial institutions such as

banks find data analytics tools and technologies effective to deal with certain risks and frauds.

The Indian banking scenario has undergone a phenomenal transformation during the last two decades. There have been multiple expansions in branch network and speed. This change has led to radical transformations in the functioning, scope, and reach of business operations of financial institutions.

There has been a dramatic growth in the size and intricacy of business. Information plays a vital role in the world of finance. There is voluminous and unstructured data or information in firms that requires a modern approach for analysis. Today, banks and firms have to accept the power of data and realize the impact of proper data analysis. Effective analysis is important to increase productivity, better decision making, and improved risk management capabilities. These factors help banks gain the competitive advantage. Most experts refer to this phenomenon as the big data evolution.

As companies increase the collection of information, there will also be a simultaneous expansion in data repositories to store, cumulate, and draw meaning from this database. Big data provides corporations with much needed insights on aligning their services in line with the specific customer requirements. This contributes to reduction in corporate inefficiencies and mobilizes data sharing with user groups.

SECURITY CONCERNS IN THE BIG DATA ERA

Large sets of consolidated data are most prone to attacks from hackers. A single cyber attack could prove detrimental to the organization. There is more payoff and recognition

at stake if an organization’s big data repository is breached by cybercriminals. If attackers set eyes on big data repositories, the effect will be quite devastating. Every terabyte of data in the repositories with information on customer data, employee data, and trade secrets could be crucial to the enterprises’ existence.

The recent popular PlayStation breach is likely to cost Sony an estimated $171 mn. A breach in the financial institution or healthcare big data repository could be more damaging, as value of the data is extremely high in these industries. Also, it will involve several government regulations, complicating the situation furthermore.

The major challenges in information explosion is collecting data, aggregation, analysis of big data, infrastructure used to store data and the technologies applied to

analyze structured and unstructured data. Each data source is likely to have its own access restrictions and security policies, making it difficult to balance appropriate security for all data sources. Big data environment may include data set with proprietary research information, a data set requiring regulatory compliance, and a separate data set with personally identifiable information. Protecting big data requires balancing analysis like this with security requirements on a case-to-case basis.

Another major challenge is standardization of security controls of big data environments distributed geographically and physically. It is essential to establish accessible locations. For instance, if data scientists across organization require to access information, it becomes important to protect the perimeter. Ensuring access to users while protecting system from probable cyber attacks is quite a conflicting situation. Since it’s a huge set up with a large number of servers, there could be inconsistency in the server configuration and this leaves some of the systems vulnerable to cyber attacks.

Other technological glitches are that traditional big data programming tools such as Hadoop and NoSQL databases face security challenges since they were not originally

designed with security in mind. This creates vulnerabilities for authentication and network security. In order to safeguard data against redundancy, centralized control of data can be the best solution. In this process, the database administrator (DBA) avoids duplication

of data, which effectively decreases the amount of data stored. Centralized control of data also eliminates the problem of extra processing, which is needed to locate the data in a large storage. Another advantage of this method to avoid data redundancy is that the user can eliminate the inconsistencies that are caused by duplication of the data files. The redundancies that exist in the database management systems can be controlled by this method, where the system assures that the multiple data copies are consistent.

Normalization is another process that protects the data against redundancy by maintaining the integrity of the database. In order to establish the integrity of the database, the DBA should follow some rules and instructions related to the parent-child tables. The DBA should adhere to the guidelines that are developed by the database community. These rules and guidelines are very simple and easy to follow and help organizations in protecting their data against redundancy.