Innovation Tax: The hefty price to pay for organisations entangled in data complexity

In today’s digital first economy data is king, feeding the thousands of new business applications created every day. Organizations have become heavily reliant on developers whose skills are required to build the software and the applications that will drive innovation and create a competitive advantage, both today and into the future. 

Being able to handle an exponentially growing amount of data while being agile and reducing time to market has become one of today’s greatest business, and technology, challenges. But why?

While the way we use data to build applications has changed to adapt to the digital economy and market demand, the underlying technology behind databases and data infrastructures unfortunately hasn’t and this has hefty consequences for organizations. 

Every day businesses are wasting precious resources (including developers’ highly sought after skills),  time and money to try and work around data management and architectural complexity. Resources that could be otherwise invested in creating actual value for the company.

As our CTO, Mark Porter recently pointed out after talking to over 300 CxOs, this complexity has transformed into a very pernicious Innovation Tax that is hindering organizations’ ability to innovate as well as drive competitive advantage and growth. 

And the most dangerous element is that this tax can be hidden, even while it keeps on growing each month. 

How has this Innovation Tax slowly grown without organizations realizing it? 

Working with data has always been the hardest part of building and evolving applications. But developers today are spending way too much time managing infrastructure complexity rather than focusing on the next generation of product, service or application. 

The reason behind this is that many organizations still rely on legacy data architectures and relational databases. Developers are unfairly expected to build new features and service modern, rich applications on top of these unsuitable and mismatched infrastructures. Relational databases (RDBMS) have been around for over 40 years but they were designed for an era when data was simple and static. That traditional design makes it difficult and time consuming for developers to build modern applications.

Large organizations in particular often have hundreds or thousands of apps, each with its own data sources and pipelines. As data storage and pipelines grow in number, a company's data architecture begins to resemble a plate of spaghetti. Developers on their end find it more difficult to collaborate because of the wide range of technologies available, each with its own frameworks, protocols, and, in some cases, languages.  

As our CTO Mark rightly shed light on, when an organisation’s tech stack can’t handle the demands of new applications, engineering teams will often bolt on single-purpose niche databases to do the job. They will then build a series of pipelines to move data back and forth. And everything will get slower and more complicated.

These complex, spaghetti architectures can’t meet the needs of modern applications, thus making it one of the biggest contributors to the Innovation Tax. People and resources are trapped into maintaining and fixing old, complex data infrastructures rather than creating value and innovating.

How organisations can stop paying their growing Innovation Tax bill 

The innovation tax is not distributed equally. The successful organizations around the world that have excelled in building sustainable growth and scalable innovation have one major thing in common: they understand the importance of empowering developers with the right tools and architecture so they can become the true innovation asset that they need to be. 

Smart organizations place a premium on simple, repeatable architecture and only use niche technologies when absolutely essential. They are also highly deliberate in their approach to security and data privacy, and ahead of data gravity which allows them to deploy a single application across various geographies and clouds without having to rewrite code or plan for months.

The key lies in meeting complex criteria without sacrificing simplicity or the developer experience, nor compromise on deployment flexibility. 

Organizations of all sizes are beginning to realize the extent to which a needlessly complicated data architecture holds them back. Realising that you are paying the Innovation Tax should be seen as an opportunity to finally break down those barriers to growth and competitive advantage. 

Businesses can either wait and keep paying more each passing year, or act now and redesign their data architecture in partnership with their developer team so both can thrive. 

The article has been written by Suvig Sharma, Regional Vice President, APAC, MongoDB