Do Indian Universities Have Enough Infrastructure to Produce Skilled Data Engineers?

While it has been established by various studies that there has been a rise in demand for data scientists in the country, it is also being reported that there is a severe shortage of skilled data engineers in India. Although there is a discussion around the importance of adoption of emerging technologies such as artificial intelligence, IoT, and data analytics, are Indian Universities at par with universities abroad in educating engineering students with these technologies? Bhavesh Goswami, founder and CEO, CloudThat Technologies talks about the demand for data engineers, and what Indian Universities must do to produced good data scientists.

DQ: Data engineering jobs seem to be on the rise in India. What has led to this development?

Bhavesh Goswami: Data is the new currency for all corporations now. Also, due to the sharp decline in storage costs, companies are storing more data than before. Also, the companies are moving from traditional data systems like RDBMS to NoSQL and Data Lake. Thus, the need for data engineers that can store and process petabytes of data in these new data technologies has increased drastically.

Leading big tech companies like Facebook, Apple, and Amazon are actively recruiting data engineers to transform, source, and analyze their growing data at scale and help extract deeper insights essential for decision-making within their organization at various levels of operations.

DQ: Are there enough skilled individuals as far as data engineering jobs concerned?

Bhavesh Goswami: Not at all. Reports after report show that there is a dearth of talent in these spaces.

As the importance of data has substantially increased over the years, companies are in urgent need of specialized data engineers who can help them turn raw data into meaningful insights for their organizations.

If statistics are to be considered, a data engineer is one of the fastest-growing job roles in the technology domain. However, the question remains do we have the talent to meet the growing demand? The answer is a straight ‘no.’

Companies are still struggling to get the right candidate for their job requirements. Since 2016, the demand for data engineers have been on the rise, and in 2019 alone, there was a 50% increase in job postings in the domain. But compared to the demand, the availability of certified and skilled data engineers is very less even today.

DQ: What must engineer students do in order to be eligible for these jobs?

Bhavesh Goswami: I would say that Microsoft Azure has the most comprehensive data certifications in the industry. Certifications like “DP-203: Data Engineering on Microsoft Azure” and “DP-300: Administering Relational Databases on Microsoft Azure” have a great mix of older and newer data technologies that are industry-ready. Anyone who prepares for these certifications and clears it has a very good chance to have an embellished resume that will attract any recruiter.

Also, along with self-study, I would recommend students to take up specialized training courses offered by various training companies in these technologies that guide their preparation in the right direction.

But in the end, all that matters is how serious and hardworking are the students. They need to give in their best to successfully clear the certification exam.

DQ: Are educational institutions in India at par with universities abroad when education in emerging technologies is concerned?

Bhavesh Goswami: Although India is the third-largest in the world when it comes to higher education, still Indian universities are struggling to establish their presence globally compared to foreign universities. Most of India's colleges lack the proper infrastructure to support education in emerging technologies like Cloud, Big Data, IoT, AI, ML, etc. Even the faculty members are not skilled or knowledgeable enough to teach students about these latest technologies.

Moreover, unlike foreign universities, Indian colleges do not offer multiple courses and specializations for the students to take up and enhance their skills and knowledge, especially in the technology or science domain. Even the curriculum updates and modifications with the latest technology changes and market dynamics happen late.

Therefore, Indian colleges and institutions have a long way to go to reach the expertise and excellence of foreign universities. The task may seem challenging, but if impactful efforts are made in the right direction, even the difficult goals are achievable.