1. Rate of change in tech makes it nearly impossible to make “strategic choices”

By far the biggest challenge associated with building new data analytics platforms is choosing a set of technologies and services that will form part of your target state. This is driven by two factors. Firstly, the number of tech vendors has exploded in the last 10 years. So there are many vendors/platforms to choose from. Secondly, the rate of change in cloud based services on AWS, Azure and GCP is relentless. I’ve been on projects where design decisions in Month 1 are rendered suboptimal in Month 6 because a newer, better service is now available. Hence leaders in charge of these modernisation initiatives are often reluctant to commit to strategic choices delaying decision making. These delays inevitably lead to more change in the landscape making decision making more problematic. Where choices are made early on, the choices become increasingly “legacy” in wake of newer options.

2. Operating model for Data Analytics continues to be pressured due to ubiquitous access to Data Analytics tech

Traditionally, “strategic” Data platforms have been centrally built and governed by technology teams with business units funding project initiatives. This operating model reflects the pre-cloud era where access to computing services was administered by IT. This model left business users with relatively few choices but to engage with central tech teams to provide a Data service (not including Access dbs and servers under people’s desks!) However as Data Analytics tech becomes universally accessible through cloud platforms, non technology business teams are increasingly building their own data services and platforms without any involvement from central teams. If you’re tasked with leading technology and governance, you might say that these are rogue business managed IT solutions, and whilst this may be true, they continue to thrive thanks to near universal access to tech on the cloud.

The challenge then, for central tech teams, is to rethink an operating model that allows business users to continue to self serve without allowing a mish mash of ungoverned services and platforms to thrive everywhere. This requires a rethink in how data platforms are designed, built and made available for consumers in the business.

3. Traditional service management can no longer cater to a forever changing “service catalogue”

A byproduct of fast moving technologies is that as Data Modernisation projects head to production, the number of code patterns associated with a growing list of technologies is disrupting traditional ITIL based service management environments. ITIL based service management teams are geared towards stability, repeatability with run-sheets and process maps associated with incident management. As a side note, those who say we do “DevOps” are just in denial about what actually happens in production. Most data projects and platforms are not “DevOps”. People who say otherwise are either do not understand DevOps or are just unaware of what actually goes on. Most platforms continue to be supported in production by service management teams that do not possess the necessary engineering skillset to triage, troubleshoot and resolve incidents. This puts increasing pressure of service management teams to meet SLAs and SLOs.

4. One size fits all governance model

Lastly, nearly all Data Modernisation initiatives start by trying to create a single, all serving data asset (again think Data Lake). The challenge however, is that the asset inevitably creates a one size fits all governance model around it. Data Governance requirements, platform governance and change management processes make accessing new features and datasets on the platform harder. Soon the new data asset is slow to deliver functionality, hard to use and no one can get what they need from it in time. Advantages stemming from modernising data architecture are soon thwarted by governance structures around it.

Do these challenges resonate? Have you experienced them on your Data Modernisation projects? Let me know your thoughts below. In my next post, I’ll describe approaches I discuss with customers beginning on their data modernisation initiatives to better meet aforementioned challenges