The insurance tech industry has evolved from old, expensive systems but complex rules and highly sensitive PII data often mean that the systems remain long after their expiry date. Huge amounts of technical debt accumulate as analysts try to band aid a deteriorating code base and in turn make the business line dependent on shadow IT departments as time goes on. Often there doesn’t exist a clear dividing line between the OLTP and OLAP workloads causing the reporting layer becoming intractable. Users end up relying on data extracts to get what they need, reducing data engineers to data babysitters.

As with other traditional industries trying to modernise workloads, the insurance industry struggles with its analytics solutions. You have analysts that are stuck keeping production running, never getting beyond descriptive analysis. Also, shifting the analysis space right and the data governance to the left is an intrinsically hard problem for most businesses. Margins are often thin and earnings requirements high, so that the paths to moving to more data-driven ops are narrow and difficult to navigate. For many, the solution is the cloud. However, moving to the cloud has many challenges for a highly regulated business such as the Insurance/pension industry.

In this session, we will discuss how Gabler is transforming our on-premises analysis (excel) platform to cloud architecture using synapse pipelines, Azure Databricks change data feed, Databricks lakehouse and synapse serverless architecture. We’ll also discuss how we are implementing data governance using unity catalog, and how we went about breaking down a stateful data model to a more event-based star model (Kimball for the win!).