Data platforms are notorious for needing infrastructure that needs to run 24x7x365. Here I share a story of a data platform from requirements, design to implementation using only serverless services in an enterprise setup. This is a real stroppy from my work at Polestar cars.

The talk has 4 major parts

1. The business context
2. Why we chose a data lake over other data paradigms?
3. Architectural walkthrough of our solution - A technical walkthrough of the solution. It highlights how we made this entire solution self service oriented.
4. Tradeoffs and lessons learnt- I will talk about the friction we faced for adoption, challenges around data security and governance etc

Some takeaways for the participants -
1. When to platform a data lakes aka centralise vs decentralise?
2. tips to ensure the data lakes remains usable
3. our take to enhance developer experience via self service mechanisms
4. When should you use a data lake over other data products like data warehouse