Session
Building the €100 data warehouse with the Azure Data Platform
When you watch some conference sessions, or read some blog posts, it always seems like everyone is drowning in petabytes of (streaming) data. They're proclaiming you need the fastest, best, most scalable, distributed shared-nothing (or everything?) multi-parallel system that can also set coffee. But guess what?
Not every company has to deal with a scale that businesses like Google, Amazon, or Tiktok have to deal with. Not every company needs streaming data with Kafka, Spark Streaming or Event Hubs. Not every company has complex unstructured data that you need to deal with in a data lake with tools like Databricks. Some companies have just a couple of gigabytes of data (or even less). Maybe a terabyte if we're pushing it. They don't need fancy streaming real-time dashboards, they just want to analyze their financial and sales data. It's ok if it just runs once a day. Their source systems are regular databases, maybe some Excel files and a SharePoint list.
In this session, you're going to learn how you can build a data analytics platform in Azure that is dirt cheap. We're going to cover the following technologies and patterns:
* cheap ingestion with Azure Data Factory using a metadata-driven framework, in combination with Azure Logic Apps and Azure Functions
* a data warehouse implementation in Azure SQL Database. We'll cover design best practices and scaling options.
* a Power BI model for presenting the data to the end users
In each section, we'll look at how costs can be contained. After all, we want a data warehouse which doesn't cost more than €100 per month!
It's assumed you have basic knowledge of Azure Data Factory, databases and SQL.
Koen Verbeeck
Senior Consultant @ AE - Data Platform MVP
Rotselaar, Belgium
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top