Teo Van Lier
Data Engineer @ mind the facts
Gent, Belgium
Actions
A Belgian data engineer who spends most of his time building data pipelines on Fabric. I work with Python, PySpark, SQL, and all the tools that help turn messy data into something useful.
I enjoy figuring out what actually works in the real world and to keeping things simple instead of over-engineered. As a consultant, I help teams make practical choices, improve performance, and build platforms that don’t fall over.
Links
Area of Expertise
Topics
It worked in the demo: running real-time Fabric solutions in production
Building a real-time demo in Microsoft Fabric takes an afternoon. Running it in production is another story. In the demo, the source never goes offline and every event arrives exactly on time.
In this session I start with a quick tour of what Fabric offers for real-time data and what a typical setup looks like: events flow in through an Eventstream, land in an Eventhouse where you can query them within seconds, and end up on a real-time dashboard or trigger an alert. Then I share what we learned taking pipelines like that into production: deploying real-time items across environments, monitoring that tells you when data stops arriving, recovering from source outages, and what an always-on workload does to a shared capacity.
Then we break things live. We start with a pipeline that works, switch off the source, let the data go stale and send events late. After each failure we add the piece that would have caught it, until the pipeline alerts you, recovers by itself and fills the gap without duplicates.
You will leave this session with:
- A way to monitor data freshness on event time, so you hear about missing or late data before your users do
- A pattern for restarting and backfilling after an outage without losing or duplicating records
- A setup for deploying real-time solutions across development, test and production
- A go-live checklist you can use on your own project the next morning
Do You Really Need Spark? How to choose the right execution model in Microsoft Fabric
Spark has become the default choice for data workloads, but distributed compute is not always the right answer.
In this session we dig into real workloads and compare Spark to native Python head to head. We will walk through scenarios like cleaning and transforming a few million rows, running aggregations on mid-sized datasets, and performing look-ups and joins on reference data. We'll look honestly at where each approach wins and why that is the case. Along the way we'll cover the trade-offs that don't always make it into the documentation: cold start times, debugging overhead, compute costs...
You will leave this session with:
- A clear understanding of what Spark is actually optimised for, and what it isn't
- Practical examples of workloads where native Python outperforms Spark in speed, cost, and simplicity
- A decision framework you can apply immediately to your own pipelines and notebooks
The ability to confidently argue on which solution suits which workload
Whether you're building new solutions in Microsoft Fabric or inheriting ones that already exist, this session will sharpen how you think about execution models and help you make better architectural decisions from now on. You'll spend less time over-engineering, less money on unnecessary compute, and more time delivering solutions that actually fit the problem.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top