Paul Mooij
Azure Solution Architect
Amsterdam, The Netherlands
With a background in both Electrical and Business Engineering, I have been working in the field of Data Warehousing & Business Intelligence since 2008. As a Business Analyst, (Technical) Project Lead, Developer, Scrum Master, Coach and Architect I am experienced with various roles and have completed projects in a variety of industries.
I love working with multidisciplinary teams in challenging environments while delivering value through data in-line with business goals and in close corporation with stakeholders.
When doing so, my approach is straightforward, constructive, visible within the organisation and result driven. Best characterised am I as being analytical, structured, adaptive, flexible and helpful.
For high quality frequent delivery of value in a sustainable way, my teams use Construction & Design principles, ETL automation, Test automation, Deploy automation and share knowledge throughout the project as far as possible within the project/environmental boundaries.
Doing so, leveraging cloud for serverless solutions is key...
Links
Area of Expertise
Topics
Serverless handling gigabyte XML in Azure
With emerging industry data standards like ISO20022, FpML, XBRL, HL7 etc. XML is getting more and more common as a standard for exchanging data. The advantages are numerous, but mainly on the definition and validation part of the story. When the XML is getting heavy - let's say > 1gb - many out-of-the-box approaches start running into processing issues and the XML gets too big to handle.
During this session we will explore and compare DOM, Streaming, LINQ, XPath, and .NET serialization for processing multi gigabyte XML with a variety of services like Azure SQL, Azure Functions, Azure Data Lake & Azure Databricks. You will see how to separate the concerns of large file handling and parsing complex XML. Running on a serverless platform with elastic scale enables for integrating this into your serverless big-data pipeline.
Hands-on stuff where solutions will be shared with attendees.
presented @ https://www.sqlsaturday.com/790
Pseudonymisation by Azure Function (GDPR)
Data protection rules such as the EU's General Data Protection Regulation (GDPR) apply to personal data. Rectical 28 of this regulation states: The application of pseudonymisation to personal data can reduce the risks to the data subjects concerned and help controllers and processors to meet their data-protection obligations.
In this session we will explore various approaches to pseudonymising your data. Eventually wrapping-up with an Azure Function hosted app, backed by Azure Key Vault stored keys and monitored key usage by Azure Log Analytics.
Using the developer's credentials for authentication during local development and using the managed service identity when deployed tot to Azure Function app. Running on a serverless platform with elastic scale enables for integrating this into your serverless big-data pipeline.
Azure Function Fest
Azure Functions are great for composing severless microservices and can be of great value in any modern (cloud) solution. In this session we will cover how to handle the separate concerns of semi structured data with serveral Azure Functions and finally chaining these together in a data pipeline which fits the bigger picture of the data architecture.
I'll take you on a tour through Azure Functions where we'll be running various demos, showing code snippets, touching modern authentication, interfacing with Azure Storage, Azure SQL, Azure Key Vault and wrapping things up with CICD through multi-stage YAML pipelines.
Hands-on stuff where solutions will be shared with attendees.
level: intermediate / advanced
duration: 60 min
SQLSaturday 790 Holland
session: Big XML Data handling in Azure Data Platform
Paul Mooij
Azure Solution Architect
Amsterdam, The Netherlands
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top