Session

Serverless handling gigabyte XML in Azure

With emerging industry data standards like ISO20022, FpML, XBRL, HL7 etc. XML is getting more and more common as a standard for exchanging data. The advantages are numerous, but mainly on the definition and validation part of the story. When the XML is getting heavy - let's say > 1gb - many out-of-the-box approaches start running into processing issues and the XML gets too big to handle.

During this session we will explore and compare DOM, Streaming, LINQ, XPath, and .NET serialization for processing multi gigabyte XML with a variety of services like Azure SQL, Azure Functions, Azure Data Lake & Azure Databricks. You will see how to separate the concerns of large file handling and parsing complex XML. Running on a serverless platform with elastic scale enables for integrating this into your serverless big-data pipeline.

Hands-on stuff where solutions will be shared with attendees.

presented @ https://www.sqlsaturday.com/790

Paul Mooij

Azure Solution Architect

Amsterdam, The Netherlands

View Speaker Profile