Pálmi Símonarson
Reykjavik Energy
Reykjavík, Iceland
Actions
Pálmi has developed into a data engineer after working for years in software development. Working on everything data related Reykjavik Energy, from Data Architecture to DBA activities, building Tableau dashboards and lecturing on data literacy and SQL.
Area of Expertise
Topics
Breaking out of the habit of bulk loading everything
Lessons learned from integrating streaming data into an existing data warehouse / analytics platform based on conventional bulk loading patterns from on-prem systems/databases. When faced with integrating several new systems into the data environment, none of which have an accessible database, some reskilling, retooling, and rethinking of the ingestion patterns was needed. How does a team that is used to work with relational databases, Integration Services, daily updates of dimensions and facts, deal with AMQP/MQTT interfaces and the quest for near real time updates? Coming from an on-prem Microsoft data stack (SQL Server, SSIS,SSAS,SSRS) we look into Azure services. What are the architecture patterns that help us process business events we would like to analyse and capture? Can we combine requirements for near real time data analysis and actions with requirements for long term (10+ years) storage and analysis? How do we compare the cost of larger ready made PaaS building blocks with custom built code? The session describes how Event Hubs, Streaming Analytics, Azure functions and a host of other cloud services got integrated with the existing platform and daily operation, developed, and run by a small team.
Sweet streams (are made of this: Hubs, Stream Analytics, Datalake)
Take the first steps into the streaming analytics. In this 60 minute session we'll build a streamining ingestion, and processing platform using
IoT Hubs,Event Hubs and Stream Analytics for ingestion and real time processing
We will look into storage options for both hot and cold options
And end by creating serverless function to handle outliers
Event Hubs and Stream Analytics offer a relatively low entry barrier into world of streaming data processing while providing a world of capabilities at the same time.
Now we need everyone to be able to “read” this!
When we have created the greatest data products, reports or dashboards and nobody understands them, we might need to look at the data and information literacy level of our readers.
In this session we look at some practical steps towards improving data literacy that we can apply in our organizations and look at some questionable insights and presentations that we might want to refrain from using.
For our organization to be equipped to reap the benefits of the data (the new oil) the whole organization needs to be comfortable with using data and information in any of its many forms.
How do we prepare our data (sets, marts, lakes, lake houses) for maximum readability and usability?
How do we present our information for maximum effectiveness? Is the message being delivered and received?
We will look at categories of data, and how they nudge us in a particular direction of presentation.
Why is metadata even more important for the end user than the creator?
What are the most effective ways of presenting information to the seeing eye?
Why is clarity so important and how do we move towards increased clarity when communicating through data and information?
At the end of the session the attendees will have some additional tools to help move the data literacy levels of their organizations in the right direction.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top