Armagan Karatosun
Cloud Data Services Expert - EUMETSAT
Griesheim, Germany
Actions
Armagan Karatosun (He/him), holds an MSc in High-Performance Computing from Istanbul Technical University with 6+ years of industry experience. As a Cloud Data Services Expert at EUMETSAT, he specializes in crafting cloud-based solutions. His focus is on creating resilient and event-driven infrastructure tailored for advanced applications like AI/ML and high-performance workloads. He is an expert in cloud technologies, particularly Kubernetes and OpenStack.
Links
Area of Expertise
Topics
European Weather Cloud: A Community Platform* for Earth Observation
The increase in data volumes from Earth Observation (EO) data in the past decade has led to an emergence of cloud-based services in recent years, but within their own data silos, often behind their respective data access methods.
In response, the European Weather Cloud (EWC) emerged as a "community cloud" for the public space sector and research institutions, allowing users to provide computing resources close to the data and foster collaboration. However, its Infrastructure as a Service (IaaS) model introduced challenges, requiring climate scientists to acquire new skills for managing and provisioning infrastructure, diverting focus from EO data processing to cloud computing basics.
Recognizing these challenges, we believe that evolving the EWC into a platform tailored for EO scientists, offering higher-level services and abstractions, is the future.
In our presentation, we will demonstrate the evolution of our services, the challenges we faced, and how we tried to overcome them.
Kubernetes Meets Climate Science: Building Large-scale Feature Detection from Climate Data Records
The Exponential growth of Earth Observation (EO) data volumes in the past decade has made downloading and processing EO data locally impractical. In response, the European public space sector launched initiatives to provide private cloud infrastructure, like the European Weather Cloud (EWC), allowing users to provision computing resources close to the data.
Leveraging these new possibilities introduced by cloud services and machine learning, the hydro-meteorological community has initiated projects to identify features from remote sensing data, including satellite imagery, to enhance early weather warnings and climate science. EUMETSAT and its Member States are now developing a collaborative environment within EWC for manual annotation, model development, and analyses to provide reliable feature identification from EO data.
Join us in our session to learn more about our solution, involving an environment for data preparation, community annotation tools, and a features database.
Terraforming Earth: How Terraform Bridges the Multi-cloud Gap for the Earth Observation Community
The increase in data volumes from Earth Observation (EO) data in the past decade has led to an emergence of cloud-based services in recent years, but within their own data silos, often behind their respective data access methods. Although there are some efforts on the "Data Federation" to unify the data access, this has not been achieved so far and still will not solve the traditional way of disseminating the EO data, which forces end users to download data from data providers for analysis and processing.
With growing data volumes, the data size often exceeds the capacity of being downloaded and processed locally. Therefore, there are several efforts to bridge the gap between the distributed nature of the EO data processing, allowing end-users to provision compute resources close to the data, eliminating the need to download the data, and focusing on processing data on their respective cloud infrastructure in combination with the emerging cloud-optimized formats such as Cloud Optimized GeoTIFF (COG) and Zarr and frameworks such as Dask.
In this session, we will dive in on how Terraform helps bridge the multi-cloud gap for the EO community. We'll examine practical applications such as the Data Proximity Compute Platform (DPC) and the European Weather Cloud (EWC), addressing various challenges in accessing and disseminating EO data today, and how it can be adapted for other use cases around the globe. We will showcase how Terraform can help to deploy, manage, and operate on multiple Openstack clouds and several Kubernetes clusters as well as how other Hashicorp products can be integrated to offer additional services to end users. In the meantime, we will also highlight the challenges that we have faced in multi-cloud applications, such as service discovery, network connectivity, and security.
Join us to learn more about our journey! We'll give you a detailed look at our setup, highlighting how Terraform can make Multi-cloud and Multi-cluster setups easier.
To Cloud or Multi-cloud: The Case Study of EO4EU Project's Multi-cluster Kubernetes Platform
In this presentation, we're going to explore how EO4EU's Multi-cloud and Multi-cluster infrastructure has been designed, deployed, and operated on two distinct Openstack clouds, WEkEO DIAS and CINECA ADA Cloud, and multiple Kubernetes clusters distributed across them.
By adopting a Kubernetes-first approach and Infrastructure as Code (IaC) and GitOps best practices, we have built an efficient toolkit to deploy, manage, and operate on multiple Openstack clouds and several Kubernetes clusters. We've tackled numerous challenges in the meantime, such as service discovery, network connectivity, and security between multiple Openstack clouds and Kubernetes clusters. We have come up with generic solutions applicable to different use cases.
Join us to learn more about our journey! We'll give you a detailed look at our setup, highlighting how GitOps and IaC can make Multi-cloud and Multi-cluster setups easier. We'll also share our blueprints that can be adapted to different use cases.
Unlocking Multi-cloud Observability: The Case Study of EO4EU Project's Observability Platform
In this presentation, we will dive into how EO4EU's Observability Platform, a suite of community projects, is tasked to observe the multi-cloud infrastructure spanning two distinct OpenStack clouds, WEkEO DIAS and CINECA ADA Cloud, and multiple Kubernetes clusters distributed across them.
Cross-cluster configuration management and replicability are ensured by the implementation of Infrastructure as Code (IaC) and GitOps best practices. By embracing a logical framework of "observer" and "observee" cluster annotations, every cluster provisioned within the platform is automatically configured to transmit metrics, logs, and tracing data to a central "observer" cluster. Simultaneously, each cluster retains data locally for a specified retention period, ensuring local data availability.
Join us to learn more about our journey! We will offer a comprehensive view of our setup, illustrating how open-source tools can come together to provide insights into your applications.
HashiTalks: Build Sessionize Event
European Geosciences Union (EGU) 2024
* A Replicable Multi-Cloud Automation Architecture for Earth Observation (https://meetingorganizer.copernicus.org/EGU24/EGU24-1857.html)
* Unifying HPC and Cloud Systems; A Containerized Approach for the Integrated Forecast System (IFS) (https://meetingorganizer.copernicus.org/EGU24/EGU24-9795.html)
CNCF-hosted Co-located Events Europe 2024 Sessionize Event
Armagan Karatosun
Cloud Data Services Expert - EUMETSAT
Griesheim, Germany
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top