Shivji Kumar Jha
Staff Engineer at Nutanix, Data & Cloud geek
Bengaluru, India
Actions
Shiv is a an architect at Nutanix and heads the cloud data platform team helping Nutanix products with data storage- databases, messaging, analytics etc. With a team of 6, Shiv works on making apache pulsar, nats-server, druid, debezium, presto etc available as a platform for all of nutanix cloud products.
Shiv loves spending time on Data storage and MLOps related research and has contributed to MySQL and pulsar codebases. Shiv is an avid reader (tech, fiction, economics etc) and is always looking at ways to simplify software architectures.
Links
Area of Expertise
Topics
Structured Event Streaming with Avro and Pulsar
Type safety is extremely important in an event streaming pipeline. Type definition and evolution can either be built in the application or relied upon the data layer to support it out of the box allowing the application to only concentrate on business logic, not how of data store and evolution.
Apache pulsar offers server as well as client side support for the structured streaming. We have been using pulsar for asynchronous communication among microservices in our nutanix beam app for over an year in production.
This talk presents the technical details on what is available in the apache pulsar server and client side, how we have used pulsar's schema support to build our use cases and our learnings from them.
Streaming app changes to Event Store
In this talk we will go through different use cases and design patterns (CDC, events, outbox pattern etc) for moving data from your web. services (and Databases) to an event store for analytics and real time computation. We will also discuss how this can be done in a type safe ways and the good, bad and ugly of each design choice.
Structured Data Stream with Pulsar
Type safety is extremely important in any application built around a message bus like Pulsar. Type definition and evolution can either be built in the application or relied upon the data layer to support it out of the box allowing the application to only concentrate on business logic, not how of data store and evolution. Apache pulsar offers server as well as client side support for the structured streaming.
We have been using pulsar for asynchronous communication among microservices in our nutanix beam app for over an year in production.
This talk presents the technical details on what is available in the apache pulsar server and client side, how we have used pulsar's schema support to build our use cases and our learnings from them.
Running a secure Apache pulsar cluster
A Pulsar instance consists of one or more Pulsar clusters. Clusters, in turn, consist of a broker cluster, a bookkeeper cluster and a zookeeper cluster. While this provides a modular and flexible design, there is a lot of bytes flowing over the network. In an application that is paranoid about security that is a lot of connections, both intra cluster and inter-cluster, to secure. At nutanix we have deploy pulsar in a secure environment and run it through a lot of security audits for infrastructure certifications.
In this talks, we will go over how we set up authentication and authorization on all the network communications to make our pulsar deployment secure.
How Pulsar stores your data
In order to leverage the best performance characters of your stream backend, it is important to understand the nitty gritty details of how pulsar stores your data. Understanding this empowers you to design your use case solutioning so as to make the best use of resources at hand as well as get the optimum amount of consistency, availability, latency and throughput for a given amount of resources at hand.
With this underlying philosophy, in this talk, we will get to the bottom of storage tier of pulsar (apache bookkeeper), the barebones of the bookkeeper storage semantics, how it is used in different use cases ( even other than pulsar), understand the object models of storage in pulsar, different kinds of data structures and algorithms pulsar uses therein and how that maps to the semantics of the storage class shipped with pulsar by default. Oh yes, you can change the storage backend too with some additional code!
This session will empower you with the right background to map your data right with pulsar.
EventSourcing Live 2021 Sessionize Event
Pulsar Virtual Summit North America 2021 Sessionize Event
Pulsar Summit Asia 2020 Sessionize Event
StreamNative Webinar Series
Lessons From Managing A Pulsar Cluster
Slides : https://www.slideshare.net/shiv4289/lessons-from-managing-a-pulsar-cluster
Video: https://www.youtube.com/watch?v=zAHxgG_U67Q
FOSSASIA 2015
MySQL Group Replication
Slides : https://www.slideshare.net/shiv4289/fossasia-2015-mysql-group-replication
Open Source India 2014
MySQL High Availability with Replication New Features
Slides : https://www.slideshare.net/shiv4289/mysql-high-availability-with-replication-new-features
MySQL Developers Day 2014
MySQL Replication and Scalability
Slides : https://www.slideshare.net/shiv4289/my-sql-replicationscalability
MySQL User Camp 2014
Global Transaction Identifiers in MySQL Replication
Slides: https://www.slideshare.net/shiv4289/mysql-user-camp-gtids
MySQL User Camp 2013
Multi-threaded Slaves in MySQL Replication
Slides: https://www.slideshare.net/shiv4289/mysql-user-campmultithreadedslaves
Open Source India 2013
MySQL: Multi-Source Replication
Slide : https://www.slideshare.net/shiv4289/my-sql-labs-multi-source-replication
Blog : https://mysqlhighavailability.com/mysql-5-7-6-introducing-multi-source-replication/
Shivji Kumar Jha
Staff Engineer at Nutanix, Data & Cloud geek
Bengaluru, India
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top