Session
How Apache Parquet plays a pivotal role in processing billions of CapitalOne transactions
CapitalOne being Tech Company in Banking business, we are 100% Cloud operated Company with Data DNA. All our workloads are Cloud Native. CapitalOne Loyalty is one of such cloud native application processing billions of credit card transactions yearly and delighting our customers with rewards. This talk will be see thro' lens into our data processing pipeline and how Apache Parquet plays a pivot role in each step of our processing. We have various design patterns implemented using Parquet and Spark and this talk will touch upon those as well and how our resiliency has increased with usage of Apache Parquet. There are multiple credit card processing streams and how Parquet helps in our choreography and replaying them will also be covered in this talk. Parquet is deeply intertwined in our pipeline and this talk will highlight on how it is connected and used in our pipeline. Overall audience will take away some interesting real world usage of Parquet in Spark data processing pipeline.
Gokul Prabagaren
Lead Software Engineer at Capital One
McLean, Virginia, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top