Session
Tale of Apache Parquet reaching the pinnacle of friendship with Data Engineers
CapitalOne being Tech Company in Banking business, we are 100% Cloud operated Company with Data DNA. CapitalOne Loyalty is one of such cloud native application processing billions of credit card transactions yearly and delighting our customers with rewards. This talk will be see thro' lens into our data processing pipeline and how Apache Parquet plays a pivot role in each step of our processing. We have various design patterns implemented using Apache Parquet and Apache Spark and this talk will touch upon those as well and how our resiliency has increased with usage of Apache Parquet. There are multiple credit card processing streams and how Parquet helps in our choreography and replaying them will also be covered in this talk. Apache Parquet is deeply intertwined in our pipeline and this talk will highlight on how it is connected and used in our pipeline. Overall audience will take away some interesting real world usage of Apache Parquet in Apache Spark data processing pipeline.
Gokul Prabagaren
Lead Software Engineer at Capital One
McLean, Virginia, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top