Data volumes are increasing at an unprecedented rate, exploding from terabytes to petabytes and sometimes exabytes of data. Traditional API approaches are laid out for transactional data and can’t handle these data volumes because they don’t scale well enough and are too expensive.

In this session, we will first look at requirements of big data applications and the boundary conditions of building a data service. Then will discuss how to build a data lake including data ingestion and ETL. And, mainly, we will see different patterns of how to build APIs for such services that can be used by internal and external consumers, including aspects such as semantics, security, cost and performance.