TV advertisement has always been the most preferable medium for marketers to reach a mass audience. The networks/channels get the money from selling ad slots for their content and It's very important for networks to predict tune-in for their content to get most out of ad slots. Traditionally networks have to rely on TV audience measurement but It's often not helping networks to estimate the tune-in because of variety in contents, different airing time, changing behaviour of TV viewers, etc. Since the inception of smart/connected TVs, now we have access to second by second viewing data of households about their TV watching behaviour with the consent.

At MIQ Digital India Pvt. Ltd. we collect and process this high volume data and apply machine learning models to predict tune-in with new viewers and repetitive viewers category to help TV networks get maximum out of their ad slot selling.

We use Apache spark MLlib to model and PySpark for data wrangling and feature engineering with Kafka based event driven microservices architecture. It uses a well defined Data Engineering ecosystem of Lakehouse architecture built on top of Delta Engine.

This talk would cover details around scaling MiQ's TV product to market across >50 advertisers ultimately generating media delivery of ~40 million dollars. Details of pipeline optimisation for data at TB scale along with cost optimisations for model generations and prediction are key aspects that would be highlighted in the talk.