Crawlers are always a cool topic, but they take a second role in this talk.

One day we've decided that running and taking care of a bunch of VMs in various regions, only to crawl the web from them, is tedious, not so scalable, and honestly we needed a new hobby project.

The idea was to build a system which can spin up some computing resources across the globe with a wide range of IP addresses without having to use proxies. The system also had to be scalable, require as little maintenance as possible, hopefully cost less than the current one to run, and since we were getting requests from some potential clients - it had to be multi tenant.

During this talk you'll be able to learn why did we go with serverless containers instead of serverless functions, how are we automatically provisioning infrastructure on a need basis on three clouds, how are the requests and responses flowing through the system, etc, etc.

Basically, at the end of the talk you should have a general idea on how to build your own system which spins up and down various tasks on multiple clouds.