There are numerous tools and programs on the market that require the provisioning of ephemeral environments to be properly tested.

But what exactly do we mean by "ephemeral environments" in this context? It refers to an infrastructure where tests are executed on the actual target system, which is spun up on-demand and destroyed once the tests are complete.

Indeed, an integration test is valid only if it is run in an environment that closely resembles production. Ephemeral infrastructure ensures complete isolation of tests, preventing collisions between different test executions.

Today, setting up this type of architecture has become easier with the rise of Infrastructure as Code (IaC) tools like Terraform and Pulumi.

The BigTesty library was created out of the need to offer integration tests for BigQuery, Google’s Data Warehouse. In this talk, we will illustrate and demonstrate how to develop this type of framework within a data ecosystem using BigTesty.

One question remains: why was the logic implemented with Pulumi rather than Terraform? We will show in this talk that Pulumi's Automation API is a true game changer in the IaC landscape, enabling the application of complex logic perfectly suited to ephemeral environments.