Since introduced in Kubernetes v1.9, webhooks have been a key feature, making up one of the cornerstones of Kubernetes extensibility. When used right, they can allow operators to have much more control over their clusters and with tooling like Kyverno and Gatekeeper it’s easier than ever to leverage their full power. But, when misused, things can get very, very messy.

So how do we ensure our webhooks are full of wonders and not woes?

By taking a look at the history of webhooks in Kubernetes, the driving force behind their adoption and through several horror stories of webhooks gone wrong, we can develop a set of best practices.