Generating and distributing TLS certificates to secure Kafka transport between nodes is a complicated and time consuming task. It involves generating certificates, securing the keys used to generate them and creating a break glass procedure in instances of certificate leaks. In the instance of certificate leaks we need to minimize the impact by issuing certificates with a short Time To Live, which introduces operational challenges of certificate rotation.

This demo-driven talk, will show how to use HashiCorp Vault, to create an automated workflow that securely generates and distributes TLS certificates to Kafka nodes, manage the automated renewal and revocation of certificates across the platform, and dynamically generate keystore files. I’ll also show how these certificates can be signed by a Hardware Security Module. By the end of this talk, you will have learned how to implement this workflow with minimal effort, when running Kafka on bare metal, virtual machines or Kubernetes.