Li Haoyi takes us through an internal Databricks tool for continuous integration:
Runbot is a bespoke continuous integration (CI) solution developed specifically for Databricks’ needs. Originally developed in 2019, Runbot incrementally replaces our aging Jenkins infrastructure with something more performant, scalable, and user friendly for both users and maintainers of the service. This blog post will explore the motivations behind developing Runbot, the core design decisions that went into it, and how we used it to greatly improve the experience of all the developers within the Databircks engineering organization.
It doesn’t look like the tool is available externally, but it’s an interesting read and helps understand some of the “why” behind the solution.