The Polyglot Pipeline
Nov 30th 2017
We create value for our customers by delivering quality products. The drawback of maintaining separate repositories for a single product is that it breaks a product down into smaller units. These units are simple to package and are independently deployable, but this freedom increases the risk of introducing a breaking change. A lower-quality product delivers less value.
The Chance to Maintain High Confidence We sought to reduce this risk by centralising the code of a recent product into a mono-repository. This promised the opportunity to test the product fully end to end as part of the packaging process or build pipeline. The chance to maintain high confidence in every code change was alluring; the challenge was to produce a single, deployable package from a multi-language repository. Hence the polyglot pipeline. The aim of our pipeline is to prove that code changes in either component continue to work together so the product works as a whole. If a code change renders the backend incompatible with the frontend, we expect an end-to-end test to fail. This causes the build pipeline to exit without producing a package. Such breaking changes are therefore not deployable. If all end-to-end tests pass, then a single deployable package is produced. This guarantees that all changes in either component are deployed in lockstep.
Our Path to the Pipeline Our path to this pipeline heavily involves docker. To avoid lengthy provisioning of our disposable CircleCI build server, we use a docker image with preinstalled dependencies for each language. We execute the build steps inside the respective containers and use volumes to mirror the build output onto the host machine. If all steps pass for each component, then the resulting output becomes an alpha version of the product against which we run our end-to-end test suite. If all tests pass, the final task of the pipeline publishes a release candidate in the form of another docker image.
Our new product doesn't exist in complete isolation. By choosing to manage a mono-repository, we excluded external service dependencies. Such services are already modelled within our ecosystem or beyond the TotallyMoney context. What to include in a mono-repository may not always be so clear-cut. We tolerate extended build times and hosting complexity to gain significant trust in every new version of our product. This ultimately lets us deliver value sooner. Image source: github  https://danluu.com/monorep  https://www.thoughtworks.com/radar/techniques/build-pipeline  https://en.wikipedia.org/wiki/Polyglot_(computing)  https://circleci.com