Since joining Hippo, I’ve been introduced to the world of Drone pipelines and wanted to share with you some of my observations and improvements I made to our project’s pipelines that you could implement in your day to day.
My previous experience was with Jenkins where I would build, test, and deploy applications. I was familiar with setting up pipelines using the Jenkins UI, but Drone pipelines brought a new perspective.
In this blog, I’ll provide an overview of my experiences and the impact of the improvement I made. I hope this will serve as a source of inspiration and insight for others who are looking to optimise their pipelines.
What are Drone pipelines?
“Pipelines help you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.”
Drone
When code is pushed to your repository a webhook is sent to Drone and a pipeline execution is triggered. The pipelines are configured in a drone.yml and these define what actions are to be taken when a certain trigger is hit.
Whether it be a build followed by a series of tests when a pull request is opened or a deployment to an environment when a merge is made to the master branch.
Steps
Each pipeline in the drone.yml file can contain several steps that define commands which are run in the root directory of your git repository.
The example below shows a pipeline with a step called ‘test’ which runs two commands: maven install and maven test.
Triggers
As mentioned previously, when code is pushed to your repository a webhook triggers the Drone pipelines to execute. You may want to control which pipelines execute so they only run in certain situations and to do this you can use Triggers.
The example below will trigger the pipeline when a push event is made against the master branch. The steps you might want to include in a pipeline that triggers in this way might include a build, deployment to a dev environment or running any terraform changes.
Here is another example which executes the steps when a pull request is opened against the master branch. In this situation you might want to add steps to run a build, tests and sonar checks in order to ensure the pull request is safe to merge.
Another example highlighted below is when you are wanting to promote a particular build to a new environment.
This can be done through the drone UI and providing a “target” as a parameter. The “target” value is usually the name of the environment we are wishing to deploy to. In this example the trigger is “promote” with the target “staging”.
This setup could be used to run a pipeline which deploys the available image to the staging environment.
In Practice
In my project there are many repositories, several have their own individual drone template files but many are linked to a common template. For reference the app can be deployed to Dev, Staging, Model or Prod environments.
The repository I was working on was using a set of pipelines from a common template which is described below:
- When a Pull Request is created against the master branch with build, test, sonar and terraform commands.
- Build pipeline when a push was made to the master branch with build, test, sonar, terraform commands.
- If the build pipeline was successful the image built in the previous pipeline is deployed to the Dev environment. The commands for this pipeline included a terraform apply and publish to the environment.
- A staging pipeline which is triggered when a Promote event is made with the target equal to “staging”. This includes commands for scanning for an image, applying terraform changes to the staging environment and deploying the available image to staging.
- As with the staging pipeline there are Promote pipelines for model and prod environments requiring “model” or “prod” target parameters respectively.
The Problem
In some cases it would be useful for a developer to deploy their working branch to a lower environment in order to test their code changes against some more substantial test data compared to what is available to them locally.
My team has a dev environment and it would be useful for us to be able to deploy our current working branch to this. The only way to do this would have been to merge the changes to master via a pull request and allow Drone to promote to Dev.
From my previous experience using Jenkins it was possible to select the branch you wished to promote to a lower environment through the UI and this proved useful on occasion. However there was no current setup in the Drone pipelines to achieve this and would be better practice compared to merging changes to master in order to test them.
The Solution
Having discussed with the Infrastructure team it was decided that we could achieve this by taking a copy of the existing pipeline template file, which was used by several repositories, and adding two new pipelines to the new template file.
This would mean other repositories would not be affected straight away unless they are changed to use the new template. The new pipelines are described below:
A pipeline which is triggered any time code is pushed to a branch. This will build the image and make it available for deployment.
Another pipeline with a trigger for a Promote event and the target must include “dev” as parameter. This was mostly based on the other Promote pipelines for staging, model and prod but in this case the image built when a branch is changed is pushed to the Dev environment.
The Drone template used in my repository was changed to the new version and we were then able to promote our feature branch to the Dev environment to test. This gave us a bit more flexibility to our testing as we can use better test data against our branch going forward .
In Conclusion
Having never come across Drone before, I can see the benefits of using the pipeline yaml files as it allows you to build and customise your pipelines quickly. If you are making the same change to multiple pipelines, you would not have to use the UI to navigate through them all one by one.
However, as it is done through the yaml file, it is not easy to read and would need some familiarisation on how the pipelines are defined and set up. This kind of exposure doesn’t normally occur unless you are working on the templates specifically.
The changes made to allow feature branch deployments to Dev helped improve my understanding.
