The development lifecycle is the lifeblood of most organizations offering digital products.
Keeping developers happy is the key to keeping that lifecycle going, so they can easily code, release and test that new code. Bumper guards are also required so things don’t get out of hand — the “just ship it” or “it worked on my laptop” nightmare.
So how can you consistently provide resources for the new code, with the required controls and ability to test the code on demand? (Not forgetting that all three steps need to be executed quickly, i.e. not days or weeks).
To solve for that, the concept of Continuous Integration/Continuous Delivery, or CI/CD, was born. Instead of adhering to the traditional software development process, which can delay code releases by days or weeks, adopting CI/CD allows for merging developer code multiple times a day, then being able to automate the code testing and pushing the code to production if test passes.
To put these principles to work I set up a CI/CD pipeline using my private cloud of choice, Rackspace Private Cloud powered by OpenStack, and currently the most popular automation server, Jenkins. Felt like a match made in heaven to me.
Below are some cliff notes on:
- how to get the Jenkins server up
- how to connect your new project to your code repository
- configure the project deployment options
Getting Jenkins running
The process of installing Jenkins on an instance running on my private cloud lab was the easiest part of this whole process.
One word of advice is to make sure the Jenkins instance can be connected to from outside of your private cloud by associating a floating-ip from a provider network to the instance. Make sure the provider network allows for external ingress traffic, to allow GitHub to execute the Jenkins web hook when the repository is updated.
From the new instance created on your private cloud to host Jenkins, execute the following commands:
sudo wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add - sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list' sudo apt-get update sudo apt-get install jenkins
Once installed, you can test if Jenkins is running by navigating to instance IP address on port 8080. Next, determine which plugins you want to install/enable. Make sure to include the GitHub Plugin and the OpenStack Cloud Plugin in that list.
Create your new project
In this example, we’ll be using GitHub as the code repository for our my-php-api app. From the Jenkins server dashboard, click on the New Item button and when prompted enter the projects name and select Freestyle project. Once the project is created you need to configure its settings.
Here are the key areas that should be updated; do not forget to save the changes before exiting:
- GitHub project: add the URL to your GitHub repository (i.e. https://github.com/wbentley15/my-php-api/)
- Source Code Management: set to Git, repository url (i.e. https://github.com/wbentley15/my-php-api.git) and branch specifier is */master
- Build Triggers: set to Build when a change is pushed to GitHub
- Build Environment: set to OpenStack Instance Creation (please note this function works only with v2 of the Keystone service)
- Build: add build step of Execute shell and add your build/test commands here (i.e. ./script/deploy and ./script/test)
Now that this is set, you must go back to your GitHub repo and configure it to call your Jenkins server when the repo is updated (aka new or updated code is committed). From the repo, click on Settings and then click on Integrations & services. Click on Add service and select the Jenkins (GitHub plugin). Add the following as the Jenkins hook url:
http://<jenkins server public IP>:8080/github-webhook/
Test application details and project deployment
In order to make this a true test, I borrowed some simple code on how to create a PHP powered API and a landing page to view the API results. The my-php-api app code can be found at the following repo: https://github.com/wbentley15/my-php-api.
Please feel free to borrow it from me as well. At this point we should focus on the two scripts inside of the script directory of the repo. This is where there are two scripts used by Jenkins to first build/deploy the code and then test if the API comes online.
Again, these are just examples of a general approach to CI/CD with Jenkins. The application build, deployment and test steps will vary per each specific application.
The deploy file within the script directory is what Jenkins will execute when a build is kicked off (remember we configured our project above to call this file). So now is the time I must come clean on something.
After spending about a half day on configuring the connection string from my Jenkins server to my OpenStack cloud, I came to realize that the OpenStack Cloud Plugin only currently works with v2 of the Keystone API. More details can be found here. A fix is in the works on this. As I am running RPC-O v13, which is the OpenStack Mitaka release and uses v3 of the Keystone API, this left me sort of high and dry.
In the perfect world the ‘deploy’ file should look like this:
#!/bin/sh ssh ubuntu@OPENSTACK_PUBLIC_IP <<EOF sudo su cd /home/ubuntu git clone https://github.com/wbentley15/my-php-api.git cp *.php /var/www/html exit EOF
Because I couldn’t get the plugin working, I had to go with a more static approach (I manually created the instance before hand and cloned the repo):
#!/bin/sh ssh email@example.com <<EOF sudo su cd /home/ubuntu/my-php-api git pull cp *.php /var/www/html exit EOF
To manually create the instance with all the dependencies I created a heat template that can be found here. I had ideas of calling the heat template directly from the project build using the controller/utility container as a proxy as an alternative approach. As you can see you can go about this in many ways. Do not be afraid to think out of the box.
The test file within the script directory is meant to provide some sort of testing for the new API updates just committed. In my case I just used a script to test if the API was active, basically returning an HTTP response code of 200. There is no limit on how far you can take your testing for the code. Just make sure the test provides a conclusive outcome as to the status of the code (success or fail).
Ship it “automagically”
Your new Jenkins project is now all set to deploy any committed code to your GitHub repo to a new (or in my case existing) instance running on your OpenStack cloud. At this point all you need to do is make a code change and commit it to GitHub. The commit will then kick off the Jenkins web hook and communicate to the Jenkins server hosted on your private cloud. Next your Jenkins project is instructed to start a build. Lastly, the project build will kick off the two scripts (deploy and test) on the instance. Automagically, your new API code has been deployed and tested while you grabbed another cup of coffee.
FYI: you can also manually kick off the build within the Jenkins dashboard by selecting your project and clicking ‘Build Now.’
After experimenting with this for a few days I can see why this is an attractive approach to integrating, deploying and testing code. It frees up some many resources while still keeping things moving. Give this example of CI/CD a try — it may seem overwhelming, but it’s not as hard as it looks!