CodeBuild —The Missing Link

Werner Vogels re:Invent
AWS CTO Werner Vogels introduces CodeBuild during his keynote at Re:Invent 2016

Despite the robust set of features and services AWS continues to roll out, it had never — until recently – offered a native way to build and test code. This all changed at Re:Invent 2016, with the release of CodeBuild, which is a build service that compiles source code, runs tests and produces ready-to-deploy software packages on AWS.

Here’s why I find this so exciting:

The problem

When I think of DevOps and continuous integration/continuous delivery (CI/CD) tools, I logically categorize them into four major groups: versioning, building, testing and deployment.

For some time now, AWS has offered a suite of tools for CI/CD automation. For versioning, they offer a managed Git service called CodeCommit. For deployment, they offer Code Deploy.

You’ll notice I skipped from versioning to deployment, which is because it wasn’t until last month that AWS offered a tool for building and testing code.

Previously, AWS offered the ability to loop in third-party software for these activities. Through the use of CodePipeline, which is an AWS DevOps orchestration tool, users can integrate tools like Jenkins, TeamCity or Salano CI to build and test their code.

The basic process for using a third party is to ship artifacts from the repository to the software, which is hosted elsewhere. After the code has been built and tested, it’s then shipped back to CodeDeploy via direct artifacts or an S3 bucket.

While this approach gives us the ability to automate building, testing and deployment, it comes with some pitfalls.

The first is the requirement of a host for your third-party software. This may be in the form of a hosted service, an EC2 instance or possibly a container.

If you choose the hosted service, then you deal with the security issues of shipping your code out of your environment. If you choose the EC2 or container route, you’re stuck building, configuring and maintaining a build server on your own.

The second pitfall of using a third party for compilation is the added cost. Hosted build services can cost anywhere from nothing to thousands of dollars per month for the enterprise. The costs incurred are usually subscription-based and you pay whether you’re using the service or not.ave unanswered questions about AWS that you’d love to have expert help with?

Have unanswered questions about AWS? Check out our webinar:

5 Common AWS Questions Answered by a Certified Expert 

For self-hosting on an EC2 or container, you also pay for running time that you may not be using. One way to circumvent this pitfall is to only run the EC2 instance or container when needed. However, you’re then required to hand roll a pipeline that starts your instance or container when needed and shuts them down when complete.

The final pitfall — and this is probably more of a personal pet peeve — is the lack of a serverless solution to build and test code. As a developer, I’ve moved most of my projects to hosting the client on S3 and running Lambdas, API Gateway and DynamoDB as my backend. All of these technologies are serverless in nature and are maintained infrastructure as code. To have to spin up a box to test my code, which will live in a serverless word, seems a bit counterproductive.

The solution

With the release of CodeBuild, AWS has provided a native solution that solves for all of those pitfalls. CodeBuild allows you to build and test code without leaving the security of your AWS environment. The basic usage of CodeBuild is very straightforward: if you can do it from a command line on your machine, you can do it in CodeBuild.

Let’s dig in and see how it works. The first thing required for CodeBuild is a buildspec.yml document in the root of your project. A buildspec file tells CodeBuild what to do when the project is run.

A simple buildspec file will look like this:

The buildspec file is broken into several phases: install, pre_build, build, post_build and artifacts. The labels are pretty self-explanatory.

In this example, during the “install” phase, I am loading my dependencies (which are mocha) globally and then all the node dependencies.

In the“pre_build” phase, I run my tests to ensure I haven’t broken anything.

In the “build” phase I then compile all my resources for deployment and the “post_build” echoes out an all complete.

The last phase, “artifacts,” tells CodeBuild which files to return to the pipeline. In this case, I’m grabbing all files under the “dist” folder. This build process is pretty specific to the node app I’m running.

However, the beauty of CodeBuild is that you can customize these phases in any way you like. It might be necessary for you to build your code and then run your tests in the “post_build” phase. No problem, just move commands around as you like.

To truly understand CodeBuild, it’s important to understand what is going on under the hood. When a CodeBuild project runs, it actually provisions a pre-selected Docker image to run your commands.

Now I know you might be saying to yourself, “wait a minute, this is not serverless”. But the reality is, it’s as serverless as any other serverless product in the fact that YOU don’t have to maintain the server.

AWS provides a large set of pre-configured images to choose from, supporting many different languages. The list, as of this writing is:

  • Golang
  • Java
  • Node
  • Ruby
  • Python
  • Android
  • Docker

Another option is to create your own Docker container with everything you need installed already. This is a good option if there is a lot of configuration needed to build and test your application.

By pre-configuring your container, your build time will be reduced. However, you’re then required to maintain that image. In most cases, I tend to lean toward a standard AWS configured container and install everything I need each time.

With node and NPM, this doesn’t take too long and allows me to keep that information in a package file that’s easily modified.

Microsoft users, you will notice there is not a container for .Net. Don’t be alarmed, Microsoft support is coming! At the time of this writing, AWS is collecting emails for further notification on email support.

I believe CodeBuild is the missing link in the AWS DevOps tooling offering.

With the addition of CodeBuild, it’s now possible to create enterprise level deployment pipelines for many different workflows without relying on external third parties. It will be interesting to see what features are added in the form of monitoring and feedback from the build process. But for the first level offering, CodeBuild looks very promising.

Visit Rackspace to find out more about CodeBuild and the other features and services our AWS experts can help you with.

And for more information about AWS best practices, download our whitepaper “Best Practices for AWS Architectural Design.”

  • Was this article helpful ?
  • Yes   No
Eric Johnson is the AWS Evangelist at Rackspace. He has been working with AWS technologies as a developer and an architect for the last six years. His passion for all things new in cloud technologies drive him to be a lifetime student and fanatical speaker on all things cloud.

5 COMMENTS

  1. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

  2. This blog giving the details of technology. This gives the details about working with the business processes and change the way. Here explains think different and work different then provide the better output. Thanks for this blog.

  3. I am looking for a job that entails the following, stocking, Sales, Inventory control, Shipping & receiving. I Have 22ys experience in all the subjects.

LEAVE A REPLY