Rackspace Quality Engineering: Unlocking the Potential of OpenStack

Rackspace OpenStack Private Cloud isn’t just another private cloud offering. One of the many critical differences that distinguish Rackspace from our competitors is that we take an engineering approach to quality.

Our quality engineering team works tirelessly to maximize the quality of the design, code and documentation of our private cloud offering, and to eliminate defects as early as possible in the software development lifecycle.

This is where Rackspace excels and stands above our competition.

“We want our customers to focus on their specialty and let us focus on architecting, deploying, testing and operating their Rackspace private cloud, which is our specialty,” says Melissa Kam, a software developer on the Rackspace Private Cloud team.

Unlike other organizations, our quality engineering team consists of developers and engineers who are specialists in their field. Having these experts run test functions in performance, test automation and security testing provides a center of excellence to draw from, and allows us to improve knowledge sharing and asset reuse across multiple teams.

This gives Rackspace the unique ability to write test automation and implement these tests into a CI/CD pipeline quickly and efficiently. It also provides our quality engineers with time to focus on specific testing such as exploratory/defect deep dive and security and performance tuning. This results in a thorough test strategy and one that meets Rackspace’s release goals, which cover the following questions:

  • Does it function?
  • Does it perform?
  • Is it secure?

As the need for flexible and complex integration on multiple platforms increases, there’s a growing demand for our quality engineering team to have a robust testing strategy that ensures any Rackspace OpenStack Private Cloud release has been fully tested. We never want our customers to become the testers — that can result in a negative user experience and broken trust between the customer and our team.

In a way, we serve as the voice of the customer. A quality engineer at Rackspace has to feel comfortable challenging the norm and looking beyond just the functionality of the product. To be the best, we have to look at every aspect of the product from architecture to feature additions and do everything we can to mitigate customer impact while delivering unparalleled outcomes for success.

Quality at Rackspace isn’t simply based on a quality assurance organization — it comes from an engineering team that encompasses a full spectrum of software engineering practices. Most expect quality to test the functional requirements and make sure the product performs as expected.

We go beyond those expectations by determining what the product cannot do. We take the product apart and find pieces that may not live up to our standards. A performance or security issue in any part of OpenStack is a quality issue, and we’re obsessed with finding and fixing problems before the product goes to our customers.

At Rackspace, our quality leaders are responsible for the signoff of all testing. To ensure that products meet our quality standards, a leader won’t sign off on a release until all testing has been met. Here is a small sample of testing functions at a high level (not an all-inclusive list):

Functional testing:

  • Unit testing
  • Integration testing
  • System testing
  • Acceptance testing
  • Regression testing
  • Smoke testing

Non-functional testing:

  • Performance testing
  • Security testing
  • Usability testing
  • Operational readiness
  • Compatibility testing
  • Compliance testing

Most quality organizations and/or their development counterparts are familiar with conducting unit testing, integration testing, system testing and acceptance testing — collectively called functional testing, where the main goal is to ensure that the product release is evaluated, has met minimum requirements and is ready to be released. While these tests are important, many companies do not have the resources necessary to do them correctly.

In continuing our commitment to quality, Rackspace also takes an engineering approach to non-functional testing such as security design and testing, performance tuning and testing, usability testing and compatibility testing. Each of these areas can include multiple sub-tests and might require quality engineers with specialized training.

Our cloud operators and our customers need us to develop software that is trustworthy and secure. All you have to do is read the news to know security is an ongoing challenge. When security isn’t a priority, it can lead to delayed or blocked deployments, litigation, negative media coverage and customer mistrust. Those are unacceptable outcomes.

Rackspace has raised the bar in regard to what’s expected in the OpenStack community. Our test efforts minimize security and privacy-related defects in design, code and documentation. By detecting and eliminating these defects early in the software development lifecycle, we release code that is more secure and safe, giving our customers additional confidence that their clouds are not at risk.

Our security testing encompasses the development process (code review), the system or hardware it runs on, the network it is connected to and the authentication and authorization methods used to gain access to the system — both by users of the application and administrators. Essentially, security engineering is comprehensive development with security in mind.

In the OpenStack community, we take a leading role in security design and testing. Our Rackspace quality engineers developed an OpenStack API testing tool called Syntribos. This tool is the first OpenStack API security testing tool, and it received excellent reviews from the Open Web Application Security Project and OpenStack communities. By giving it to those communities, we’ve taken our security expertise to a wider audience, because we believe security is critical to success.

Another pillar of quality engineering is performance engineering. Performance testing at Rackspace is designed to do one thing: ensure that our private cloud and its environment is measured, validated and exceeds our quality requirements. We examine responsiveness, speed, scalability and stability under real world conditions, and work to eliminate any and all performance bottlenecks from the environment, application, tooling and system.

To do this, our performance engineers are trained and experienced in providing real world test scenarios, bringing added value to performance enhancement recommendations. These engineers have worked with many customers to make improvements to their cloud environments using quantitative testing such as performance, load, stress and capacity testing.

Our performance test strategy was created based on years of ongoing feedback from our customers, who we listen to and respond to as needed to provide the leading private cloud as a service experience. In addition to serving as their voice, our quality engineers become the eyes and ears of our customers and are essential to the development and delivery of our OpenStack private cloud offering.

You have many choices when it comes to a private cloud, but many of them won’t go as far as we do to continuously troubleshoot your cloud, and work to prevent downtime. At Rackspace, quality is just as important as the development, delivery and operations of our private cloud, which is why our quality engineers are part of the entire software development process.

Our quality organization doesn’t reuse upstream OpenStack Tempest tests and call it a day, either.  We develop use cases to determine and learn if we can make it do things it shouldn’t do (security and performance). We look at the errors those tests reveal and dig deeper, working hard until we can effectively say we can no longer tunnel through this breach. Our distinctive use of automation, proper use cases and test planning supports the success of our testing and ultimately our product.

We like to think of ourselves as champions for our customers. We, like you, can appreciate a great product and great Fanatical Support. The success of Rackspace OpenStack Private Cloud ultimately comes from satisfied customers and we’re here to make sure that happens.

When using Rackspace OpenStack Private Cloud, you’ll have the confidence to know that the product has been torn apart section by section and tuned to perform for your applications.

Rackspace has successfully scaled OpenStack to thousands of nodes and operates private clouds for some of the largest companies in the world. To learn more and ask questions about whether private cloud as a service might be a good fit for your organization, take advantage of a free strategy session with a private cloud expert — no strings attached. SIGN UP NOW.

Jim Freeman is the Director of Quality Engineering for Rackspace. Since joining Rackspace in 2010, he’s led the strategy and implementation for quality, security and performance testing for many Rackspace offerings including VMWare, Fanatical Support for AWS and Rackspace OpenStack Private Cloud. Jim is a member of the OpenStack Quality Group and has been influential in leading and implementing tests within the OpenStack Community. His contributions include Cloud Café, Tempest and Syntribos. Jim is experienced in building and leading organizations in software development, quality engineering, security engineering and operations. Prior to joining Rackspace, he served in the U.S. Army and held roles with Harris County and CH2M HILL. He currently resides in San Antonio. When he is not focused on work, Jim plays guitar in a local band, wakeboards and enjoys spending time with his wife, family and friends.



Please enter your comment!
Please enter your name here