The federation of multiple clouds in the real world isn’t far out of reach, and through a CERN openlab research project, CERN and Rackspace are probing the possibility of true federated hybrid clouds built on OpenStack.
At OpenStack Summit Hong Kong, I shared the stage with Tim Bell, CERN’s manager of infrastructure, to provide an update on the CERN openlab research project that is digging deeper into cloud federation and hybrid clouds.
To see more on how CERN and Rackspace are working together, check out this video:
Currently, CERN is working with Rackspace to develop solutions to connect multiple OpenStack clouds and examining how a hybrid cloud can help fuel its research into the origins of the universe.
CERN currently operates its infrastructure on a decade-old grid system, which manages dozens of data centers used to collect and process 30 or more petabytes of data per year, creating “big” data management challenges. Some are managed by CERN directly, and others are managed by research institutions that “time-share” computing resource with CERN for use on analyzing research data. CERN is currently migrating its own data centers to an OpenStack environment, and some partner facilities on the grid are doing the same. While these OpenStack clouds can interoperate with the grid system, they can’t really begin to simplify management until these siloed, independent OpenStack clouds are able to communicate with each other.
With CERN, we’re looking into how we take those similar, yet independent clouds and make them a viable replacement for the grid system. Success would mean demonstration of federated identity and aggregated services between Rackspace Public Cloud, CERN’s own private cloud and the Rackspace Private Cloud at CERN (all of which run OpenStack).
View the full presentation:
We’re working to develop a reference architecture for OpenStack cloud federation, which can then lead to blueprints, code contributions and presentations on our findings.
To get started, we deployed a 20-node Rackspace Private Cloud at CERN in parallel with CERN’s private cloud to investigate the federation between the two in areas like identity, image management, metering and federated service catalogs. We want to demonstrate how we can burst workloads from the private clouds to public clouds.
This will help us understand the true power of hybrid clouds, which until now have been largely single-site or multi-site with little integration. This will help us build use cases for future hybrid distributed infrastructures.
Our priorities are to federate identity across clouds, creating the ability to log into and manage multiple cloud services with a single set of credentials using Keystone. We’re also working toward aggregated services to create a services catalog which includes resources across multiple clouds using Keystone, and build out image management and portability across multiple clouds using Glance.
Beyond these areas, future research areas for federation include enhancing compute services, improving usage data, building automated workload management and more.
The goal of this project is to ultimately demonstrate interoperability between once-disparate cloud environments and learn how we can apply these principles in the real world to build truly federated hybrid clouds with OpenStack. For CERN, this will enable them to continue and further their fundamental physics research into things like matter and anti-matter. And for OpenStack, this should further mature the stack, and extend how users can leverage it for the enterprise and complex use cases of tomorrow.