More Transparency Helps Get On The Road To Recovery

At Rackspace, one of our core values is full disclosure and transparency. It’s not an easy value to practice when data centers go down and other things go wrong. But we’ve learned over the years that it always speeds our learning and our recovery.  That’s why I was pleased to see transparency in action early this week in Washington, D.C. I was invited to the White House to get an inside look at the workings of and share ideas with our nation’s top tech staff, including Todd Park, CTO of the U.S.; Steven VanRoekel, CIO of the US; and Jeff Zients, who has been tapped to fix after its troubled start.

These officials have a huge job on their hands. has been one of the most spectacular public failures of any website ever. The site crashed. It didn’t scale. It couldn’t handle the capacity. It plainly didn’t work.

On Monday, however, I was part of a group of about a half-dozen private sector IT experts who got a behind the scenes look at exactly what’s being done to right the ship. We were brought into the White House situation room for a briefing with Chief of Staff Denis McDonough. And we took a tour of the operations center for at the offices of QSSI, the company leading the efforts to pull all of the once disparate contractors under one roof and get working.

I’ve spoken publicly about some of the technology pitfalls that may have led to the challenges faced by This week, after much initial skepticism, I left D.C. confident that the health care site is on track to be fixed. There are daily ops reviews and stand-ups where key officials and IT experts review what’s going well and what’s not, and implement maintenance plans for the site. It’s a 24×7 operation to ensure that is up and running at full capacity by the end of the month. The call centers are staffed and more people are getting onto the site each day.

I’ve seen firsthand the hardware upgrades, software upgrades and bug fixes. I was shown where bottlenecks were found and fixed. We talked about the list of things that need to be done. I’ve seen the detailed instrumentation, the monitoring and the accountability on the team.

I also got to meet some of the private sector advisors who were brought into the fold as part of the surge to fix the site, and I heard how the federal government is looking to adjust the way it contracts jobs like to include vendors and companies that may not have been contracted for government work in the past.

It was an impressive display of transparency. And I’m confident the effort is moving in the right direction. The site is at a stage where a lot of the initial embarrassment is behind its designers and managers. The finger pointing is over — at least inside the Administration and among the contractors.

During the trip home, I thought a lot about how our customers at Rackspace who run complex websites could avoid the pitfalls that occurred with I came up with a list of five key points:

  1. Testing: It is imperative to test ahead of time, and to build in enough time for usability and scalability testing. This can uncover any potential bottlenecks. There is no substitute for testing before going live – whether you’re running a national health insurance site or one for a small startup.
  2. Roll out slowly: Launch a website to a limited audience before going for a big bang. Beta approaches have proven effective in learning how a website can scale and what kind of load it can handle. Invite users via email, let them trickle in, then push it near capacity, increase resources and make adjustments as needed.
  3. Monitoring: Every website is going to have a problem at some point. A rock-solid monitoring strategy and the right tools can alert you to issues before they become big problems.
  4. Automation: Automation is key. Manually logging in to each server to provision your application or implement fixes is no longer effective. Automation streamlines the process and can eliminate human error. This applies to businesses of all sizes.
  5. Expertise: You won’t always have all the answers. It’s important to tap a trusted partner that can look at your architecture with you and help you plan. A helpful, fresh pair of eyes is essential.

I’m honored to have been invited to the White House to check out the inner workings of The government opened the door and shined a light on the problems the site encountered and the progress it’s made. There’s still some work to do before the site is working the way that it should. But this level of transparency and insight into how they’re fixing the problems gave me a new level of confidence that the project is back on track.

John Engates joined Rackspace in August 2000, just a year after the company was founded, as Vice President of Operations, managing the datacenter operations and customer-service teams. Two years later, when Rackspace decided to add new services for larger enterprise customers, John created and helped develop the Intensive Hosting business unit. John played an active role in the evolution and evangelism of Rackspace’s cloud-computing strategy and cloud products. John met frequently with customers to hear about their needs and concerns, and to discuss Rackspace’s vision for the future of cloud computing. John's final positions was as the company’s Chief Evangelist. John is also an internationally recognized cloud computing expert and a sought-after speaker at technology conferences, including CA World, the Goldman Sachs Techtonics Conference and Cloud Expo. He speaks on the future of cloud computing, enterprise cloud adoption, data center efficiency, green data center best practices, and more. Prior to joining Rackspace, John was a founder and General Manager at Internet Direct, one of the original Internet service providers in Texas. John is a graduate of the University of Texas at San Antonio and holds a B.B.A. in Accounting.


  1. John,
    Thank you for this post and your Tesla intro;

    The present healthcare delivery system is dangerous to patients and inefficient. In his brilliant book The End of Illness, Dr. David Agus MD. talks about personal treatment and the role of data in transforming treatment from present poor results to what he labeled the “End of Illness”. We will need an obscene amount of data in the cloud that can benefit the individual and the society.
    Rackspace platform will be very important in the new health care delivery. One drop of blood requires many TB of stored data to be meaningful.
    We should help ACA and medical providers to store and share this vital info.

    Eli Levinson,
    San Antonio,


Please enter your comment!
Please enter your name here