At Rackspace, one of our core values is full disclosure and transparency. It’s not an easy value to practice when data centers go down and other things go wrong. But we’ve learned over the years that it always speeds our learning and our recovery. That’s why I was pleased to see transparency in action early this week in Washington, D.C. I was invited to the White House to get an inside look at the workings of HealthCare.gov and share ideas with our nation’s top tech staff, including Todd Park, CTO of the U.S.; Steven VanRoekel, CIO of the US; and Jeff Zients, who has been tapped to fix HealthCare.gov after its troubled start.
These officials have a huge job on their hands. HeathCare.gov has been one of the most spectacular public failures of any website ever. The site crashed. It didn’t scale. It couldn’t handle the capacity. It plainly didn’t work.
On Monday, however, I was part of a group of about a half-dozen private sector IT experts who got a behind the scenes look at exactly what’s being done to right the ship. We were brought into the White House situation room for a briefing with Chief of Staff Denis McDonough. And we took a tour of the operations center for HealthCare.gov at the offices of QSSI, the company leading the efforts to pull all of the once disparate contractors under one roof and get HealthCare.gov working.
I’ve spoken publicly about some of the technology pitfalls that may have led to the challenges faced by HealthCare.gov. This week, after much initial skepticism, I left D.C. confident that the health care site is on track to be fixed. There are daily ops reviews and stand-ups where key officials and IT experts review what’s going well and what’s not, and implement maintenance plans for the site. It’s a 24×7 operation to ensure that HealthCare.gov is up and running at full capacity by the end of the month. The call centers are staffed and more people are getting onto the site each day.
I’ve seen firsthand the hardware upgrades, software upgrades and bug fixes. I was shown where bottlenecks were found and fixed. We talked about the list of things that need to be done. I’ve seen the detailed instrumentation, the monitoring and the accountability on the team.
I also got to meet some of the private sector advisors who were brought into the fold as part of the surge to fix the site, and I heard how the federal government is looking to adjust the way it contracts jobs like HealthCare.gov to include vendors and companies that may not have been contracted for government work in the past.
It was an impressive display of transparency. And I’m confident the effort is moving in the right direction. The site is at a stage where a lot of the initial embarrassment is behind its designers and managers. The finger pointing is over — at least inside the Administration and among the contractors.
During the trip home, I thought a lot about how our customers at Rackspace who run complex websites could avoid the pitfalls that occurred with HealthCare.gov. I came up with a list of five key points:
- Testing: It is imperative to test ahead of time, and to build in enough time for usability and scalability testing. This can uncover any potential bottlenecks. There is no substitute for testing before going live – whether you’re running a national health insurance site or one for a small startup.
- Roll out slowly: Launch a website to a limited audience before going for a big bang. Beta approaches have proven effective in learning how a website can scale and what kind of load it can handle. Invite users via email, let them trickle in, then push it near capacity, increase resources and make adjustments as needed.
- Monitoring: Every website is going to have a problem at some point. A rock-solid monitoring strategy and the right tools can alert you to issues before they become big problems.
- Automation: Automation is key. Manually logging in to each server to provision your application or implement fixes is no longer effective. Automation streamlines the process and can eliminate human error. This applies to businesses of all sizes.
- Expertise: You won’t always have all the answers. It’s important to tap a trusted partner that can look at your architecture with you and help you plan. A helpful, fresh pair of eyes is essential.
I’m honored to have been invited to the White House to check out the inner workings of HealthCare.gov. The government opened the door and shined a light on the problems the site encountered and the progress it’s made. There’s still some work to do before the site is working the way that it should. But this level of transparency and insight into how they’re fixing the problems gave me a new level of confidence that the project is back on track.