Cloud computing is dramatically increasing the pace of IT innovation, which has been accelerated by open source software. Rather than solving the same problems as others behind closed doors, the open source model allows you to collaborate with and benefit from other people that have the same needs as you. We have built the Rackspace Cloud on many well-known software projects like Linux, as well as smaller projects to meet specific needs. We also give back to the community through contributions to a number of projects including the recently released OpenStack.
With this rapid evolution of both hardware and software, the expectations we place on our databases have changed significantly. Web applications have a different set of needs from a database than an enterprise client-server application, and alternative database technologies have sprung up to meet this need. This doesn’t mean your web application should automatically use one of the new database technologies, but it certainly means you shouldn’t keep using the same database you have for the last twenty years without considering your options. The hardware we run our applications on today is also dramatically different – your smartphone has a faster processor and more memory than a server did when many databases were originally authored. Data sets are now measured in terabytes and petabytes for many applications.
If you’re still running an application with a smaller data set measured in the 10s or maybe even the 100s of gigabytes, you should probably run database software that is easy to configure, maintain, and that has a rich ecosystem of tools around it. On the other hand, if you have a big data problem it is time to analyze some of the new solutions available as open source projects. The attached presentation provides a high level overview of a few database alternatives, but many more also exist. It also links to the write-up about Mozilla’s Test Pilot project, where they talk about the process they used to select a database that met their needs.
Please make sure you spend the time doing the analysis, as that time spent will be inversely proportionate to the amount of time you’ll spend re-working a decision made in haste – measure twice, cut once.