OnMetal Shows Phenomenal Performance In Early Benchmarks

Since we launched the limited availability of OnMetal™ Cloud Servers – single-tenant bare-metal servers provisioned quickly via the cloud – last month, our customers have shown great interest. Some of them are looking to move away from the unpredictable nature of virtualized, multi-tenant environments, while others are intrigued by our promise of “elasticity of the cloud, plus economy of colocation” – particularly the economy side of it.

Today, OnMetal moves into general availability and we unveil its pricing.

The scalability economics of OnMetal is driven by the following variables:

  • Consistency of performance and the need for over-provisioning.
  • Cost of management.
  • Price/performance ratio: how much work you can get done for every dollar spent.

In a previous post, I addressed the consistency, over-provisioning and the management components. Here, let’s look at price per performance, since the numbers are now available.

Total monthly price for Managed Infrastructure service level:

*OnMetal billing is done by minute, though for simplicity I’m using monthly numbers here assuming there are 438,000 minutes in a month.
**This is my back-of-the-napkin approximation of where real production environments usually end up. Your results could be more or less – to learn more about the available discounts go here.

Now, let’s look at what kind of consistent performance you get for your money. We performed a number of early benchmarks to show how OnMetal performance stacks up against a competitor.

Sysbench Benchmark

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. I will let the SysBench repo site speak for itself.

The idea of this benchmark suite is to quickly get an impression of system performance without setting up complex database benchmarks or even without installing a database at all. Current features allow you to test the following system parameters:

  • file I/O performance
  • scheduler performance
  • memory allocation and transfer speed
  • POSIX threads implementation performance
  • database server performance (OLTP benchmark)

For our testing of single core performance in OnMetal, we used the single-threaded CPU benchmark, specified as follows:

Single Thread:

sysbench --num-threads=1 --test=cpu --cpu-max-prime=128000 run

We used SysBench version 0.4.12, which we installed with the distro’s package manager (apt-get install sysbench).

The SysBench CPU benchmark is designed to complete 10,000 “jobs” as fast as possible, and measure the time it takes, in seconds, to complete all 10,000. Each job consists of computing all primes from three to 128,000 using the Sieve of Eratosthenes algorithm. When running in a single thread, each job is completed one after another.

SysBench CPU can be a good indicator of the CPU performance and, to a lesser extent, of the memory I/O performance (mutex lock for shared counter, etc.) of a system.

The results show a quite noticeable improvement by about 15 percent:

This test was run on June 27, 2014.

UnixBench Benchmark

You can find more information in the UnixBench repo site for an explanation of each workload. As for our results, we reported the System Benchmark Index score, which is an aggregation of all the scores of each workload.

Except for Dhrystone and Whetstone, these workloads are heavy on syscalls. Due to Xen’s architecture, historically, it hasn’t performed that well on the UnixBench benchmark due to the overhead involved executing the hypercalls (hypervisor syscalls). Tests that involve a lot of memory page table manipulation (i.e. process creation) tend also to score pretty bad with Xen.

We used UnixBench version 5.1.3, but with a patch to allow it use more than 16 cores – most people run it this way, and it is actually a known issue.

To run it, download the tarball from the repo and run:

tar xvf UnixBench5.1.3.tar.gz 
cp fix-limitation.patch UnixBench5.1.3/
cd UnixBench5.1.3 
patch Run fix-limitation.patch

This test was run on June 27, 2014.

The OnMetal Compute flavor performed better than the AWS instance (c3.4xlarge) in both SysBench and UnixBench tests. In addition, OnMetal Compute has 10 cores versus the eight cores of AWS’ c3.4xlarge. OnMetal price per core is also much lower than AWS’ thereby delivering amazing performance for a lower price.

Pgbench Benchmark

We haven’t done much yet with our “high RAM” OnMetal flavor, but 512GB of RAM at these prices speaks for itself, I think. 😉

We did, however, play with the PostgreSQL benchmark on the High I/O flavor and compare it to the highest performing virtual server available on our own cloud – Performance Cloud Server 2-120. We used the standard pgbench tool.

We’ve used software raid – md raid0 on the OnMetal flavor, and the rest of the configuration was done as stated here:


All filesystems were ext4, and the pgbench itself was run from an onmetal compute v1 using Debian Jessie, running Linux kernel 3.14 using standard PostgreSQL 9.3 as packaged by Debian.

The results are below. As you can see, even without any special tuning the OnMetal instance delivers nearly three times as much performance as the Rackspace equivalent virtualized offering:

This test was run on June 26, 2014.

These results prove that the performance impact of virtualization is hard to ignore, especially if you run large-scale applications.

We plan to perform more sophisticated benchmarking very soon, and we’ll publish the results here. One particular area I’m interested in is web application performance measured as an aggregate of the three most common layers: request handling, caching and database all tied together.

Stay tuned!

  • Joshua Kugler

    Are the prices quoted above for managed OnMetal instances or unmanaged?

    • AndrewRHickey

      Thanks for the question, Josh.

      Those prices are with the Managed Infrastructure service level. We have changed our service level structure recently and, technically speaking, we do not offer unmanaged accounts anymore, but this service level is closer to our old “unmanaged” offering.


  • Justizin

    This is compared to KVM, yes?

    • AndrewRHickey

      Thanks for the question, Justizin.

      No, both instances we’ve been comparing to are running on Xen. KVM should do better on CPU overhead though.


  • Gary Brown

    Assuming 30-day months, there are 30 x 1440 = 43,200 minutes in a month. How did you get 438,000 minutes? Multiplied by the 10 cores in the compute server?

  • imfine

    This is absolutely true. Any company that spends 20k / month on hardware is wasting money if they are virtualized. Virtual servers don’t scale cost effectively and cause bottlenecks. My own benchmarking has shown AWS is 6-10x more costly than a dedicated setup once you take into account AWS network latency issues and EBS penalties.

  • Nice Ev!

  • Some questions/comments:
    1. Were you using HVM virtualization on the c3.4xlarge? I regularly get single-thread UnixBench scores around 1750.

    2. It’s somewhat disingenuous to only show a single-threaded UnixBench and compare the 4-core c3 instance, since the c3.large is going to have the same single-threaded performance.

    3. Why don’t you show any multi-threaded CPU scores?

    • AndrewRHickey

      Hi Joe, I reached out to Ev to get answers to your questions. Here are his responses:

      “Joe, yes we’ve been using HVM. On picking a test to run: we’ve actually been running a huge variety of benchmarks, but for the purposes of this blog post we’ve picked the simplest tests a casual reader can painlessly reproduce.

      You’re right, I think we should go ahead and publish some “meatier” tests, no matter how complicated, especially workload-specific ones, like Cassandra on high-IO instances or multi-tier web benchmarks. Stay tuned!”