We are living in the age of Big Data – there is so much information around us at any given moment that it is impossible to manually sift through and interpret it all. My team at Rackspace is responsible for the system engineering of our Cloud Databases product, and we collect 200,000 metrics every minute to understand the overall health of our system. Data visualization is more than a buzzword to me, it is vital to my team to do our job effectively.
As your business relies more heavily on cloud applications, you want to begin measuring things. For the business, this could be looking at incoming customers, number of products sold or churn. For technical folks, you might want to know the total traffic through your system, the utilization of different servers or the total bandwidth consumed. Regardless, as soon as you have a cloud application you want to start collecting data.
But how do you interpret massive amounts of data that is actionable? Our team built a dashboard that pulls our metrics across our entire infrastructure that allows us in real-time to evaluate our service and respond appropriately. As you look to make important system or business decisions, the adage of a picture is worth 1,000 words rings true.
Regardless of your data needs and sophistication as a developer, there are several options for you to visualize your data.
1. Third Party Vendor
There are several third party vendors who can help the basic user visualize data. Companies like Chart.io or Geckoboard can help you immediately begin to create valuable charts to display your data. All you have to do is simply push your data to a particular place and provide that vendor with the endpoint to pull that information. This can help users with visualizing their trends without having to write any code. Some developers may not choose this option because you are limited to the data interpretation by what the vendors offer.
2. Graphite Server
For users who don’t want to ship their data to a third party, a common solution is creating a Graphite server. Graphite is an open source program that stores time-series data, which then allows a user to render specific graphs on demand. You should set up a separate server from your application with a Graphite installation depending on how much data is sent as it could be disk intensive. The Graphite server has a web front end that allows developers/engineers to set up their graphs, which allows a certain amount of customization. If you wish to customize more depth you will need to use the Render URL API that is built in.
3. Customized Graphs
If you want to make a unique set of data visualization charts, the best way to do this is to customize your Graphite server. Using specialized libraries such as D3.js, SVG or Rickshaw, you can create a custom dashboard that is unique for your business for your users to consume.
If you are just starting with some of the data visualization tools, I would suggest the following tips:
- Start by graphing the basics such as looking at business trends or operational utilization.
- Use line graphs, as those are the graphs that most people understand.
- Give your leadership access so that they can see the trends of your business in real time.
- Use a dark background for those people who are constantly monitoring the charts, as it is easier on the eyes.