This is another post in the series about Cloud Monitoring and libraries we have open-sourced. In the first week I talked about Whiskey, a powerful Node.js test framework, in the second week Gary talked about the Cassandra CQL driver and in the third week I talked about node-elementtree, a Node.js library for building and parsing XML documents. This week I’ll talk about node-swiz, a Node.js library for serializing, deserializing and validating objects in RESTful APIs.
An important building block of RESTful APIs is a system for serializing/deserializing responses and validating incoming data. Many other frameworks already have a library available (e.g. django-piston for Python / Django), but when we started to work with Node.js there was no such library so we decided to build our own library called node-swiz.
Node-swiz is very powerful and flexible and can be used as a building block for very simple or complex RESTful APIs. The library itself introduces four main concepts which are described below with examples.
A Swiz definition describes how an object is serialized, deserialized and validated.
Here is an example object definition for a Server object:
Swiz struct module exposes two factory methods – Obj for an object and Field for a field.
Each object takes the following arguments:
1. Name – in our example is Server
2. An object with the following properties:
a. fields – an array of Field objects (more details below)
b. singular – singular name for this object (defaults to the object name if not provided) used when serializing lists in XML documents
c. plural – plural name for this object used when serializing lists in XML documents
Each field has a name (first argument which is passed to the factory method) and takes an object with many options as a second argument:
- src – name of the key on the object which is used as a source for the actual value. If not provided it defaults to the field name. This can also be name of a method on the object which is called when serializing it. This method gets passed in a callback and must pass a value as a second argument to this callback. You can find an example of how to do that in the Serialization section below.
- desc – field description.
- attribute – if true value will be included as an attribute instead of a tag when serializing to XML.
- coerceTo – type to coerce the value to when deserializing a payload. This is only applicable when deserializing XML.
- singular – name of a container tag when serializing a list to XML.
- plural – name of an item tag which is wrapped inside a container tag when serializing a list to XML.
- enumerated – a list of enumerations for this value.
- filterFrom – a list of targets from which this field should be excluded when serializing an object. User can pass in a target to the Swiz constructor in the options object. Below is an example which shows how to accomplish that in the Serialization section.
Serializing an object is easy and can be done in 2 steps:
1. Instantiate a Swiz object with a list of your definitions
2. Call a serialize method on the instantiated object and pass in requested format (JSON or XML) and your object
Here is an example which shows how to do that:
JSON output (serializing for target called public):
In this case public_ips attribute is not included in the serialized object, because the definition contains filterFrom option and we are serializing for target called public.
As you can see, our Server class has getSerializerType method. This is a special method which must be present if you want to serialize an object and tells Swiz which definition to use when serializing it.
And the output:
Deserializing a JSON document is the same, but instead of passing swiz.SERIALIZATION.SERIALIZATION_XML to the deserialize method you pass in swiz.SERIALIZATION.SERIALIZATION_JSON.
Imagine how cool would it be if you could use the same definitions you already use for serializing and deserializing objects to also validate user data. That’s exactly what you can do with Swiz’s component called Valve. Valve is a Swiz component for validating data and making sure it conforms to the rules defined in the object definition.
Example which shows how to validate an object:
And the output:
If the validation succeeds, Swiz passes a cleaned object to the callback as a second argument. In the first example you can also see how toBoolean() transformation function works. We passed in 1 for the active key value, but Valve casted it to a boolean.
As you can see in the second example, if the validation fails, Swiz passes back a special error object to the callback as a first argument. This object contains a detailed error message and the key which failed the validation.
How we use node-swiz in Cloud Monitoring
In Cloud Monitoring we use Swiz extensively inside the API server to deserialize and validate all the incoming data for a particular request. However, another interesting point is we use swiz and valve on a non-HTTP protocol, but over a plain TLS TCP socket. Inside our upcoming agent endpoint we use swiz/valve to validate payload of the JSON-RPC messages sent by the agent.
Serialization and deserialization
Our API supports both JSON and XML formats and we use Swiz to deserialize all the request payloads and serialize all the responses. For serializing and deserializing XML within Swiz, it uses the previously described node-elementreelibrary.
After the API request comes in and it’s parsed by Swiz we use Valve to validate the payload and make sure it conforms to all the rules specified in the definitions. If the validation fails we return a detailed error back to the user.
Automatic documentation generation
Another place where we use Swiz is for generating parts of our documentation. Every time we push to master our buildbot automatically starts a build which, among other things, generates parts of the documentation using Swiz which are then included in the main documentation file. Keeping documentation in sync with code is important but usually an overlooked thing. Using Swiz to generate parts of the documentation allows us to automate this process and makes sure that our documentation is always accurate and up to date.
Entities attributes table is automatically generated using Swiz definitions
Check type details field table is automatically generated using Swiz definitions
I hope you found this post helpful! Don’t forget to play with the library and check out the code on Github – https://github.com/racker/node-swiz. We will see you around soon when we talk about another Node.js library we have open-sourced.
If you think this kind of work would be challenging and fun, and you enjoy contributing to open-source projects, we are always looking for talented engineers.