Coercing JSON with malli

One of the problems with JSON is its limited selection of datatypes. For example, if you want represent timestamps in JSON, you have to encode them as strings or numbers. There’s no way to tag the specific values as timestamps, so if you’re building an application that consumes such JSON, you have to build a mechanism for coercing the strings or numbers into your programming language’s timestamp datatype.

For example, consider JSON data like this:

{"start":       "2019-01-01T00:00:00Z",
 "end":         "2019-01-31T23:59:59Z",
 "description": "The month of January",
 "tags":        ["month"]}

We’re programming in Clojure, so what we would like to actually have is this:

{:start       #inst "2019-01-01T00:00:00.000-00:00",
 :end         #inst "2019-01-31T23:59:59.000-00:00",
 :description "The month of January",
 :tags        #{:month}}

If you’re building a Clojure web backend and you’re receiving JSON via a HTTP request, your routing library probably has a nice, schema-driven way of handling the coercion. When the data is coming from some other sources, there’s usually no “built-in” solution.

I’ve seen a bunch of solutions to this problem. Sometimes the coercion code is intermingled with business logic. This is not great: it’s hard to tell where and when the data gets converted to the proper datatypes and when it’s just strings. Usually the error handling is not great, either.

A better solution would be to have an explicit coercion function and call it before handing the data to the business logic. Something like this:

(defn coerce [data]
  (-> data
      (update :start parse-timestamp)
      (update :end parse-timestamp)
      (update :tags #(into #{} (map keyword) %))))

This is a good start, but it’d be nice to make this more declarative, just like the API definitions in compojure-api and reitit. It’s not too hard to do this yourself with Schema or clojure.spec, at least if you use spec-tools. However, for the sake of novelty, I’m going to use malli.

Malli is the new data specification library by Metosin. It’s more like Schema than clojure.spec: its main use cases are data validation and transformation at the edges of the program,whereas clojure.spec is focused more on the shape of data inside the program. You can read more in malli’s README. Malli has not yet been released, but I think it’s starting to look promising.

Let’s define a schema, then:

(def Event
  [:map
   [:start inst?]
   [:end inst?]
   [:description string?]
   [:tags [:set keyword?]]])

We can now re-write our coercion function using malli:

(require '[malli.core :as m]
         '[malli.transform :as mt])

(defn coerce [data]
  (m/decode Event data mt/json-transformer))

That’s it. mt/json-transformer is a built-in transfomer that knows how to decode instants and keywords from strings and how to coerce a vector into a set. And now you have a schema for your data, which you can use for validation.

You don’t have to use Malli, but do yourself a favor and do not mix data coercion logic with business logic.